Method for constructing calibration board coordinate system in camera coordinate system and hand-eye calibration method

Abstract

The present application discloses a method for constructing a calibration board coordinate system in a camera coordinate system and a hand-eye calibration method. The construction method includes: obtaining the three-dimensional point cloud of the calibration board, segmenting the planar point cloud from the three-dimensional point cloud, and extracting the edge contour point cloud from the planar point cloud; estimating the straight-line parameters of the edge contour point cloud to obtain several straight lines; traversing any pair of straight lines formed by two straight lines; determining the coordinates of the calibration board center point in the camera coordinate system of the camera according to the coordinates of all target intersection points in the camera coordinate system; constructing the X-axis of the calibration board coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system; constructing the Z-axis of the calibration board coordinate system according to the normal vector of the planar point cloud in the camera coordinate system; and constructing the Y-axis of the calibration board coordinate system according to the X-axis of the calibration board coordinate system and the Z-axis of the calibration board coordinate system. The construction method of the present application can accurately construct the calibration board coordinate system in the camera coordinate system.

Description

Technical Field

[0001] The present application relates to the technical field of robotics, and in particular, to a method for constructing a calibration board coordinate system in a camera coordinate system and a hand-eye calibration method. Background Art

[0002] Constructing a calibration board coordinate system in a camera coordinate system plays an important role in the field of robotics. Currently, the common practice is to collect the three-dimensional point cloud of the calibration board, then identify the corner points of the calibration board based on the three-dimensional point cloud, and construct the calibration board coordinate system in the camera coordinate system based on the corner points of the calibration board. However, there will be point cloud deviation or missing in the actual collection of the three-dimensional point cloud, and the accuracy of directly constructing the calibration board coordinate system through the corner points is relatively low. Summary of the Invention

[0003] The present application provides a method for constructing a calibration board coordinate system and a hand-eye calibration method, which can accurately construct the calibration board coordinate system in the camera coordinate system.

[0004] In a first aspect of an embodiment of the present application, a method for constructing a calibration board coordinate system in a camera coordinate system is provided. The method includes: obtaining a three-dimensional point cloud of a calibration board photographed by a camera, where the calibration board is a rectangular calibration board; segmenting a plane point cloud from the three-dimensional point cloud, and extracting edge contour point cloud from the plane point cloud; performing linear parameter estimation on the edge contour point cloud to obtain a plurality of lines; traversing any pair of lines formed by the lines, and during the traversal, in response to an angle between a direction vector corresponding to a first line in the line pair and a direction vector corresponding to a second line in the line pair being greater than a first angle threshold, determining a first intersection point of a common perpendicular of the first line and the second line and the first line, and a second intersection point of the common perpendicular and the second line, and determining a center point of a line segment formed by the first intersection point and the second intersection point as a target intersection point of the first line and the second line; determining coordinates of a center point of the calibration board in the camera coordinate system according to coordinates of all the target intersection points in the camera coordinate system; constructing an X-axis of the calibration board coordinate system according to coordinates of two adjacent target intersection points in the camera coordinate system; constructing a Z-axis of the calibration board coordinate system according to a normal vector of the plane point cloud in the camera coordinate system; constructing a Y-axis of the calibration board coordinate system according to the X-axis and the Z-axis of the calibration board coordinate system; and determining a pose description matrix of the calibration board coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration board in the camera coordinate system, the X-axis of the calibration board coordinate system, the Y-axis of the calibration board coordinate system, and the Z-axis of the calibration board coordinate system.

[0005] In a second aspect of the embodiments of the present application, a hand-eye calibration method is provided. The method is applied to a robot with a camera installed on the end flange. The method includes: placing a calibration board at a fixed position, where the calibration board is a rectangular calibration board; teaching the robot to move to N observation poses in sequence, and obtaining the pose description matrix of the flange coordinate system in the base coordinate system when the robot is in the i-th observation pose, where N is greater than or equal to 3 and i is an integer from 1 to N; respectively determining the pose description matrix of the calibration board coordinate system of the calibration board in the camera coordinate system according to the three-dimensional point cloud of the calibration board collected by the camera when the robot is in the i-th observation pose, where the method described in any one of the above is used to determine the pose description matrix of the calibration board coordinate system in the camera coordinate system; determining the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system according to all the pose description matrices and all the pose description matrices. where, respectively according to the three-dimensional point cloud of the calibration board collected by the camera when the robot is in the i-th observation pose, determine the pose description matrix of the calibration board coordinate system of the calibration board in the camera coordinate system where the method described in any one of the above is used to determine the pose description matrix of the calibration board coordinate system in the camera coordinate system ; according to all the pose description matrices and all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0006] In a third aspect of the embodiments of the present application, an electronic device is provided. The electronic device includes a processor, a memory, and a communication circuit. The processor is respectively coupled to the memory and the communication circuit. Program data is stored in the memory. The processor executes the program data in the memory to implement the steps in the above method.

[0007] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the above method.

[0008] The beneficial effect is that the method of the present application takes into account that there will be partial point cloud deviation or missing in the process of collecting three-dimensional point cloud in the actual scene. If the calibration board coordinate system is directly constructed by identifying corner points, it will cause large errors. Therefore, the present application uses the approximate intersection points of edge straight lines to replace corner points, which can ensure the accuracy of establishing the calibration board coordinate system in the camera coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings, where:

[0010] Figure 1It is a schematic flowchart of an implementation manner of the method for constructing a calibration board coordinate system in the camera coordinate system of the present application;

[0011] Figure 2 It is a schematic diagram of the camera coordinate system and the calibration board coordinate system;

[0012] Figure 3 It is a schematic flowchart of an implementation manner of the hand-eye calibration method of the present application;

[0013] Figure 4 It is a schematic flowchart of another implementation manner of the hand-eye calibration method of the present application;

[0014] Figure 5 It is a schematic structural diagram of an implementation manner of the electronic device of the present application;

[0015] Figure 6 It is a schematic structural diagram of an implementation manner of the computer-readable storage medium of the present application. Specific embodiments

[0016] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0017] It should be noted that the terms "first" and "second" in the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0018] First of all, it should be noted that the present application is related to the base coordinate system of the robot, the flange coordinate system of the robot, the camera coordinate system of the camera, and the calibration board coordinate system of the calibration board. The base coordinate system of the robot refers to a fixed coordinate system defined on the robot base, and the flange coordinate system refers to a fixed coordinate system defined on the end flange of the robot, and the camera coordinate system Refers to a fixed coordinate system defined inside the camera and used to describe the coordinates of the three-dimensional point cloud, i.e., the calibration board coordinate system Refers to a fixed coordinate system defined at the center of the calibration board.

[0019] In this application, the transformation matrix represents the pose transformation of the coordinate system in the coordinate system For example, the matrix represents the pose description of the calibration board coordinate system in the camera coordinate system .

[0020] Refer to Figure 1 , Figure 1 which is a schematic flowchart of an implementation manner of a method for constructing a calibration board coordinate system in the camera coordinate system in this application. The method includes:

[0021] S110: Obtain the three-dimensional point cloud of the calibration board captured by the camera.

[0022] Wherein, the calibration board is a rectangular calibration board.

[0023] Specifically, after placing the calibration board at a fixed position, the three-dimensional camera captures the calibration board to obtain the three-dimensional point cloud of the calibration board. Among them, the three-dimensional camera can specifically be an infrared binocular camera.

[0024] Due to errors in the three-dimensional camera point cloud imaging process and the fact that the point cloud is obtained by discrete sampling, usually the point cloud of the calibration board is not a continuous and complete cube, but there will be missing parts. For example, there is missing point cloud in the upper left corner of the point cloud map. Due to the errors in the point cloud data collected in the actual scene, there may be deviations or missing parts at the corner points of the point cloud on the surface of the calibration board. Therefore, directly constructing the calibration board coordinate system through corner point recognition in the related art will introduce large errors. For this reason, this application proposes a new method.

[0025] In the solution of this application, the calibration boards are all rectangular calibration boards, that is, the calibration boards are cuboid structures.

[0026] S120: Segment the plane point cloud from the three-dimensional point cloud and extract the edge contour point cloud from the plane point cloud.

[0027] Specifically, based on the spatial plane model, the RANSAC algorithm can be used to segment the plane point cloud from the three-dimensional point cloud, and further the A-shape algorithm can be used to extract the edge contour point cloud from the plane point cloud.

[0028] In one embodiment, in order to improve the accuracy of subsequent algorithms, before step S120, it further includes: performing at least one of downsampling and outlier filtering on the three-dimensional point cloud. Among them, downsampling the three-dimensional point cloud can reduce the number of data points in the point cloud and improve the efficiency of subsequent algorithms, while performing outlier filtering on the three-dimensional point cloud can remove outliers, which can not only improve the speed of the algorithm but also improve the accuracy of the algorithm.

[0029] S130: Estimate the straight-line parameters for the edge contour point cloud to obtain a number of straight lines.

[0030] Specifically, the RANSAC algorithm can be used to estimate the straight-line parameters for the edge contour point cloud based on the spatial straight-line model to obtain a number of straight lines.

[0031] It can be understood that since the calibration board is a rectangular calibration board, four straight lines can be obtained in step S130.

[0032] S140: Traverse any pair of straight lines formed by two straight lines. During the traversal, in response to the angle between the direction vector corresponding to the first straight line in the straight-line pair and the direction vector corresponding to the second straight line in the straight-line pair being greater than the first angle threshold, determine the first intersection point of the common perpendicular of the first straight line and the second straight line and the first straight line, and the second intersection point of the common perpendicular and the second straight line, and determine the center point of the line segment formed by the first intersection point and the second intersection point as the target intersection point of the first straight line and the second straight line.

[0033] Specifically, traverse any pair of straight lines formed by two straight lines among a number of straight lines. During the traversal of the straight-line pair, determine whether the angle between the direction vectors corresponding to the first straight line and the second straight line included in the straight-line pair is greater than the first angle threshold. If it is not greater than the first angle threshold, no subsequent processing is performed on this straight-line pair. If it is greater than the first angle threshold, find the first intersection point formed by the common perpendicular of the first straight line and the second straight line and the first straight line, and the second intersection point formed by the common perpendicular of the first straight line and the second straight line and the second straight line. Finally, determine the center point of the line segment formed by the first intersection point and the second intersection point, and determine this center point as the approximate intersection point between the first straight line and the second straight line, that is, the target intersection point.

[0034] Among them, denote the first straight line as the first straight line , denote the second straight line as the second straight line , denote the direction vector corresponding to the first straight line in the camera coordinate system as , denote the direction vector corresponding to the second straight line in the camera coordinate system as , then determine the direction vector corresponding to the first straight line and the second straight line The corresponding direction vector The included angle between :

[0035]

[0036] Denote the first included angle threshold as , so if the included angle corresponding to a certain pair of straight lines satisfies: , then determine the target intersection point of the first straight line and the second straight line in the pair of straight lines according to the above method.

[0037] Among them, if the common perpendicular of the first straight line and the second straight line intersects the first straight line at the first intersection point , and the coordinate of the first intersection point in the camera coordinate system is denoted as , and the common perpendicular of the first straight line and the second straight line intersects the second straight line at the second intersection point , then determine the coordinate of the target intersection point P of the first straight line and the second straight line in the camera coordinate system according to the following formula:

[0038]

[0039]

[0040]

[0041] In an embodiment, let the first target point be any point on the first straight line , the second target point be any point on the second straight line , the vector be the vector formed by the first intersection point and the origin of the camera coordinate system in the camera coordinate system, the vector be the vector formed by the second intersection point and the origin of the camera coordinate system in the camera coordinate system, and determine the vector and the vector according to the following formula:

[0042]

[0043]

[0044]

[0045]

[0046] Among them, is the first target point on the first straight line in the camera coordinate system and the vector formed by the origin of the camera coordinate system, is the second target point on the second straight line in the camera coordinate system and the vector formed by the origin of the camera coordinate system, in the camera coordinate system, the first target point and the second target point form the vector, is the vector normalization operation.

[0047] After obtaining the vector the coordinates of the first intersection point in the camera coordinate system can be obtained. After obtaining the vector the coordinates of the second intersection point in the camera coordinate system can be obtained.

[0048] Since the calibration plate is a rectangular calibration plate, 4 straight lines will be obtained. Through step S140, four target intersection points can be obtained, and then these four target intersection points are used as the four corner points of the rectangular calibration plate for subsequent steps.

[0049] In one embodiment, assume that a certain straight line pair includes straight line a and straight line b, and the angle between the vector corresponding to straight line a and the direction vector corresponding to straight line b is greater than the first angle threshold. After substituting straight line a as the first straight line and straight line b as the second straight line into the above formula, a target intersection point can be obtained, denoted as target intersection point H, and the coordinates of target intersection point H in the camera coordinate system are After substituting straight line a as the second straight line and straight line b as the first straight line into the above formula, a target intersection point can also be obtained, denoted as target intersection point M, and the coordinates of target intersection point M in the camera coordinate system are It can be understood that target intersection point H or target intersection point M is close or coincident. Among them, target intersection point H can be finally used as the target intersection point between straight line a and straight line b, or target intersection point M can be finally used as the target intersection point between straight line a and straight line b. It can also be determined according to target intersection point H and target intersection point M that the final coordinates of the target intersection point between straight line a and straight line b in the camera coordinate system are .

[0050] S150: Determine the coordinates of the center point of the calibration plate in the camera coordinate system according to the coordinates of all target intersection points in the camera coordinate system.

[0051] Specifically, taking the four target intersection points as the four corner points of the rectangular calibration board, finally, based on the coordinates of all target intersection points in the camera coordinate system, the coordinates of the center point of the calibration board in the camera coordinate system can be obtained.

[0052] In one embodiment, the four target intersection points are respectively denoted as point , point , point and point . The coordinates of point in the camera coordinate system are . The coordinates of point in the camera coordinate system are . The coordinates of point in the camera coordinate system are . The coordinates of point in the camera coordinate system are . Then, the coordinates of the center point of the calibration board in the camera coordinate system are determined according to the following formula :

[0053]

[0054]

[0055]

[0056] That is to say, step S150 specifically includes: obtaining the average value of the X components of the coordinates of all target intersection points in the camera coordinate system to obtain the X component of the coordinates of the center point of the calibration board in the camera coordinate system; obtaining the average value of the Y components of the coordinates of all target intersection points in the camera coordinate system to obtain the Y component of the coordinates of the center point of the calibration board in the camera coordinate system; obtaining the average value of the Z components of the coordinates of all target intersection points in the camera coordinate system to obtain the Z component of the coordinates of the center point of the calibration board in the camera coordinate system.

[0057] S160: Construct the X-axis of the calibration board coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system.

[0058] Specifically, the vector constructed by any two adjacent target intersection points in the camera coordinate system can be determined as the vector corresponding to the X-axis of the calibration board coordinate system.

[0059] In one embodiment, step S160 specifically includes: in response to the angle between the target unit vector formed by two adjacent target intersection points in the camera coordinate system and the vector corresponding to the positive direction of the X-axis of the camera coordinate system being less than the second angle threshold, determining the target unit vector as the vector corresponding to the positive direction of the X-axis of the calibration board coordinate system.

[0060] Specifically, if the angle between the target unit vector formed by two adjacent target intersection points and the positive X-axis direction of the camera coordinate system is less than the second angle threshold, then this target unit vector is determined as the vector corresponding to the positive X-axis direction of the calibration board coordinate system, that is, it is constrained that the positive X-axis of the calibration board coordinate system is consistent with the positive X-axis of the camera coordinate system, as Figure 2 shown.

[0061] Among them, the second angle threshold can be set according to actual needs.

[0062] S170: Construct the Z-axis of the calibration board coordinate system according to the normal vector of the plane point cloud in the camera coordinate system.

[0063] Specifically, after segmenting the plane point cloud, the normal vector of the plane point cloud can be determined, and according to this normal vector, the Z-axis of the calibration board coordinate system can be constructed.

[0064] In one implementation, step S170 specifically includes: in response to the angle between the target normal vector of the plane point cloud in the camera coordinate system and the vector corresponding to the negative Z-axis direction of the camera coordinate system being less than the third angle threshold, determining the target normal vector as the vector corresponding to the positive Z-axis direction of the calibration board coordinate system.

[0065] Specifically, the third angle threshold can be set according to actual needs, and the above setting can constrain the positive Z-axis direction of the calibration board coordinate system to be consistent with the negative Z-axis direction of the camera coordinate system. For details, please refer to Figure 2 .

[0066] In other implementations, it can also be constrained that the positive Z-axis direction of the calibration board coordinate system is consistent with the positive Z-axis direction of the camera coordinate system.

[0067] S180: Construct the Y-axis of the calibration board coordinate system according to the X-axis of the calibration board coordinate system and the Z-axis of the calibration board coordinate system.

[0068] Specifically, after obtaining the X-axis of the calibration board coordinate system and the Z-axis of the calibration board coordinate system, the vector corresponding to the X-axis of the calibration board coordinate system can be cross-multiplied with the vector corresponding to the Z-axis of the calibration board coordinate system to obtain the vector corresponding to the Y-axis of the calibration board coordinate system, thereby obtaining the Y-axis of the calibration board coordinate system.

[0069] S190: Determine the pose description matrix of the calibration board coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration board in the camera coordinate system, the X-axis of the calibration board coordinate system, the Y-axis of the calibration board coordinate system, and the Z-axis of the calibration board coordinate system.

[0070] Specifically, after the foregoing steps, it can be obtained that in the camera coordinate system, the unit vector corresponding to the X-axis of the calibration board coordinate system is , and the unit vector corresponding to the Y-axis of the calibration board coordinate system is , the unit vector corresponding to the Z-axis of the calibration board coordinate system is , the coordinates corresponding to the center point of the calibration board are , so the pose description matrix of the calibration board coordinate system in the camera coordinate system can be determined according to the following formula :

[0071]

[0072] In one embodiment, before step S190, the X-axis of the calibration board coordinate system is also corrected to ensure that the X-axis, Y-axis, and Z-axis of the calibration board coordinate system are pairwise orthogonal. Specifically, the X-axis of the calibration board coordinate system can be updated according to the following formula:

[0073]

[0074] Among them, is the vector corresponding to the X-axis of the calibration board coordinate system, is the vector corresponding to the Y-axis of the calibration board coordinate, is the vector corresponding to the Z-axis of the calibration board coordinate system.

[0075] It should be noted that in other embodiments, the X-axis of the calibration board coordinate system may not be corrected before step S190.

[0076] Refer to Figure 3 , in the flowchart of an embodiment of the calibration method of the present application, this method is applied to a robot with a camera installed on the end flange. This method includes:

[0077] S210: Place the calibration board at a fixed position.

[0078] Among them, the calibration board is a rectangular calibration board.

[0079] Specifically, place the calibration board at any position in the robot's motion space.

[0080] S220: Teach the robot to move to N observation poses successively, and obtain the pose description matrix of the flange coordinate system in the base coordinate system when the robot is at the i-th observation pose, where N is greater than or equal to 3, and i is an integer from 1 to N.

[0081] Specifically, manually teach the robot to move so that the robot moves to N different observation poses successively (N≥3). Among them, at each observation pose, the calibration board is within the acquisition field of view of the camera on the robot's end flange, ensuring that the camera can collect complete three-dimensional point cloud data of the calibration board each time. Among them, the common value of N is 10-20.

[0082] Among them, when the robot is in the i-th observation pose, the pose transformation relationship between the flange coordinate system of the robot end flange and the base coordinate system of the robot is read through the robot controller , where X, Y, and Z are the three-dimensional coordinates of the position of the flange coordinate system relative to the base coordinate system, and A, B, and C are all Euler angles, which are the roll angle, pitch angle, and yaw angle respectively. Through the pose transformation relationship a pose description matrix of the flange coordinate system in the base coordinate system can be generated .

[0083] S230: Respectively, according to the three-dimensional point cloud of the calibration board collected by the camera when the robot is in the i-th observation pose, determine the pose description matrix of the calibration board coordinate system of the calibration board in the camera coordinate system.

[0084] Among them, the pose description matrix of the calibration board coordinate system in the camera coordinate system when the robot is in the i-th observation pose is determined by using the construction method in any of the above embodiments . Among them, the specific process of determining the pose description matrix of the calibration board coordinate system in the camera coordinate system can be seen in the above relevant content and will not be elaborated here.

[0085] S240: According to all the pose description matrices and all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0086] Specifically, after obtaining all the pose description matrices and all the pose description matrices , the equation in the form of MX = XN can be solved by the Tasi algorithm, where X is the required calibration pose homogeneous matrix . Specifically, this process includes:[[]]

[0087] Determine the calibration pose homogeneous matrix according to the following formula :[[]]

[0088]

[0089]

[0090]

[0091] where j and k are both integers from 1 to N for i. Among them, the process of obtaining the above by the Tasi algorithm belongs to the prior art and will not be elaborated here.

[0092] Refer to Figure 4, in another embodiment, the calibration method includes:

[0093] S210: Place the calibration board at a fixed position.

[0094] S220: Teach the robot to move to N observation poses successively, and obtain the pose description matrix of the flange coordinate system in the base coordinate system when the robot is at the i-th observation pose, where N is greater than or equal to 3, and i is an integer from 1 to N.

[0095] S230: Respectively determine the pose description matrix of the calibration board coordinate system in the camera coordinate system according to the three-dimensional point cloud of the calibration board collected by the camera when the robot is at the i-th observation pose.

[0096] S240: According to all the pose description matrices and all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0097] S250: Obtain the coordinates of the center point of the calibration board in the camera coordinate system when the robot is at the i-th observation pose .

[0098] Among them, the specific process of determining the coordinates of the center point of the calibration board in the camera coordinate system can be seen in the above relevant content and will not be elaborated here.

[0099] S260: Respectively determine the coordinates of the center point of the calibration board in the base coordinate system when the robot is at the i-th observation pose .

[0100] Specifically, the coordinates of the center point of the calibration board in the base coordinate system when the robot is at the i-th observation pose can be determined according to the following formula :

[0101]

[0102] S270: According to the coordinates of the center points of all calibration boards in the base coordinate system , determine the target error.

[0103] Specifically, since the position of the calibration board is fixed and the base of the robot is also fixed, in an ideal state, when the robot moves to any observation pose, the calculated coordinates of the center point of the calibration board in the base coordinate system should be equal. If they are not equal, it means there is an error.

[0104] Therefore, according to all the calculated , the error of the algorithm can be obtained, where the error of the algorithm is caused by factors such as the randomness of the robot's movement and the pose of manual teaching.

[0105] In one embodiment, step S270 includes:

[0106] S271: Determine the error corresponding to the i-th observed pose according to the following formula :

[0107]

[0108] Specifically, is the coordinate of the center point of the calibration board calculated when the robot is in the first observed pose in the base coordinate system.

[0109] The meaning of the above formula is: calculate the distance value between the coordinate of the center point of the calibration board in the base coordinate system when the robot is in the i-th pose and the coordinate of the center point of the calibration board calculated when the robot is in the first observed pose in the base coordinate system, and take this distance value as the error corresponding to the i-th observed pose .

[0110] S272: Determine the target error according to all the errors . .

[0111] In one embodiment, according to all the errors , calculate the average value of the errors to obtain the target error .

[0112] In another embodiment, the maximum value, minimum value, median value or mode value, etc. among all the errors can also be determined as the target error .

[0113] S280: Determine whether the target error is less than the error threshold.

[0114] If it is determined that the target error is less than the error threshold, execute step S290; otherwise, execute step S300.

[0115] Among them, the error threshold can be set in advance according to the implementation requirements.

[0116] S290: Determine the latest calibrated pose homogeneous matrix as the final calibrated pose homogeneous matrix of the camera coordinate system in the flange coordinate system.

[0117] Specifically, if the target error is less than the error threshold, it means that the error in the algorithm process is small and can be ignored. Therefore, directly use the current latest calibrated pose homogeneous matrix As the final calibrated pose homogeneous matrix of the camera coordinate system in the flange coordinate system.

[0118] S300: The pose description matrix corresponding to the maximum error and the pose description matrix are removed.

[0119] Specifically, after removing the pose description matrix corresponding to the maximum error and the pose description matrix , based on the remaining pose description matrices and the remaining pose description matrices , return to execute step S240, that is, according to all the remaining pose description matrices and all the remaining pose description matrices , recalculate the calibrated pose homogeneous matrix of the camera coordinate system in the flange coordinate system until the finally obtained target error is less than the error threshold.

[0120] It should be noted that in other embodiments, if the target error is not less than the error threshold, teaching can be performed again, that is, return to execute step S220.

[0121] Refer to Figure 5 , Figure 5 is a schematic structural diagram of an embodiment of an electronic device of the present application. The electronic device 200 includes a processor 210, a memory 220, and a communication circuit 230. The processor 210 is respectively coupled to the memory 220 and the communication circuit 230. Program data is stored in the memory 220. The processor 210 realizes the steps in the method of any of the above embodiments by executing the program data in the memory 220. For the detailed steps, reference can be made to the above embodiments and will not be elaborated here.

[0122] Among them, the electronic device 200 can be any device with algorithm processing capabilities such as a computer, a mobile phone, a robot control cabinet, etc., and is not limited here.

[0123] Refer to Figure 6 , Figure 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 400 stores a computer program 410, and the computer program 410 can be executed by a processor to realize the steps in any of the above methods.

[0124] Among them, the computer-readable storage medium 400 may specifically be a device such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store the computer program 410, or it may also be a server storing the computer program 410. The server can send the stored computer program 410 to other devices for running, or it can also run the stored computer program 410 by itself.

[0125] The above are only embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall equally be included in the patent protection scope of the present application.

Claims

1. A hand-eye calibration method, characterized in that: The method is applied to a robot having a camera mounted on an end flange, and the method comprises: Placing a calibration plate at a fixed position, wherein the calibration plate is a rectangular calibration plate; The teaching robot moves to N observation postures in succession, and obtains the posture description matrix of the flange coordinate system in the base coordinate system when the robot is in the i-th observation posture , where N is greater than or equal to 3, and i is an integer from 1 to N; Determine the posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system according to the three-dimensional point cloud of the calibration plate collected by the camera when the robot is in the i-th observation posture ; According to all the pose description matrices And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system ; Wherein, the method further comprises: Get the coordinates of the center point of the calibration plate in the camera coordinate system when the robot is in the i-th observation posture ; According to the following formula, the coordinates of the center point of the calibration plate in the base coordinate system are determined when the robot is in the i-th observation posture: : According to the coordinates of the center points of all the calibration plates in the base coordinate system , determine the target error; In response to the target error being less than the error threshold, the latest homogeneous matrix of the calibration pose is converted to Determining the final calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system; Wherein, the coordinates of the center points of all the calibration plates in the base coordinate system are , the steps for determining the target error include: Determine the error corresponding to the i-th observation pose according to the following formula: : According to all the errors , determine the target error ; Wherein, the method further comprises: In response to the target error being not less than the error threshold, the maximum error The corresponding posture description matrix And the pose description matrix Eliminate, and then based on the remaining posture description matrix And the remaining pose description matrix , returns the execution according to all the pose description matrices And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system steps.

2. The method according to claim 1, characterized in that: According to all the errors , determine the target error The steps include: Find all errors The target error is obtained by taking the average value of .

3. The method according to claim 1, characterized in that The following steps are used to determine the pose description matrix of the calibration plate coordinate system in the camera coordinate system: : Acquire a three-dimensional point cloud of the calibration plate photographed by a camera, wherein the calibration plate is a rectangular calibration plate; Segmenting a plane point cloud from the three-dimensional point cloud, and extracting an edge contour point cloud from the plane point cloud; Estimating straight line parameters of the edge contour point cloud to obtain a plurality of straight lines; Traversing a line pair formed by any two of the straight lines, in response to the angle between a direction vector corresponding to a first straight line in the line pair and a direction vector corresponding to a second straight line in the line pair being greater than a first angle threshold, determining a first intersection point of a common perpendicular line between the first straight line and the second straight line and the first straight line, and a second intersection point of the common perpendicular line and the second straight line, and determining a center point of a line segment formed by the first intersection point and the second intersection point as a target intersection point between the first straight line and the second straight line; Determine the coordinates of the center point of the calibration plate in the camera coordinate system according to the coordinates of all the target intersection points in the camera coordinate system; Constructing the X-axis of the calibration plate coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system; Constructing the Z axis of the calibration plate coordinate system according to the normal vector of the plane point cloud in the camera coordinate system; Constructing the Y axis of the calibration plate coordinate system according to the X axis of the calibration plate coordinate system and the Z axis of the calibration plate coordinate system; Determine the pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration plate in the camera coordinate system, the X-axis of the calibration plate coordinate system, the Y-axis of the calibration plate coordinate system and the Z-axis of the calibration plate coordinate system.

4. The method according to claim 3, characterized in that The step of determining a first intersection point of a common perpendicular line between the first straight line and the second straight line and the first straight line, and a second intersection point of the common perpendicular line and the second straight line comprises: Determine the vector formed by the first intersection point and the origin of the camera coordinate system in the camera coordinate system according to the following formula: , the vector formed by the second intersection point and the origin of the camera coordinate system : According to the vector , determine the coordinates of the first intersection point in the camera coordinate system, and according to the vector , determine the coordinates of the second intersection point in the camera coordinate system; in, is the direction vector of the first straight line in the camera coordinate system, is the vector formed by the first target point on the first straight line in the camera coordinate system and the origin of the camera coordinate system, is the direction vector of the second straight line in the camera coordinate system, is a vector formed by the second target point on the second straight line in the camera coordinate system and the origin of the camera coordinate system, is the vector formed by the first target point and the second target point in the camera coordinate system, It is a vector normalization operation, the first target point is any point on the first straight line, and the second target point is any point on the second straight line.

5. The method according to claim 3, characterized in that: The step of determining the coordinates of the center point of the calibration plate in the camera coordinate system of the camera according to the coordinates of all the target intersection points in the camera coordinate system comprises: Calculate the average value of the X-components of the coordinates of all the target intersection points in the camera coordinate system to obtain the X-component of the coordinates of the center point of the calibration plate in the camera coordinate system; Calculate the average value of the Y components of the coordinates of all the target intersection points in the camera coordinate system to obtain the Y component of the coordinates of the center point of the calibration plate in the camera coordinate system; The average value of the Z components of the coordinates of all the target intersection points in the camera coordinate system is calculated to obtain the Z component of the coordinates of the center point of the calibration plate in the camera coordinate system.

6. The method according to claim 3, characterized in that The step of constructing the X-axis of the calibration plate coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system comprises: In response to the angle between the target unit vector formed by two adjacent target intersection points in the camera coordinate system and the vector corresponding to the positive direction of the X-axis of the camera coordinate system being less than a second angle threshold, the target unit vector is determined as the vector corresponding to the positive direction of the X-axis of the calibration plate coordinate system.

7. The method according to claim 3, characterized in that The step of constructing the Z axis of the calibration plate coordinate system according to the normal vector of the plane point cloud in the camera coordinate system comprises: In response to the angle between the target normal vector of the planar point cloud in the camera coordinate system and the vector corresponding to the negative direction of the Z axis of the camera coordinate system being less than a third angle threshold, the target normal vector is determined as the vector corresponding to the positive direction of the Z axis of the calibration plate coordinate system.

8. The method according to claim 1, characterized in that Before determining the pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration plate in the camera coordinate system, the X-axis of the calibration plate coordinate system, the Y-axis of the calibration plate coordinate system, and the Z-axis of the calibration plate coordinate system, the method further includes: Update the X-axis of the calibration plate coordinate system according to the following formula: in, is the vector corresponding to the X-axis of the calibration plate coordinate system, is the vector corresponding to the Y axis of the calibration plate coordinate system, is the vector corresponding to the Z axis of the calibration plate coordinate system.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory and a communication circuit, the processor is coupled to the memory and the communication circuit respectively, the memory stores program data, and the processor implements the steps in the method as described in any one of claims 1-8 by executing the program data in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.

Images