Method for constructing calibration plate coordinate system in camera coordinate system and hand-eye calibration method

Abstract

The invention discloses a method for constructing a calibration plate coordinate system in a camera coordinate system and a hand-eye calibration method, and the construction method comprises the steps: obtaining a three-dimensional point cloud of a calibration plate, segmenting a plane point cloud from the three-dimensional point cloud, and extracting an edge contour point cloud from the plane point cloud; performing linear parameter estimation on the edge contour point cloud to obtain a plurality of straight lines; traversing a straight line pair formed by any two straight lines; according to the coordinates of all the target intersection points in the camera coordinate system, determining the coordinates of the center point of the calibration plate in the camera coordinate system of the camera; constructing an X axis of the calibration plate coordinate system according to coordinates of two adjacent target intersection points in the camera coordinate system; according to the normal vector of the plane point cloud in the camera coordinate system, constructing a Z axis of the calibration plate coordinate system; and constructing a Y axis of the calibration plate coordinate system according to the X axis of the calibration plate coordinate system and the Z axis of the calibration plate coordinate system. According to the construction method, the calibration plate coordinate system can be accurately constructed in the camera coordinate system.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method for constructing a calibration plate coordinate system in a camera coordinate system and a hand-eye calibration method. Background Art

[0002] Constructing a calibration plate coordinate system in the camera coordinate system plays an important role in the field of robotics. The current common practice is to collect the three-dimensional point cloud of the calibration plate, and then identify the corner points of the calibration plate based on the three-dimensional point cloud, and construct the calibration plate coordinate system in the camera coordinate system based on the corner points of the calibration plate. However, in actual acquisition of the three-dimensional point cloud, there may be point cloud deviation or missing. The practice of directly constructing the calibration plate coordinate system through corner points has a low accuracy rate. Summary of the invention

[0003] The present application provides a method for constructing a calibration plate coordinate system and a hand-eye calibration method, which can accurately construct a calibration plate coordinate system in a camera coordinate system.

[0004] According to a first aspect of an embodiment of the present application, there is provided a method for constructing a calibration plate coordinate system in a camera coordinate system, the method comprising: obtaining a three-dimensional point cloud captured by a camera on a calibration plate, wherein the calibration plate is a rectangular calibration plate; segmenting a plane point cloud from the three-dimensional point cloud, and extracting an edge contour point cloud from the plane point cloud; estimating straight line parameters of the edge contour point cloud to obtain a plurality of straight lines; traversing a straight line pair formed by any two of the straight lines, and during the traversal process, in response to an angle between a direction vector corresponding to a first straight line in the straight line pair and a direction vector corresponding to a second straight line in the straight line pair being greater than a first angle threshold, determining a first intersection point of a common perpendicular line between the first straight line and the second straight line and the first straight line, and a second intersection point of the common perpendicular line and the second straight line, and dividing the first intersection point and the second intersection point into a coordinate system. The center point of the line segment is determined as the target intersection point of the first straight line and the second straight line; the coordinates of the center point of the calibration plate in the camera coordinate system are determined according to the coordinates of all the target intersection points in the camera coordinate system; the X-axis of the calibration plate coordinate system is constructed according to the coordinates of two adjacent target intersection points in the camera coordinate system; the Z-axis of the calibration plate coordinate system is constructed according to the normal vector of the plane point cloud in the camera coordinate system; the Y-axis of the calibration plate coordinate system is constructed according to the X-axis of the calibration plate coordinate system and the Z-axis of the calibration plate coordinate system; the pose description matrix of the calibration plate coordinate system in the camera coordinate system is determined according to the coordinates of the center point of the calibration plate in the camera coordinate system, the X-axis of the calibration plate coordinate system, the Y-axis of the calibration plate coordinate system and the Z-axis of the calibration plate coordinate system.

[0005] A second aspect of an embodiment of the present application provides a hand-eye calibration method, which is applied to a robot with a camera mounted on an end flange, and the method comprises: placing a calibration plate in a fixed position, wherein the calibration plate is a rectangular calibration plate; teaching the robot to move to N observation postures in succession, and obtaining a posture description matrix of the flange coordinate system in the base coordinate system when the robot is in the i-th observation posture , where N is greater than or equal to 3, and i is an integer from 1 to N; determining the posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system according to the three-dimensional point cloud of the calibration plate collected by the camera when the robot is in the i-th observation posture , wherein the pose description matrix of the calibration plate coordinate system in the camera coordinate system is determined by using any of the above methods ; According to all the posture description matrices And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0006] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a communication circuit, wherein the processor is coupled to the memory and the communication circuit, respectively, and program data is stored in the memory. The processor implements the steps in the above method by executing the program data in the memory.

[0007] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the above method.

[0008] The beneficial effect is: the method of the present application takes into account that in the process of three-dimensional point cloud acquisition in actual scenes, some point cloud deviations or omissions may exist. If the calibration plate coordinate system is directly constructed by identifying corner points, it will cause large errors. Therefore, the present application replaces the corner points with the approximate intersection points of the edge straight lines, which can ensure the accuracy of establishing the calibration plate coordinate system in the camera coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 It is a flowchart of an implementation method of constructing a calibration plate coordinate system in a camera coordinate system according to the present application; Figure 2 It is a schematic diagram of the camera coordinate system and the calibration plate coordinate system; Figure 3 It is a flowchart of an implementation method of the hand-eye calibration method of the present application; Figure 4 It is a flowchart of another implementation method of the hand-eye calibration method of the present application; Figure 5 It is a structural schematic diagram of an embodiment of the electronic device of the present application; Figure 6 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0011] It should be noted that the terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0012] First of all, it should be noted that this application involves the base coordinate system of the robot, the flange coordinate system of the robot, the camera coordinate system of the camera, and the calibration plate coordinate system of the calibration plate. Refers to a fixed coordinate system defined on the robot base, flange coordinate system Refers to a fixed coordinate system defined on the end flange of the robot, the camera coordinate system Refers to the fixed coordinate system defined inside the camera, used to describe the coordinates of the three-dimensional point cloud, the calibration plate coordinate system Refers to a fixed coordinate system defined at the center of the calibration plate.

[0013] In this application, the transformation matrix Representing coordinate system In the coordinate system The pose transformation in, for example, the matrix Indicates the calibration plate coordinate system In the camera coordinate system The pose description in .

[0014] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a method for constructing a calibration plate coordinate system in a camera coordinate system of the present application, the method comprising: S110: Obtain a three-dimensional point cloud of the calibration plate photographed by the camera.

[0015] Among them, the calibration plate is a rectangular calibration plate.

[0016] Specifically, after the calibration plate is placed in a fixed position, the 3D camera shoots the calibration plate to obtain a 3D point cloud of the calibration plate. The 3D camera may be an infrared binocular camera.

[0017] Since there are errors in the point cloud imaging process of the 3D camera and the point cloud is obtained by discrete sampling, the point cloud of the calibration plate is usually not a continuous and complete cube, but there will be missing points. For example, there is a missing point in the upper left corner of the point cloud. It is precisely because of the errors in the point cloud data collected in the actual scene that there may be deviations or missing points at the corner points of the point cloud on the surface of the calibration plate. Therefore, in the related technology, directly constructing the calibration plate coordinate system through corner point recognition will introduce large errors. For this reason, this application proposes a new method.

[0018] In the solution of the present application, the calibration plates are all rectangular calibration plates, that is, the calibration plates are cuboid structures.

[0019] S120: Segment a plane point cloud from the three-dimensional point cloud, and extract an edge contour point cloud from the plane point cloud.

[0020] Specifically, based on the spatial plane model, the RANSAC algorithm can be used to segment the plane point cloud from the three-dimensional point cloud, and the A-shape algorithm can be further used to extract the edge contour point cloud from the plane point cloud.

[0021] In one embodiment, in order to improve the accuracy of the subsequent algorithm, before step S120, it also includes: performing at least one of downsampling and outlier filtering on the three-dimensional point cloud. Among them, downsampling the three-dimensional point cloud can reduce the number of data points in the point cloud and improve the efficiency of the subsequent algorithm, and performing outlier filtering on the three-dimensional point cloud can remove outliers, which can improve the speed of the algorithm and improve the accuracy of the algorithm.

[0022] S130: Estimating straight line parameters for the edge contour point cloud to obtain a plurality of straight lines.

[0023] Specifically, the RANSAC algorithm can be used to estimate the line parameters of the edge contour point cloud based on the spatial line model to obtain a number of lines.

[0024] It can be understood that, since the calibration plate is a rectangular calibration plate, four straight lines can be obtained in step S130.

[0025] S140: Traverse a straight line pair formed by any two straight lines. During the traversal process, in response to the angle between the direction vector corresponding to the first straight line in the straight line pair and the direction vector corresponding to the second straight line in the straight line pair being greater than a first angle threshold, determine the first intersection point of the common perpendicular line between the first straight line and the second straight line and the first straight line, and the second intersection point of the common perpendicular line and the second straight line, and determine the center point of the line segment formed by the first intersection point and the second intersection point as the target intersection point of the first straight line and the second straight line.

[0026] Specifically, a straight line pair formed by any two straight lines among a number of straight lines is traversed. In the process of traversing the straight line pair, it is determined whether the angle between the direction vectors corresponding to the first straight line and the second straight line included in the straight line pair is greater than a first angle threshold. If it is not greater than the first angle threshold, no subsequent processing is performed on the straight line pair. If it is greater than the first angle threshold, a first intersection point formed by the intersection of the common perpendicular line of the first straight line and the second straight line with the first straight line is obtained, and a second intersection point formed by the intersection of the common perpendicular line of the first straight line and the second straight line with the second straight line is obtained. Finally, the center point of the line segment formed by the first intersection point and the second intersection point is determined, and the center point is determined as the approximate intersection point between the first straight line and the second straight line, that is, the target intersection point.

[0027] The first straight line is recorded as the first straight line , and the second straight line is recorded as the second straight line , the first straight line in the camera coordinate system The corresponding direction vector is recorded as , the second straight line in the camera coordinate system The corresponding direction vector is recorded as , then determine the first straight line according to the following formula The corresponding direction vector With the second straight line The corresponding direction vector The angle between :

[0028] The first angle threshold is denoted as , so if the angle between a pair of straight lines is satisfy: , then determine the target intersection point of the straight line with the first straight line and the second straight line according to the above method.

[0029] If the first straight line , the second straight line The common perpendicular line and the first straight line The first intersection The coordinates in the camera coordinate system are , the first straight line , the second straight line The common perpendicular line and the second straight line The second intersection of The coordinates in the camera coordinate system are , then determine the first straight line according to the following formula With the second straight line The coordinates of the target intersection point P in the camera coordinate system :

[0030]

[0031]

[0032] In one embodiment, let the first target point The first straight line Any point on the second target point The second straight line For any point on is the first intersection point in the camera coordinate system The vector formed by the origin of the camera coordinate system, vector is the second intersection point in the camera coordinate system The vector formed by the origin of the camera coordinate system is determined according to the following formula And vector :

[0033]

[0034]

[0035]

[0036] in, is the first target point on the first straight line in the camera coordinate system The vector formed by the origin of the camera coordinate system, The second target point on the second straight line in the camera coordinate system The vector formed by the origin of the camera coordinate system, is the first target point in the camera coordinate system With the second target point The vector formed by It is the vector normalization operation.

[0037] In getting the vector Then, we can get the first intersection point The coordinates in the camera coordinate system are obtained by Then, we can get the second intersection point The coordinates in the camera coordinate system.

[0038] Since the calibration plate is a rectangular calibration plate, four straight lines are obtained, and four target intersection points can be obtained through step S140, and then the four target intersection points are used as four corner points of the rectangular calibration plate for subsequent steps.

[0039] In one embodiment, assuming that a straight line pair includes a straight line a and a straight line b, and the angle between the vector corresponding to the straight line a and the direction vector corresponding to the straight line b is greater than the first angle threshold, after the straight line a is used as the first straight line and the straight line b is used as the second straight line in the above formula, a target intersection point can be obtained, which is recorded as the target intersection point H. The coordinates of the target intersection point H in the camera coordinate system are: , after substituting straight line a as the second straight line and straight line b as the first straight line into the above formula, we can also get a target intersection point, which is recorded as the target intersection point M. The coordinates of the target intersection point M in the camera coordinate system are It can be understood that the target intersection point H or the target intersection point M is close or coincident, wherein the target intersection point H can be finally used as the target intersection point between the straight line a and the straight line b, or the target intersection point M can be finally used as the target intersection point between the straight line a and the straight line b, and the final coordinates of the target intersection point between the straight line a and the straight line b in the camera coordinate system can also be determined according to the target intersection point H and the target intersection point M. .

[0040] S150: Determine the coordinates of the center point of the calibration plate in the camera coordinate system according to the coordinates of all target intersection points in the camera coordinate system.

[0041] Specifically, the four target intersection points are used as the four corner points of the rectangular calibration plate, and finally the coordinates of the center point of the calibration plate in the camera coordinate system can be obtained according to the coordinates of all target intersection points in the camera coordinate system.

[0042] In one embodiment, the four target intersection points are respectively recorded as points ,point ,point And point ,point The coordinates in the camera coordinate system are ,point The coordinates in the camera coordinate system are ,point The coordinates in the camera coordinate system are ,point The coordinates in the camera coordinate system are , then determine the coordinates of the center point of the calibration plate in the camera coordinate system according to the following formula :

[0043]

[0044]

[0045] That is to say, step S150 specifically includes: calculating the average value of the X component in the coordinates of all target intersection points in the camera coordinate system to obtain the X component of the coordinates of the center point of the calibration plate in the camera coordinate system; calculating the average value of the Y component in the coordinates of all target intersection points in the camera coordinate system to obtain the Y component of the coordinates of the center point of the calibration plate in the camera coordinate system; calculating the average value of the Z component in the coordinates of all target intersection points in the camera coordinate system to obtain the Z component of the coordinates of the center point of the calibration plate in the camera coordinate system.

[0046] S160: Constructing an X-axis of the calibration plate coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system.

[0047] Specifically, a vector constructed by any two adjacent target intersection points in the camera coordinate system may be determined as a vector corresponding to the X-axis of the calibration plate coordinate system.

[0048] In one embodiment, step S160 specifically includes: in response to the angle between the target unit vector formed by two adjacent target intersection points in the camera coordinate system and the vector corresponding to the positive direction of the X-axis of the camera coordinate system being less than a second angle threshold, determining the target unit vector as the vector corresponding to the positive direction of the X-axis of the calibration plate coordinate system.

[0049] Specifically, if the angle between the target unit vector formed by two adjacent target intersection points and the positive direction of the X-axis of the camera coordinate system is less than the second angle threshold, the target unit vector is determined as the vector corresponding to the positive direction of the X-axis of the calibration plate coordinate system, that is, the positive direction of the X-axis of the calibration plate coordinate system is constrained to be consistent with the positive direction of the X-axis of the camera coordinate system, such as Figure 2 shown.

[0050] The second angle threshold can be set according to actual needs.

[0051] S170: Constructing the Z axis of the calibration plate coordinate system according to the normal vector of the plane point cloud in the camera coordinate system.

[0052] Specifically, after the plane point cloud is segmented, the normal vector of the plane point cloud can be determined, and the Z axis of the calibration plate coordinate system can be constructed according to the normal vector.

[0053] In one embodiment, step S170 specifically includes: in response to the angle between the target normal vector of the plane point cloud in the camera coordinate system and the vector corresponding to the negative direction of the Z axis of the camera coordinate system being less than a third angle threshold, determining the target normal vector as the vector corresponding to the positive direction of the Z axis of the calibration plate coordinate system.

[0054] Specifically, the third angle threshold can be set according to actual needs. The above setting can constrain the positive direction of the Z axis of the calibration plate coordinate system to be consistent with the negative direction of the Z axis of the camera coordinate system. For details, see Figure 2 .

[0055] In other implementations, the positive direction of the Z axis of the calibration plate coordinate system may also be constrained to be consistent with the positive direction of the Z axis of the camera coordinate system.

[0056] S180: Constructing the Y axis of the calibration plate coordinate system according to the X axis of the calibration plate coordinate system and the Z axis of the calibration plate coordinate system.

[0057] Specifically, after obtaining the X-axis and Z-axis of the calibration plate coordinate system, the vector corresponding to the X-axis of the calibration plate coordinate system can be cross-multiplied with the vector corresponding to the Z-axis of the calibration plate coordinate system to obtain the vector corresponding to the Y-axis of the calibration plate coordinate system, thereby obtaining the Y-axis of the calibration plate coordinate system.

[0058] S190: Determine a pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration plate in the camera coordinate system, the X axis of the calibration plate coordinate system, the Y axis of the calibration plate coordinate system, and the Z axis of the calibration plate coordinate system.

[0059] Specifically, after the above steps, we can get the unit vector corresponding to the X-axis of the calibration plate coordinate system in the camera coordinate system: , the unit vector corresponding to the Y axis of the calibration plate coordinate system is , the unit vector corresponding to the Z axis of the calibration plate coordinate system is , the coordinates of the center point of the calibration plate are , so the pose description matrix of the calibration plate coordinate system in the camera coordinate system can be determined according to the following formula :

[0060] In one embodiment, before step S190, the X-axis of the calibration plate coordinate system is also calibrated to ensure that the X-axis, Y-axis, and Z-axis of the calibration plate coordinate system are orthogonal to each other. Specifically, the X-axis of the calibration plate coordinate system can be updated according to the following formula:

[0061] in, is the vector corresponding to the X-axis of the calibration plate coordinate system, is the vector corresponding to the Y axis of the calibration plate coordinates, is the vector corresponding to the Z axis of the calibration plate coordinate system.

[0062] It should be noted that, in other implementations, the X-axis of the calibration plate coordinate system may not be calibrated before step S190.

[0063] See also Figure 3 , a flow chart of an embodiment of a calibration method of the present application, the method is applied to a robot having a camera mounted on an end flange, the method comprising: S210: placing the calibration plate at a fixed position.

[0064] Among them, the calibration plate is a rectangular calibration plate.

[0065] Specifically, the calibration plate is placed at any position in the robot motion space.

[0066] S220: The teaching robot moves to N observation postures in succession, and obtains the posture description matrix of the flange coordinate system in the base coordinate system when the robot is in the i-th observation posture, where N is greater than or equal to 3, and i is an integer from 1 to N.

[0067] Specifically, the robot is manually taught to move so that it moves to N different observation positions (N≥3) in succession. At each observation position, the calibration plate is within the acquisition field of view of the camera on the robot end flange, ensuring that the camera can acquire the complete 3D point cloud data of the calibration plate each time. The value of N is usually 10~20.

[0068] Among them, when the robot is in the i-th observation posture, the posture transformation relationship of the flange coordinate system of the robot end flange relative to the robot base coordinate system is read by the robot controller: , where X, Y, and Z are the three-dimensional coordinates of the flange coordinate system relative to the base coordinate system, and A, B, and C are Euler angles, which are roll angle, pitch angle, and yaw angle, respectively. The position description matrix of the flange coordinate system in the base coordinate system can be generated .

[0069] S230: Determine a posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system according to the three-dimensional point cloud of the calibration plate collected by the camera when the robot is in the i-th observation posture.

[0070] The construction method in any of the above embodiments is used to determine the posture description matrix of the calibration plate coordinate system in the camera coordinate system when the robot is in the i-th observation posture. Among them, determine the pose description matrix of the calibration plate coordinate system in the camera coordinate system The specific process can be found in the above-mentioned related content and will not be described in detail here.

[0071] S240: Describe the matrix based on all the poses And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0072] Specifically, after obtaining all the pose description matrices And all the pose description matrices After that, the Tasi algorithm can be used to solve the equation of the form MX=XN, where X is the homogeneous matrix of the calibration pose required. Specifically, the process includes: Determine the homogeneous matrix of the calibration pose according to the following formula :

[0073]

[0074]

[0075] Wherein, j and k are both integers from 1 to N. The above is obtained by Tasi algorithm The process belongs to the prior art and will not be described in detail here.

[0076] See also Figure 4 In another embodiment, the calibration method includes: S210: placing the calibration plate at a fixed position.

[0077] S220: The teaching robot moves to N observation postures in succession, and obtains the posture description matrix of the flange coordinate system in the base coordinate system when the robot is in the i-th observation posture, where N is greater than or equal to 3, and i is an integer from 1 to N.

[0078] S230: Determine a posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system according to the three-dimensional point cloud of the calibration plate collected by the camera when the robot is in the i-th observation posture.

[0079] S240: Describe the matrix based on all the poses And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

[0080] S250: Obtain the coordinates of the center point of the calibration plate in the camera coordinate system when the robot is in the i-th observation posture .

[0081] The coordinates of the center point of the calibration plate in the camera coordinate system are determined The specific process can be found in the above-mentioned related content, which will not be repeated here.

[0082] S260: Determine the coordinates of the center point of the calibration plate in the base coordinate system when the robot is in the i-th observation posture .

[0083] Specifically, the coordinates of the center point of the calibration plate in the base coordinate system when the robot is in the i-th observation posture can be determined according to the following formula: :

[0084] S270: Based on the coordinates of the center points of all calibration plates in the base coordinate system , determine the target error.

[0085] Specifically, since the position of the calibration plate is fixed and the base of the robot is also fixed, under ideal conditions, when the robot moves to any observation position, the calculated coordinates of the center point of the calibration plate in the base coordinate system should be equal. If they are not equal, it means there is an error.

[0086] Therefore, according to all the calculated , we can get the error of the algorithm, where the error of the algorithm is caused by factors such as the randomness of the robot movement and the posture taught by humans.

[0087] In one embodiment, step S270 includes: S271: Determine the error corresponding to the i-th observation posture according to the following formula :

[0088] Specifically, It is the coordinate of the center point of the calibration plate in the base coordinate system calculated when the robot is in the first observation posture.

[0089] The meaning of the above formula is: calculate the distance between the coordinates of the center point of the calibration plate in the base coordinate system when the robot is in the i-th posture and the coordinates of the center point of the calibration plate in the base coordinate system calculated when the robot is in the first observation posture, and use this distance value as the error corresponding to the i-th observation posture .

[0090] S272: Based on all errors , determine the target error .

[0091] In one embodiment, based on all errors , calculate the error average and get the target error .

[0092] In another embodiment, all errors can also be The maximum value, minimum value, median value or mode value in is determined as the target error .

[0093] S280: Determine whether the target error is less than an error threshold.

[0094] If it is determined that the target error is less than the error threshold, step S290 is executed, otherwise step S300 is executed.

[0095] Among them, the error threshold can be set in advance according to implementation requirements.

[0096] S290: The latest calibration pose homogeneous matrix Determine the final calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system.

[0097] Specifically, if the target error is less than the error threshold, it means that the error in the algorithm process is small and can be ignored. Therefore, the current latest calibration pose homogeneous matrix is ​​directly converted to As the final calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system.

[0098] S300: The posture description matrix corresponding to the maximum error And the pose description matrix Eliminate.

[0099] Specifically, the maximum error The corresponding pose description matrix And the pose description matrix After elimination, based on the remaining pose description matrix And the remaining pose description matrix , return to step S240, that is, according to all the remaining posture description matrices And all the remaining pose description matrices , recalculate the homogeneous matrix of the camera coordinate system in the flange coordinate system , until the final target error is less than the error threshold.

[0100] It should be noted that, in other implementations, if the target error is not less than the error threshold, re-teaching may be performed, that is, returning to execute step S220.

[0101] See also Figure 5 , Figure 5 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 200 includes a processor 210, a memory 220 and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230 respectively. The memory 220 stores program data. The processor 210 implements the steps in any of the above-mentioned implementation methods by executing the program data in the memory 220. The detailed steps can be found in the above-mentioned implementations and will not be repeated here.

[0102] The electronic device 200 may be any device with algorithm processing capability, such as a computer, a mobile phone, a robot control cabinet, etc., and is not limited here.

[0103] See also Figure 6 , Figure 6 The computer-readable storage medium 400 stores a computer program 410, which can be executed by a processor to implement the steps in any of the above methods.

[0104] The computer-readable storage medium 400 may specifically be a device that can store the computer program 410, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server that stores the computer program 410. The server may send the stored computer program 410 to other devices for execution, or may also execute the stored computer program 410 by itself.

[0105] The above description is only an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for constructing a calibration plate coordinate system in a camera coordinate system, characterized in that: The method comprises: Acquire a three-dimensional point cloud of a calibration plate photographed by a camera, wherein the calibration plate is a rectangular calibration plate; Segmenting a plane point cloud from the three-dimensional point cloud, and extracting an edge contour point cloud from the plane point cloud; Estimating straight line parameters of the edge contour point cloud to obtain a plurality of straight lines; Traversing a line pair formed by any two of the straight lines, in response to the angle between a direction vector corresponding to a first straight line in the line pair and a direction vector corresponding to a second straight line in the line pair being greater than a first angle threshold, determining a first intersection point of a common perpendicular line between the first straight line and the second straight line and the first straight line, and a second intersection point of the common perpendicular line and the second straight line, and determining a center point of a line segment formed by the first intersection point and the second intersection point as a target intersection point between the first straight line and the second straight line; Determine the coordinates of the center point of the calibration plate in the camera coordinate system according to the coordinates of all the target intersection points in the camera coordinate system; Constructing the X-axis of the calibration plate coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system; Constructing the Z axis of the calibration plate coordinate system according to the normal vector of the plane point cloud in the camera coordinate system; Constructing the Y axis of the calibration plate coordinate system according to the X axis of the calibration plate coordinate system and the Z axis of the calibration plate coordinate system; Determine the pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration plate in the camera coordinate system, the X-axis of the calibration plate coordinate system, the Y-axis of the calibration plate coordinate system and the Z-axis of the calibration plate coordinate system.

2. The method according to claim 1, characterized in that The step of determining a first intersection point of a common perpendicular line between the first straight line and the second straight line and the first straight line, and a second intersection point of the common perpendicular line and the second straight line comprises: Determine the vector formed by the first intersection point and the origin of the camera coordinate system in the camera coordinate system according to the following formula: , the vector formed by the second intersection point and the origin of the camera coordinate system : According to the vector , determine the coordinates of the first intersection point in the camera coordinate system, and according to the vector , determine the coordinates of the second intersection point in the camera coordinate system; in, is the direction vector of the first straight line in the camera coordinate system, is the vector formed by the first target point on the first straight line in the camera coordinate system and the origin of the camera coordinate system, is the direction vector of the second straight line in the camera coordinate system, is a vector formed by the second target point on the second straight line in the camera coordinate system and the origin of the camera coordinate system, is the vector formed by the first target point and the second target point in the camera coordinate system, It is a vector normalization operation, the first target point is any point on the first straight line, and the second target point is any point on the second straight line.

3. The method according to claim 1, characterized in that The step of determining the coordinates of the center point of the calibration plate in the camera coordinate system of the camera according to the coordinates of all the target intersection points in the camera coordinate system comprises: Calculate the average value of the X-components of the coordinates of all the target intersection points in the camera coordinate system to obtain the X-component of the coordinates of the center point of the calibration plate in the camera coordinate system; Calculate the average value of the Y components of the coordinates of all the target intersection points in the camera coordinate system to obtain the Y component of the coordinates of the center point of the calibration plate in the camera coordinate system; The average value of the Z components of the coordinates of all the target intersection points in the camera coordinate system is calculated to obtain the Z component of the coordinates of the center point of the calibration plate in the camera coordinate system.

4. The method according to claim 1, characterized in that: The step of constructing the X-axis of the calibration plate coordinate system according to the coordinates of two adjacent target intersection points in the camera coordinate system comprises: In response to the angle between the target unit vector formed by two adjacent target intersection points in the camera coordinate system and the vector corresponding to the positive direction of the X-axis of the camera coordinate system being less than a second angle threshold, the target unit vector is determined as the vector corresponding to the positive direction of the X-axis of the calibration plate coordinate system.

5. The method according to claim 1, characterized in that The step of constructing the Z axis of the calibration plate coordinate system according to the normal vector of the plane point cloud in the camera coordinate system comprises: In response to the angle between the target normal vector of the planar point cloud in the camera coordinate system and the vector corresponding to the negative direction of the Z axis of the camera coordinate system being less than a third angle threshold, the target normal vector is determined as the vector corresponding to the positive direction of the Z axis of the calibration plate coordinate system.

6. The method according to claim 1, characterized in that Before determining the pose description matrix of the calibration plate coordinate system in the camera coordinate system according to the coordinates of the center point of the calibration plate in the camera coordinate system, the X-axis of the calibration plate coordinate system, the Y-axis of the calibration plate coordinate system, and the Z-axis of the calibration plate coordinate system, the method further includes: Update the X-axis of the calibration plate coordinate system according to the following formula: in, is the vector corresponding to the X-axis of the calibration plate coordinate system, is the vector corresponding to the Y axis of the calibration plate coordinates, is the vector corresponding to the Z axis of the calibration plate coordinate system.

7. A hand-eye calibration method, characterized in that: The method is applied to a robot having a camera mounted on an end flange, and the method comprises: Placing a calibration plate at a fixed position, wherein the calibration plate is a rectangular calibration plate; The teaching robot moves to N observation postures in succession, and obtains the posture description matrix of the flange coordinate system in the base coordinate system when the robot is in the i-th observation posture , where N is greater than or equal to 3, and i is an integer from 1 to N; Determine the posture description matrix of the calibration plate coordinate system of the calibration plate in the camera coordinate system according to the three-dimensional point cloud of the calibration plate collected by the camera when the robot is in the i-th observation posture , wherein the pose description matrix of the calibration plate coordinate system in the camera coordinate system is determined by the method described in any one of claims 1 to 6 ; According to all the pose description matrices And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system .

8. The method according to claim 7, characterized in that The method further comprises: Get the coordinates of the center point of the calibration plate in the camera coordinate system when the robot is in the i-th observation posture ; According to the following formula, the coordinates of the center point of the calibration plate in the base coordinate system are determined when the robot is in the i-th observation posture: : According to the coordinates of the center points of all the calibration plates in the base coordinate system , determine the target error; In response to the target error being less than the error threshold, the latest homogeneous matrix of the calibration pose is converted to The final calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system is determined.

9. The method according to claim 8, characterized in that The coordinates of the center points of all the calibration plates in the base coordinate system , the steps for determining the target error include: Determine the error corresponding to the i-th observation pose according to the following formula: : According to all the errors , determine the target error .

10. The method according to claim 9, characterized in that The method further comprises: In response to the target error being not less than the error threshold, the maximum error The corresponding posture description matrix And the pose description matrix Eliminate, and then based on the remaining posture description matrix And the remaining pose description matrix , returns the execution according to all the pose description matrices And all the pose description matrices , determine the calibration pose homogeneous matrix of the camera coordinate system in the flange coordinate system steps.

11. The method according to claim 9, characterized in that According to all the errors , determine the target error The steps include: Find all errors The target error is obtained by taking the average value of .

12. An electronic device, characterized in that: The electronic device includes a processor, a memory and a communication circuit, the processor is coupled to the memory and the communication circuit respectively, the memory stores program data, and the processor implements the steps in the method as described in any one of claims 1-11 by executing the program data in the memory.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the steps in the method according to any one of claims 1 to 11.

Images