Automatic calibration device and method based on optimal pose generation

Abstract

The invention discloses an automatic calibration device and method based on optimal pose generation, and relates to the technical field of camera calibration, and the method comprises the steps: employing an eye-in-hand calibration algorithm or an eye-to-hand calibration algorithm to obtain preposed data; a recommended calibration posture of the calibration plate under the pixel coordinate system of the to-be-calibrated camera and current parameters of the to-be-calibrated camera are obtained, the tail end position of the mechanical arm is solved and adjusted according to the recommended calibration posture and the current parameters, the to-be-calibrated camera is used for imaging the calibration plate under the recommended calibration posture, and calibration data are obtained; repeating for multiple times until a calibration data set is obtained; and carrying out filtering processing on the calibration data set based on the global geometric constraint to obtain a final data set, and carrying out camera calibration by using the final data set to obtain parameters of the to-be-calibrated camera. The method can solve the problem that the attitude of the calibration plate needs to be manually set in the calibration process, can solve the problem that the robustness is reduced due to the fact that existing calibration only depends on visual features, and improves the calibration performance.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to an automatic calibration device and method based on optimal posture generation. Background Art

[0002] Camera calibration aims to determine the intrinsic and extrinsic parameters of the camera in practical tasks such as robotics, augmented reality, and 3D reconstruction. The intrinsic parameters mainly include the focal length, optical center coordinates, distortion factors, etc. of the camera, and the extrinsic parameters describe the rotation and translation of the camera. The intrinsic and extrinsic parameters constitute the mapping relationship between the two-dimensional world of the camera and the real three-dimensional world. In theory, the camera parameter information is directly provided to users during the camera manufacturing process. However, certain errors are inevitably introduced during the production or assembly and transportation of the camera. For some applications with high precision requirements, these tiny errors will have a great impact, especially in industrial operations and medical applications, where the accuracy of camera parameters is crucial.

[0003] With the continuous development of technology, a variety of camera calibration methods have been proposed to meet different application requirements and scenarios. Using a calibration plate with a known structure to calibrate the camera is a common calibration scheme currently used in laboratories. Among them, the most widely used calibration mode is the corner feature of the chessboard, which is more robust to distortion and perspective deviation than other types of calibration modes (such as 3D objects and dots).

[0004] In industrial applications and laboratories, the most commonly used method is the traditional Zhang calibration method, which does not require the use of very high-precision three-dimensional calibration objects, but only requires a standard chessboard. The quality of the chessboard has little effect on the calibration results compared to the 3D calibration object. In the modern society where printers are everywhere, this calibration scheme greatly saves the cost of calibration, and is therefore favored by a large number of visual researchers. Zhang's calibration method solves the initialization of parameters based on the correspondence between the real world and the image, and refines these parameters by minimizing the reprojection error. The goal of calibration is to minimize the reprojection error, which assumes that the two-dimensional detection is accurate. However, there is inevitably the influence of noise in the calibration process, which conflicts with the assumption and leads to instability in parameter solution and verification. In addition, when performing calibration, users need to predict in advance whether the pose and acquisition times of the calibration plate will lead to successful calibration, which is a challenge for users without professional domain knowledge. Even researchers working in this field often find it difficult to quantify what is good calibration data.

[0005] In recent years, some researchers have tried to improve the entire calibration process based on Zhang's calibration method. For example, the document "Efficient Pose Selection for Interactive Camera Calibration, Pavel Rojtberg et al., IEEE International Symposium on Mixed and Augmented Reality, 2018, pp. 31-36" proposed a pose selection scheme that minimizes the uncertainty of calibration parameters by generating a series of optimal calibration poses and avoiding a single pose. The pose selection scheme performs accurate calibration by capturing about 9 key frames. On this basis, the document "Camera calibration with pose guidance, Yuzhuo Ren et al., ICASSP, 2021, pp. 2180-2184" proposed a calibration scheme of "pose-guided camera calibration", and developed a camera calibration system that guides users to move the checkerboard calibration plate to the optimal position. However, although the above calibration scheme has high calibration accuracy and good consistency of experimental results in the experiment, it still requires users to perform tedious alignment operations, and it is easy to introduce human errors and affect the calibration results. Summary of the invention

[0006] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides an automatic calibration device and method based on optimal posture generation. The device and method solve the problem of manual setting of the calibration plate posture during the calibration process, and solve the problem of reduced robustness caused by the existing calibration relying only on visual features. It can reduce the proportion of human intervention in the calibration process and improve the calibration performance.

[0007] The technical solution of the present invention is as follows:

[0008] In a first aspect, an automatic calibration device based on optimal posture generation is provided, the device comprising: a mechanical arm, a calibration plate and a data processing unit;

[0009] The base of the mechanical arm can be fixedly installed at a first given position, the end of the mechanical arm is used to fix one of the camera to be calibrated and the calibration plate, and the other of the camera to be calibrated and the calibration plate can be fixedly installed at a second given position;

[0010] The data processing unit can be connected to the robotic arm and the camera to be calibrated respectively, and is used for motion control of the robotic arm, for obtaining the recommended calibration pose of the calibration plate in the pixel coordinate system of the camera to be calibrated, for solving the end position of the robotic arm according to the recommended calibration pose, for performing imaging control on the camera to be calibrated to obtain calibration data, for filtering the calibration data based on global geometric constraints, and for processing the calibration data to obtain parameters of the camera to be calibrated.

[0011] In a second aspect, an automatic calibration method based on optimal posture generation is provided, the method is applied to the automatic calibration device based on optimal posture generation, the method comprising:

[0012] Step 1, according to the implementation situation, adopting the "eye-in-hand" calibration algorithm to obtain the first pre-data or adopting the "eye-to-hand" calibration algorithm to obtain the second pre-data;

[0013] The implementation includes: the camera to be calibrated is fixedly installed at the end of the robotic arm or the calibration plate is fixedly installed at the end of the robotic arm;

[0014] The first pre-set data includes: a transformation matrix between the robot base coordinate system and the calibration plate coordinate system, and a transformation matrix between the robot end coordinate system and the camera coordinate system to be calibrated; the second pre-set data includes: a transformation matrix between the robot base coordinate system and the camera coordinate system to be calibrated, and a transformation matrix between the robot end coordinate system and the calibration plate coordinate system;

[0015] Step 2: Obtain a recommended calibration pose of the calibration plate in the pixel coordinate system of the camera to be calibrated and the current parameters of the camera to be calibrated, solve and adjust the end position of the manipulator according to the recommended calibration pose and the current parameters of the camera to be calibrated, and use the camera to be calibrated to image the calibration plate in the recommended calibration pose to obtain calibration data;

[0016] Step 3, repeating step 2 until a calibration data set including a first preset amount of calibration data is obtained;

[0017] Step 4: Based on the global geometric constraints, the calibration data set is filtered to obtain a final data set, and the camera is calibrated using the final data set to obtain the parameters of the camera to be calibrated.

[0018] In some optional implementations, the step of acquiring the first pre-data by using an “eye-in-hand” calibration algorithm includes:

[0019] Move the end of the robot arm to a first position point, control the camera to be calibrated to image the calibration plate, move the end of the robot arm to a second position point, and control the camera to be calibrated to image the calibration plate;

[0020] According to the moving position information of the end of the manipulator and the imaging data of the camera to be calibrated, the second transformation matrix calculation model is solved to obtain the transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the camera to be calibrated;

[0021] According to the moving position information of the end of the manipulator, the imaging data of the camera to be calibrated, and the transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the camera to be calibrated, the first transformation matrix calculation model is solved to obtain the transformation matrix between the coordinate system of the manipulator base and the coordinate system of the calibration plate;

[0022] The first transformation matrix calculation model expression is:

[0023]

[0024] in, Represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system, a represents the position point index, a=1,2, It represents the transformation matrix from the coordinate system of the end of the robot arm to the coordinate system of the base of the robot arm when the end of the robot arm is at the ath position. It represents the transformation matrix from the camera coordinate system to be calibrated to the robot arm end coordinate system when the robot arm end is at the ath position. Represents the transformation matrix from the calibration plate coordinate system to the camera coordinate system to be calibrated when the end of the robotic arm is at the ath position;

[0025] The second transformation matrix calculation model expression is:

[0026]

[0027] in, express The inverse operation of Represents the transformation matrix from the camera coordinate system to be calibrated to the end coordinate system of the robotic arm. express The inverse operation of .

[0028] In some optional implementations, the step of acquiring the second pre-data by using an “eye-to-hand” calibration method includes:

[0029] Move the end of the robotic arm to a third position point, control the camera to be calibrated to image the calibration plate, move the end of the robotic arm to a fourth position point, and control the camera to be calibrated to image the calibration plate;

[0030] According to the moving position information of the end of the manipulator and the imaging data of the camera to be calibrated, the fourth transformation matrix calculation model is solved to obtain the transformation matrix between the manipulator base coordinate system and the camera coordinate system to be calibrated;

[0031] According to the moving position information of the end of the manipulator, the imaging data of the camera to be calibrated, and the transformation matrix between the coordinate system of the manipulator base and the coordinate system of the camera to be calibrated, the third transformation matrix calculation model is solved to obtain the transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the calibration plate;

[0032] The third transformation matrix calculation model expression is:

[0033]

[0034] in, Represents the transformation matrix from the calibration plate coordinate system to the robot end coordinate system, b represents the position point index, b = 3, 4, It represents the transformation matrix from the robot base coordinate system to the robot end coordinate system when the robot end is at the bth position point. It represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system when the end of the robot is at the bth position. Represents the transformation matrix from the calibration plate coordinate system to the camera coordinate system to be calibrated when the end of the robotic arm is at the bth position;

[0035] The fourth transformation matrix calculation model expression is:

[0036]

[0037] in, express The inverse operation of Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, express The inverse operation of .

[0038] In some optional implementations, solving the end position of the robotic arm according to the recommended calibration pose and the current parameters of the camera to be calibrated includes:

[0039] The PnP algorithm is used to calculate the recommended calibration pose and obtain the transformation matrix from the pixel coordinate system to the calibration plate coordinate system.

[0040] Using the relative pose calculation model, the transformation matrix from the pixel coordinate system to the calibration board coordinate system and the current parameters of the camera to be calibrated are calculated and processed to obtain the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system;

[0041] Using the robot end position calculation model, the transformation matrix from the camera coordinate system to be calibrated to the calibration plate coordinate system is calculated and processed to obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system;

[0042] According to the transformation matrix from the robot end coordinate system to the robot base coordinate system, the position of the robot end relative to the robot base is determined.

[0043] In some optional implementations, the relative posture calculation model expression is:

[0044]

[0045] in, Represents the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system, K -1 It represents the inverse operation of K, K represents the current intrinsic parameter matrix of the camera to be calibrated, and [R|t] represents the transformation matrix from the pixel coordinate system to the calibration plate coordinate system.

[0046] In some optional implementations, when the first pre-data is obtained in step 1, the robot arm end position calculation model expression is:

[0047]

[0048] in, Represents the transformation matrix from the robot end coordinate system to the robot base coordinate system, Represents the transformation matrix from the robot end coordinate system to the camera coordinate system to be calibrated, Represents the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system, Represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system;

[0049] When the second pre-data is obtained in step 1, the calculation model expression of the end position of the robot arm is:

[0050]

[0051] in, Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, express The inverse operation of Represents the transformation matrix from the robot end coordinate system to the calibration plate coordinate system.

[0052] In some optional implementations, when the first pre-data is obtained in step 1, filtering the calibration data set based on the global geometric constraint to obtain the final data set includes:

[0053] For each calibration data, the first transformation matrix is ​​used to solve the model, and the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to each calibration data acquisition process is calculated;

[0054] The obtained transformation matrix from the calibration plate coordinate system to the robot base coordinate system is split into a rotation matrix and a translation vector;

[0055] Calculate the mean of the rotation matrices corresponding to all calibration data to obtain the average rotation matrix;

[0056] Calculate the mean of the translation vectors corresponding to all calibration data to obtain the average translation vector;

[0057] Using the data filtering judgment model, the average rotation matrix, the average translation vector, the rotation matrix and the translation vector corresponding to the calibration data are processed to obtain the filtering judgment result;

[0058] According to the filtering judgment result, the calibration data is deleted or retained;

[0059] The first transformation matrix solves the model expression as follows:

[0060]

[0061] Where i represents the calibration data index, represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system corresponding to the i-th calibration data acquisition process, represents the transformation matrix from the robot end coordinate system to the robot base coordinate system corresponding to the i-th calibration data acquisition process, Represents the transformation matrix from the camera coordinate system to be calibrated to the end coordinate system of the robotic arm. Represents the transformation matrix from the calibration plate coordinate system obtained through the i-th calibration data to the camera coordinate system to be calibrated.

[0062] In some optional implementations, when the second pre-data is obtained in step 1, filtering the calibration data set based on the global geometric constraint to obtain the final data set includes:

[0063] For each calibration data, the second transformation matrix is ​​used to solve the model, and the transformation matrix from the calibration plate coordinate system to the robot end coordinate system corresponding to each calibration data acquisition process is calculated;

[0064] The obtained transformation matrix from the calibration plate coordinate system to the robot end coordinate system is split into a rotation matrix and a translation vector;

[0065] Calculate the mean of the rotation matrices corresponding to all calibration data to obtain the average rotation matrix;

[0066] Calculate the mean of the translation vectors corresponding to all calibration data to obtain the average translation vector;

[0067] Using the data filtering judgment model, the average rotation matrix, the average translation vector, the rotation matrix and the translation vector corresponding to the calibration data are processed to obtain the filtering judgment result;

[0068] According to the filtering judgment result, the calibration data is deleted or retained;

[0069] The second transformation matrix solves the model expression as follows:

[0070]

[0071] Where i represents the calibration data index, It represents the transformation matrix from the calibration plate coordinate system to the robot end coordinate system corresponding to the i-th calibration data acquisition process, represents the transformation matrix from the robot base coordinate system to the robot end coordinate system corresponding to the i-th calibration data acquisition process, Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, Represents the transformation matrix from the calibration plate coordinate system obtained through the i-th calibration data to the camera coordinate system to be calibrated.

[0072] In some optional implementations, the data filtering judgment model expression is:

[0073]

[0074] Among them, Δt i Represents the difference between the translation vector corresponding to the i-th calibration data and the average translation vector, Represents the average value of the difference between the translation vector corresponding to all calibration data and the average translation vector, ΔR i Represents the difference between the rotation matrix corresponding to the i-th calibration data and the average rotation matrix, It represents the average of the difference between the rotation matrix corresponding to all calibration data and the average rotation matrix, and scale represents the scale factor;

[0075] When result = 1, the calibration data corresponding to the current rotation matrix and translation vector are retained;

[0076] When result = 0, the calibration data corresponding to the current rotation matrix and translation vector are deleted.

[0077] The main advantages of the technical solution of the present invention are as follows:

[0078] The automatic calibration device and method based on optimal posture generation of the present invention can solve the problem of manually setting the posture of the calibration plate during the calibration process, can solve the problem of reduced robustness caused by the existing calibration relying only on visual features, reduce the proportion of human intervention in the calibration process, and improve the calibration performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0080] Figure 1 A schematic diagram of the composition of an automatic calibration device based on optimal posture generation disclosed in an embodiment of the present invention, wherein a data processing unit is not shown;

[0081] Figure 2 A schematic diagram of another automatic calibration device based on optimal posture generation disclosed in an embodiment of the present invention, wherein a data processing unit is not shown;

[0082] Figure 3 A schematic diagram of a flow chart of an automatic calibration method based on optimal posture generation disclosed in an embodiment of the present invention;

[0083] Figure 4 A schematic diagram of the principle of an “eye-in-hand” calibration algorithm disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0085] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0086] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0087] The present invention discloses an automatic calibration device and method based on optimal posture generation, which solves the problem of manually setting the posture of the calibration plate during the calibration process, solves the problem of reduced robustness caused by existing calibration relying only on visual features, and can reduce the proportion of human intervention in the calibration process and improve calibration performance. The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0088] Embodiment 1

[0089] refer to Figure 1 , Figure 1 The following is a schematic diagram of the composition of an automatic calibration device based on optimal posture generation disclosed in an embodiment of the present invention. To solve the above technical problems, an embodiment of the present invention provides an automatic calibration device based on optimal posture generation, the device comprising: a robot arm 100, a calibration board 200 and a data processing unit;

[0090] The base of the mechanical arm 100 can be fixedly installed at a first given position, the end of the mechanical arm 100 is used to fix the camera 300 to be calibrated, and the calibration plate 200 can be fixedly installed at a second given position;

[0091] The data processing unit can be connected to the robot arm 100 and the camera to be calibrated 300 respectively, and is used for motion control of the robot arm 100, for obtaining the recommended calibration pose of the calibration plate 200 in the pixel coordinate system of the camera to be calibrated 300, for solving the end position of the robot arm according to the recommended calibration pose, for performing imaging control on the camera to be calibrated 300 to obtain calibration data, for filtering the calibration data based on global geometric constraints, and for processing the calibration data to obtain parameters of the camera to be calibrated 300.

[0092] It should be noted that the data processing unit Figure 1 Not shown.

[0093] It should be noted that the first given position and the second given position can be specifically set according to actual needs.

[0094] In the embodiment of the present invention, the calibration plate may be a checkerboard calibration plate.

[0095] In an embodiment of the present invention, the data processing unit may be a CPU; the data processing unit may be respectively connected to the above-mentioned robotic arm 100 and the camera to be calibrated 300 via a communication bus or wirelessly; the data processing unit may be provided with or connected to a memory, and the memory may be used to store data information acquired by the data processing unit and control instruction information generated; wherein the memory may be a high-speed RAM memory or a ROM memory.

[0096] The automatic calibration device based on optimal posture generation provided by the embodiment of the present invention can solve the problem of manual setting of the posture of the calibration plate during the calibration process by automatically calculating the optimal posture of the camera to be calibrated to shoot the calibration plate and using a robotic arm to complete the relative posture adjustment of the camera to be calibrated and the calibration plate. It can also solve the problem of reduced robustness caused by the existing calibration relying only on visual features, reduce the proportion of human intervention in the calibration process, and improve the calibration performance.

[0097] Embodiment 2

[0098] refer to Figure 2 , Figure 2 The following is a schematic diagram of another automatic calibration device based on optimal posture generation disclosed in an embodiment of the present invention. To solve the above technical problems, an embodiment of the present invention provides an automatic calibration device based on optimal posture generation, the device comprising: a robot arm 100, a calibration board 200 and a data processing unit;

[0099] The base of the robot arm 100 can be fixedly installed at a first given position, the calibration plate 200 is fixedly installed at the end of the robot arm 100, and the camera to be calibrated 300 can be fixedly installed at a second given position;

[0100] The data processing unit can be connected to the robot arm 100 and the camera to be calibrated 300 respectively, and is used for motion control of the robot arm 100, for obtaining the recommended calibration pose of the calibration plate 200 in the pixel coordinate system of the camera to be calibrated 300, for solving the end position of the robot arm according to the recommended calibration pose, for performing imaging control on the camera to be calibrated 300 to obtain calibration data, for filtering the calibration data based on global geometric constraints, and for processing the calibration data to obtain parameters of the camera to be calibrated 300.

[0101] It should be noted that the data processing unit Figure 2 Not shown.

[0102] It should be noted that the first given position and the second given position can be specifically set according to actual needs.

[0103] In the embodiment of the present invention, the calibration plate may be a checkerboard calibration plate.

[0104] In an embodiment of the present invention, the data processing unit may be a CPU; the data processing unit may be respectively connected to the above-mentioned robotic arm 100 and the camera to be calibrated 300 via a communication bus or wirelessly; the data processing unit may be provided with or connected to a memory, and the memory may be used to store data information acquired by the data processing unit and control instruction information generated; wherein the memory may be a high-speed RAM memory or a ROM memory.

[0105] The automatic calibration device based on optimal posture generation provided by the embodiment of the present invention can solve the problem of manual setting of the posture of the calibration plate during the calibration process by automatically calculating the optimal posture of the camera to be calibrated to shoot the calibration plate and using a robotic arm to complete the relative posture adjustment of the camera to be calibrated and the calibration plate. It can also solve the problem of reduced robustness caused by the existing calibration relying only on visual features, reduce the proportion of human intervention in the calibration process, and improve the calibration performance.

[0106] Embodiment 3

[0107] refer to Figure 3 , Figure 3 The present invention discloses a flow chart of an automatic calibration method based on optimal pose generation disclosed in an embodiment of the present invention. To solve the above technical problems, an embodiment of the present invention provides an automatic calibration method based on optimal pose generation, which can be applied to the automatic calibration device based on optimal pose generation described in Embodiments 1 and 2.

[0108] like Figure 3 As shown, the automatic calibration method based on optimal posture generation provided by the embodiment of the present invention may include the following steps 1 to 4:

[0109] Step 1, according to the implementation situation, adopting the "eye-in-hand" calibration algorithm to obtain the first pre-data or adopting the "eye-to-hand" calibration algorithm to obtain the second pre-data;

[0110] In one embodiment of the present invention, corresponding to the automatic calibration device based on optimal posture generation described in the above-mentioned embodiments one and two, the implementation includes: the camera to be calibrated is fixedly installed at the end of the robotic arm or the calibration plate is fixedly installed at the end of the robotic arm.

[0111] In one embodiment of the present invention, when the camera to be calibrated is fixedly installed at the end of the robotic arm, the "eye-in-hand" calibration algorithm is used to obtain the first pre-data. The first pre-data includes: the transformation matrix between the robotic arm base coordinate system and the calibration plate coordinate system, and the transformation matrix between the robotic arm end coordinate system and the camera to be calibrated coordinate system.

[0112] In one embodiment of the present invention, when the calibration plate is fixedly mounted at the end of the manipulator, the "eye-to-hand" calibration algorithm is used to obtain the second pre-data. The second pre-data includes: a transformation matrix between the manipulator base coordinate system and the camera coordinate system to be calibrated, and a transformation matrix between the manipulator end coordinate system and the calibration plate coordinate system.

[0113] It should be noted that the robot base coordinate system represents an artificial coordinate system fixed in the robot workspace. When the robot base is fixed in one position, the robot base coordinate system remains constant relative to the world coordinate system. The calibration plate coordinates represent an artificial coordinate system fixed to the calibration plate. The robot end coordinate system represents an artificial coordinate system fixed to the end of the robot. When the end of the robot moves, the robot end coordinate system also moves synchronously. The camera coordinate system to be calibrated represents the coordinate system of the camera to be calibrated. The Z axis of the camera coordinate system to be calibrated coincides with the optical axis of the camera to be calibrated, and the X axis and Y axis are parallel to the X axis and Y axis of the camera image, respectively.

[0114] Step 2: Obtain a recommended calibration pose of the calibration plate in the pixel coordinate system of the camera to be calibrated and the current parameters of the camera to be calibrated, solve and adjust the end position of the manipulator according to the recommended calibration pose and the current parameters of the camera to be calibrated, and use the camera to be calibrated to image the calibration plate in the recommended calibration pose to obtain calibration data;

[0115] Step 3, repeating step 2 until a calibration data set including a first preset amount of calibration data is obtained;

[0116] It should be noted that the specific value of the first preset number is set according to actual needs. The larger the value of the first preset number, the higher the cost of obtaining and processing the calibration data, and the calibration performance of the camera to be calibrated is usually higher; the smaller the value of the first preset number, the lower the cost of obtaining and processing the calibration data, and the calibration performance of the camera to be calibrated is usually lower.

[0117] Step 4: Based on the global geometric constraints, the calibration data set is filtered to obtain a final data set, and the camera is calibrated using the final data set to obtain the parameters of the camera to be calibrated.

[0118] By implementing the automated calibration method based on optimal pose generation provided by the embodiment of the present invention in the automated calibration device based on optimal pose generation described in the above-mentioned embodiment 1 or embodiment 2, the problem of manually setting the posture of the calibration plate during the calibration process can be solved, the problem of reduced robustness caused by the existing calibration relying solely on visual features can be solved, the proportion of human intervention in the calibration process can be reduced, and the calibration performance can be improved.

[0119] refer to Figure 4 , Figure 4The schematic diagram of the principle of an "eye-in-hand" calibration algorithm disclosed in an embodiment of the present invention. Further, in an optional implementation of the embodiment of the present invention, in the above step 1, the "eye-in-hand" calibration algorithm is used to obtain the first pre-data, including the following steps 101-103:

[0120] Step 101, moving the end of the robot arm to a first position point, controlling the camera to be calibrated to image the calibration plate, moving the end of the robot arm to a second position point, controlling the camera to be calibrated to image the calibration plate;

[0121] Step 102, solving a second transformation matrix calculation model according to the movement position information of the end of the manipulator and the imaging data of the camera to be calibrated, and obtaining a transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the camera to be calibrated;

[0122] Step 103, solve the first transformation matrix calculation model according to the moving position information of the end of the robot arm, the imaging data of the camera to be calibrated, and the transformation matrix between the coordinate system of the end of the robot arm and the coordinate system of the camera to be calibrated, and obtain the transformation matrix between the coordinate system of the robot arm base and the coordinate system of the calibration plate.

[0123] In an embodiment of the present invention, when the camera to be calibrated is fixedly installed at the end of the robotic arm and the calibration plate is fixedly installed at a fixed position, the relative position of the camera to be calibrated and the end of the robotic arm remains constant, and the relative position of the calibration plate and the base of the robotic arm also remains constant.

[0124] Based on the above analysis, in the embodiment of the present invention, the first transformation matrix calculation model expression is:

[0125]

[0126] The second transformation matrix calculation model expression is:

[0127]

[0128] in, Represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system, a represents the position point index, a=1,2, It represents the transformation matrix from the coordinate system of the end of the robot arm to the coordinate system of the base of the robot arm when the end of the robot arm is at the ath position. It represents the transformation matrix from the camera coordinate system to be calibrated to the robot arm end coordinate system when the robot arm end is at the ath position. It represents the transformation matrix from the calibration plate coordinate system to the camera coordinate system to be calibrated when the end of the robot arm is at the ath position. express The inverse operation of Represents the transformation matrix from the camera coordinate system to be calibrated to the end coordinate system of the robotic arm. express The inverse operation of .

[0129] It should be noted that due to and It can be obtained based on the position information of the end of the robot arm at the first position point and the second position point. and It can be obtained based on the imaging data obtained at the first position and the second position. It is equivalent to AX = XB, where is a known quantity, As a known quantity, by using the existing mathematical solution method for AX=XB to solve the second transformation matrix calculation model, X can be obtained, that is, After completing the transformation matrix from the camera coordinate system to be calibrated to the robot end coordinate system After solving the problem, the transformation matrix from the calibration plate coordinate system to the robot base coordinate system can be obtained by substituting the corresponding transformation matrix into the first transformation matrix calculation model for solution:

[0130] In the embodiment of the present invention, by adopting the above process, the first pre-data can be acquired quickly and accurately.

[0131] Further, in an optional implementation of the embodiment of the present invention, in the above step 1, the method of obtaining the second pre-data by using the "eye-to-hand" calibration method includes the following steps 111-113:

[0132] Step 111, moving the end of the robot arm to a third position point, controlling the camera to be calibrated to image the calibration plate, moving the end of the robot arm to a fourth position point, controlling the camera to be calibrated to image the calibration plate;

[0133] Step 112, solving the fourth transformation matrix calculation model according to the movement position information of the end of the manipulator and the imaging data of the camera to be calibrated, and obtaining the transformation matrix between the manipulator base coordinate system and the camera coordinate system to be calibrated;

[0134] Step 113, solve the third transformation matrix calculation model according to the moving position information of the end of the robot arm, the imaging data of the camera to be calibrated, and the transformation matrix between the coordinate system of the robot arm base and the coordinate system of the camera to be calibrated, and obtain the transformation matrix between the coordinate system of the end of the robot arm and the coordinate system of the calibration plate.

[0135] In an embodiment of the present invention, when the calibration plate is fixedly mounted at the end of the robotic arm and the camera to be calibrated is fixedly mounted at a fixed position, the relative position of the calibration plate and the end of the robotic arm remains constant, and the relative position of the camera to be calibrated and the base of the robotic arm also remains constant.

[0136] Based on the above analysis, in the embodiment of the present invention, the third transformation matrix calculation model expression is:

[0137]

[0138] The fourth transformation matrix calculation model expression is:

[0139]

[0140] in, Represents the transformation matrix from the calibration plate coordinate system to the robot end coordinate system, b represents the position point index, b = 3, 4, It represents the transformation matrix from the robot base coordinate system to the robot end coordinate system when the robot end is at the bth position point. It represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system when the end of the robot is at the bth position. It represents the transformation matrix from the calibration plate coordinate system to the camera coordinate system to be calibrated when the end of the robot arm is at the bth position. express The inverse operation of Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, express The inverse operation of .

[0141] It should be noted that due to and It can be obtained based on the position information of the end of the robot arm at the third position point and the fourth position point. and It can be obtained based on the imaging data obtained at the third and fourth positions. It is equivalent to AX = XB, where is a known quantity, As a known quantity, by using the existing mathematical solution method for AX=XB to solve the fourth transformation matrix calculation model, X can be obtained, that is, After completing the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system After solving the problem, the transformation matrix from the calibration plate coordinate system to the robot end coordinate system can be obtained by substituting the corresponding transformation matrix into the third transformation matrix calculation model for solution:

[0142] In the embodiment of the present invention, by adopting the above processing, the second pre-data can be obtained quickly and accurately.

[0143] Furthermore, in an optional implementation of the embodiment of the present invention, in the above step 2, a recommended calibration pose of the calibration plate in the pixel coordinate system of the camera to be calibrated and the current parameters of the camera to be calibrated are obtained by running the existing calibration wizard program.

[0144] Further, in an optional implementation of the embodiment of the present invention, in the above step 2, the step of solving the end position of the robot arm according to the recommended calibration pose and the current parameters of the camera to be calibrated includes the following steps 201-204:

[0145] Step 201, using the PnP (Perspective-n-Point) algorithm, the recommended calibration pose is calculated and processed to obtain a transformation matrix from the pixel coordinate system to the calibration plate coordinate system;

[0146] Step 202, using the relative posture calculation model, calculate and process the transformation matrix from the pixel coordinate system to the calibration board coordinate system and the current parameters of the camera to be calibrated, and obtain the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system;

[0147] Step 203, using the robot end position calculation model, calculate and process the transformation matrix from the camera coordinate system to be calibrated to the calibration plate coordinate system, and obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system;

[0148] Step 204 , determining the position of the end of the robot arm relative to the base of the robot arm according to the transformation matrix from the coordinate system of the end of the robot arm to the coordinate system of the base of the robot arm.

[0149] In the embodiment of the present invention, the relative posture calculation model expression is:

[0150]

[0151] in, Represents the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system, K -1 It represents the inverse operation of K, K represents the current intrinsic parameter matrix of the camera to be calibrated, and [R|t] represents the transformation matrix from the pixel coordinate system to the calibration plate coordinate system.

[0152] In the embodiment of the present invention, when the first pre-data is obtained in step 1, the calculation model expression of the robot end position is:

[0153]

[0154] in, Represents the transformation matrix from the robot end coordinate system to the robot base coordinate system, Represents the transformation matrix from the robot end coordinate system to the camera coordinate system to be calibrated, Represents the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system, Represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system.

[0155] When the second pre-data is obtained in step 1, the calculation model expression of the robot end position is:

[0156]

[0157] in, Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, express The inverse operation of Represents the transformation matrix from the robot end coordinate system to the calibration plate coordinate system.

[0158] In the embodiment of the present invention, through the above method, the relative posture adjustment of the camera to be calibrated and the calibration plate can be automatically completed according to the optimal posture of the calibration plate taken by the camera to be calibrated, thereby solving the problem of manually setting the posture of the calibration plate during the calibration process, reducing the proportion of human intervention in the calibration process, and improving the calibration performance.

[0159] Further, in an optional implementation of the embodiment of the present invention, in the above step 4, when the first pre-data is obtained in step 1, the calibration data set is filtered based on the global geometric constraint to obtain the final data set, including the following steps 401-406:

[0160] Step 401, for each calibration data, using the first transformation matrix to solve the model, and calculate the transformation matrix from the calibration plate coordinate system to the robot base coordinate system corresponding to each calibration data acquisition process;

[0161] Step 402, splitting the obtained transformation matrix from the calibration plate coordinate system to the robot base coordinate system into a rotation matrix and a translation vector;

[0162] Step 403, calculating the mean of the rotation matrices corresponding to all calibration data to obtain an average rotation matrix;

[0163] Step 404, calculating the mean of the translation vectors corresponding to all calibration data to obtain an average translation vector;

[0164] Step 405, using the data filtering judgment model, processing the average rotation matrix, the average translation vector, the rotation matrix and the translation vector corresponding to the calibration data to obtain a filtering judgment result;

[0165] Step 406, deleting or retaining the calibration data according to the filtering judgment result.

[0166] In the embodiment of the present invention, in the above step 401, the first transformation matrix solves the model expression as follows:

[0167]

[0168] Where i represents the calibration data index, represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system corresponding to the i-th calibration data acquisition process, represents the transformation matrix from the robot end coordinate system to the robot base coordinate system corresponding to the i-th calibration data acquisition process, Represents the transformation matrix from the camera coordinate system to be calibrated to the end coordinate system of the robotic arm. Represents the transformation matrix from the calibration plate coordinate system obtained through the i-th calibration data to the camera coordinate system to be calibrated.

[0169] In an embodiment of the present invention, in the above step 403, when calculating the mean of the rotation matrices corresponding to all the calibration data, the rotation matrix corresponding to each calibration data is converted into the Euler angle form, and then the average rotation angle on the XYZ axis is calculated respectively according to the rotation matrix in the Euler angle form corresponding to all the calibration data, and then the average rotation angle on the XYZ axis is converted back into a rotation matrix. The rotation matrix obtained by the conversion is the mean of the rotation matrices corresponding to all the calibration data, that is, the average rotation matrix.

[0170] In the embodiment of the present invention, in the above step 403, the average rotation matrix is ​​expressed as:

[0171]

[0172] Where R represents the average rotation matrix, Represents the average rotation angle on the XYZ axis calculated by the rotation matrix corresponding to all calibration data.

[0173] In an embodiment of the present invention, in the above step 404, when calculating the mean of the translation vectors corresponding to all the calibration data, the average translation vector components on the XYZ axes are calculated respectively according to the translation vectors corresponding to all the calibration data, and then the average translation vector components on the XYZ axes are converted back into a translation vector. The converted translation vector is the mean of the translation vectors corresponding to all the calibration data, that is, the average translation vector.

[0174] In the embodiment of the present invention, in the above step 404, the average translation vector is expressed as:

[0175]

[0176] Where t represents the average translation vector, (t x ,t y ,t z ) represents the average translation vector component on the XYZ axis, (x i ,y i ,z i ) represents the translation vector component on the XYZ axis corresponding to the i-th calibration data, N represents the total amount of calibration data, and T represents transposition.

[0177] In the embodiment of the present invention, in the above step 405, the data filtering judgment model expression is:

[0178]

[0179] Among them, Δt i Represents the difference between the translation vector corresponding to the i-th calibration data and the average translation vector, Represents the average value of the difference between the translation vector corresponding to all calibration data and the average translation vector, ΔR i Represents the difference between the rotation matrix corresponding to the i-th calibration data and the average rotation matrix, It represents the average value of the difference between the rotation matrix corresponding to all calibration data and the average rotation matrix, and scale represents the scale factor.

[0180] In the embodiment of the present invention, the difference Δt between the translation vector corresponding to the i-th calibration data and the average translation vector i Calculated by:

[0181] Δt i =‖tt i ‖;

[0182] Among them, t i Represents the translation vector corresponding to the i-th calibration data.

[0183] The average value of the difference between the translation vector corresponding to all calibration data and the average translation vector Calculated by:

[0184]

[0185] In the embodiment of the present invention, the difference ΔR between the rotation matrix corresponding to the i-th calibration data and the average rotation matrix i Calculated by:

[0186]

[0187] Among them, arccos represents the calculation of the inverse cosine trigonometric function, trace represents the trace of the matrix, and R irepresents the rotation matrix corresponding to the i-th calibration data, and T represents the transposition operation.

[0188] The average value of the difference between the rotation matrix corresponding to all calibration data and the average rotation matrix Calculated by:

[0189]

[0190] It should be noted that the proportional factor is set according to actual needs, and different proportional factors represent different filtering processes.

[0191] In the embodiment of the present invention, in the above step 406, when result=1, the calibration data corresponding to the current rotation matrix and translation vector are retained; when result=0, the calibration data corresponding to the current rotation matrix and translation vector are deleted.

[0192] The objective function of traditional camera calibration is to minimize the reprojection error, which relies on two-dimensional corner point detection. That is to say, the calibration process seeks a camera model that minimizes the reprojection error of most points. However, there is noise in the actual calibration environment. Since the noise is unknown and unexplainable, the camera model may not be accurate even if the reprojection error is small. In an embodiment of the present invention, in addition to considering the reprojection error, global geometric constraints are also considered. The obtained calibration data set is filtered by means of global geometric constraints. Specifically, by filtering the calibration data set in the above manner, it is possible to filter out calibration data that lacks stability, and obtain a better calibration data set as the final data set, thereby obtaining an optimized calibration result.

[0193] Further, in an optional implementation of the embodiment of the present invention, in the above step 4, when the second pre-data is obtained in step 1, the calibration data set is filtered based on the global geometric constraint to obtain the final data set, including the following steps 411-416:

[0194] Step 411, for each calibration data, solve the model using the second transformation matrix to calculate the transformation matrix from the calibration plate coordinate system to the robot end coordinate system corresponding to each calibration data acquisition process;

[0195] Step 412, splitting the obtained transformation matrix from the calibration plate coordinate system to the robot end coordinate system into a rotation matrix and a translation vector;

[0196] Step 413, calculating the mean of the rotation matrices corresponding to all calibration data to obtain an average rotation matrix;

[0197] Step 414, calculating the mean of the translation vectors corresponding to all calibration data to obtain an average translation vector;

[0198] Step 415, using the data filtering judgment model, processing the average rotation matrix, the average translation vector, the rotation matrix and the translation vector corresponding to the calibration data to obtain a filtering judgment result;

[0199] Step 416, delete or retain the calibration data according to the filtering judgment result.

[0200] In the embodiment of the present invention, in the above step 411, the second transformation matrix solves the model expression as follows:

[0201]

[0202] Where i represents the calibration data index, It represents the transformation matrix from the calibration plate coordinate system to the robot end coordinate system corresponding to the i-th calibration data acquisition process, represents the transformation matrix from the robot base coordinate system to the robot end coordinate system corresponding to the i-th calibration data acquisition process, Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, Represents the transformation matrix from the calibration plate coordinate system obtained through the i-th calibration data to the camera coordinate system to be calibrated.

[0203] In an embodiment of the present invention, in the above step 413, when calculating the mean of the rotation matrices corresponding to all the calibration data, the rotation matrix corresponding to each calibration data is converted into the Euler angle form, and then the average rotation angle on the XYZ axis is calculated respectively according to the rotation matrix in the Euler angle form corresponding to all the calibration data, and then the average rotation angle on the XYZ axis is converted back into a rotation matrix. The rotation matrix obtained by the conversion is the mean of the rotation matrices corresponding to all the calibration data, that is, the average rotation matrix.

[0204] In the embodiment of the present invention, in the above step 413, the average rotation matrix is ​​expressed as:

[0205]

[0206] Where R represents the average rotation matrix, Represents the average rotation angle on the XYZ axis calculated by the rotation matrix corresponding to all calibration data.

[0207] In an embodiment of the present invention, in the above step 414, when calculating the mean of the translation vectors corresponding to all the calibration data, the average translation vector components on the XYZ axes are calculated respectively according to the translation vectors corresponding to all the calibration data, and then the average translation vector components on the XYZ axes are converted back into a translation vector. The converted translation vector is the mean of the translation vectors corresponding to all the calibration data, that is, the average translation vector.

[0208] In the embodiment of the present invention, in the above step 414, the average translation vector is expressed as:

[0209]

[0210] Where t represents the average translation vector, (t x ,t y ,t z ) represents the average translation vector component on the XYZ axis, (x i ,y i ,z i ) represents the translation vector component on the XYZ axis corresponding to the i-th calibration data, N represents the total amount of calibration data, and T represents transposition.

[0211] In the embodiment of the present invention, in the above step 415, the data filtering judgment model expression is:

[0212]

[0213] Among them, Δt i Represents the difference between the translation vector corresponding to the i-th calibration data and the average translation vector, Represents the average value of the difference between the translation vector corresponding to all calibration data and the average translation vector, ΔR i Represents the difference between the rotation matrix corresponding to the i-th calibration data and the average rotation matrix, It represents the average value of the difference between the rotation matrix corresponding to all calibration data and the average rotation matrix, and scale represents the scale factor.

[0214] In the embodiment of the present invention, the difference Δt between the translation vector corresponding to the i-th calibration data and the average translation vector i Calculated by:

[0215] Δt i =‖tt i ‖;

[0216] Among them, t i Represents the translation vector corresponding to the i-th calibration data.

[0217] The average value of the difference between the translation vector corresponding to all calibration data and the average translation vector Calculated by:

[0218]

[0219] In the embodiment of the present invention, the difference ΔR between the rotation matrix corresponding to the i-th calibration data and the average rotation matrix i Calculated by:

[0220]

[0221] Among them, arccos represents the calculation of the inverse cosine trigonometric function, trace represents the trace of the matrix, and R i represents the rotation matrix corresponding to the i-th calibration data, and T represents the transposition operation.

[0222] The average value of the difference between the rotation matrix corresponding to all calibration data and the average rotation matrix Calculated by:

[0223]

[0224] It should be noted that the proportional factor is set according to actual needs, and different proportional factors represent different filtering processes.

[0225] In the embodiment of the present invention, in the above step 416, when result=1, the calibration data corresponding to the current rotation matrix and translation vector are retained; when result=0, the calibration data corresponding to the current rotation matrix and translation vector are deleted.

[0226] The objective function of traditional camera calibration is to minimize the reprojection error, which relies on two-dimensional corner point detection. That is to say, the calibration process seeks a camera model that minimizes the reprojection error of most points. However, there is noise in the actual calibration environment. Since the noise is unknown and unexplainable, the camera model may not be accurate even if the reprojection error is small. In an embodiment of the present invention, in addition to considering the reprojection error, global geometric constraints are also considered. The obtained calibration data set is filtered by means of global geometric constraints. Specifically, by filtering the calibration data set in the above manner, it is possible to filter out calibration data that lacks stability, and obtain a better calibration data set as the final data set, thereby obtaining an optimized calibration result.

[0227] Further, in an optional implementation of the embodiment of the present invention, in the above step 4, based on the final data set, the camera calibration can be performed using an existing known calibration method to obtain the parameters of the camera to be calibrated. For example, when the calibration board is a checkerboard calibration board, the camera calibration is performed using the Zhang Zhengyou calibration method to obtain the parameters of the camera to be calibrated.

[0228] It can be seen that the automatic calibration method based on optimal posture generation provided by the embodiment of the present invention can solve the problem of manually setting the posture of the calibration plate during the calibration process, and can solve the problem of reduced robustness caused by the existing calibration relying only on visual features, thereby reducing the proportion of human intervention in the calibration process and improving the calibration performance.

[0229] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0230] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, a magnetic disk storage, a magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0231] Finally, it should be noted that the automatic calibration device and method based on optimal posture generation disclosed in the embodiment of the present invention discloses only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An automatic calibration device based on optimal posture generation, characterized in that: The device comprises: a mechanical arm, a calibration plate and a data processing unit; The base of the mechanical arm can be fixedly installed at a first given position, the end of the mechanical arm is used to fix one of the camera to be calibrated and the calibration plate, and the other of the camera to be calibrated and the calibration plate can be fixedly installed at a second given position; The data processing unit can be connected to the robotic arm and the camera to be calibrated respectively, and is used for motion control of the robotic arm, for obtaining the recommended calibration pose of the calibration plate in the pixel coordinate system of the camera to be calibrated, for solving the end position of the robotic arm according to the recommended calibration pose, for performing imaging control on the camera to be calibrated to obtain calibration data, for filtering the calibration data based on global geometric constraints, and for processing the calibration data to obtain parameters of the camera to be calibrated.

2. An automatic calibration method based on optimal pose generation, characterized in that: The automatic calibration device based on optimal posture generation according to claim 1, wherein the method comprises: Step 1, according to the implementation situation, adopting the "eye-in-hand" calibration algorithm to obtain the first pre-data or adopting the "eye-to-hand" calibration algorithm to obtain the second pre-data; The implementation includes: the camera to be calibrated is fixedly installed at the end of the robotic arm or the calibration plate is fixedly installed at the end of the robotic arm; The first pre-set data includes: a transformation matrix between the robot base coordinate system and the calibration plate coordinate system, and a transformation matrix between the robot end coordinate system and the camera coordinate system to be calibrated; the second pre-set data includes: a transformation matrix between the robot base coordinate system and the camera coordinate system to be calibrated, and a transformation matrix between the robot end coordinate system and the calibration plate coordinate system; Step 2: Obtain a recommended calibration pose of the calibration plate in the pixel coordinate system of the camera to be calibrated and the current parameters of the camera to be calibrated, solve and adjust the end position of the manipulator according to the recommended calibration pose and the current parameters of the camera to be calibrated, and use the camera to be calibrated to image the calibration plate in the recommended calibration pose to obtain calibration data; Step 3, repeating step 2 until a calibration data set including a first preset amount of calibration data is obtained; Step 4: Based on the global geometric constraints, the calibration data set is filtered to obtain a final data set, and the camera is calibrated using the final data set to obtain the parameters of the camera to be calibrated.

3. The automatic calibration method based on optimal pose generation according to claim 2, characterized in that: The method of using the "eye-in-hand" calibration algorithm to obtain the first pre-data includes: Move the end of the robot arm to a first position point, control the camera to be calibrated to image the calibration plate, move the end of the robot arm to a second position point, and control the camera to be calibrated to image the calibration plate; According to the moving position information of the end of the manipulator and the imaging data of the camera to be calibrated, the second transformation matrix calculation model is solved to obtain the transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the camera to be calibrated; According to the moving position information of the end of the manipulator, the imaging data of the camera to be calibrated, and the transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the camera to be calibrated, the first transformation matrix calculation model is solved to obtain the transformation matrix between the coordinate system of the manipulator base and the coordinate system of the calibration plate; The first transformation matrix calculation model expression is: in, Represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system, a represents the position point index, a=1,2, It represents the transformation matrix from the coordinate system of the end of the robot arm to the coordinate system of the base of the robot arm when the end of the robot arm is at the ath position. It represents the transformation matrix from the camera coordinate system to be calibrated to the robot arm end coordinate system when the robot arm end is at the ath position. Represents the transformation matrix from the calibration plate coordinate system to the camera coordinate system to be calibrated when the end of the robotic arm is at the ath position; The second transformation matrix calculation model expression is: in, express The inverse operation of Represents the transformation matrix from the camera coordinate system to be calibrated to the end coordinate system of the robotic arm. express The inverse operation of .

4. The automatic calibration method based on optimal pose generation according to claim 2, characterized in that: The method of obtaining the second pre-data by adopting the "eye-to-hand" calibration method includes: Move the end of the robot arm to the third position point, control the camera to be calibrated to image the calibration plate, move the end of the robot arm to the fourth position point, and control the camera to be calibrated to image the calibration plate; According to the moving position information of the end of the manipulator and the imaging data of the camera to be calibrated, the fourth transformation matrix calculation model is solved to obtain the transformation matrix between the manipulator base coordinate system and the camera coordinate system to be calibrated; According to the moving position information of the end of the manipulator, the imaging data of the camera to be calibrated, and the transformation matrix between the coordinate system of the manipulator base and the coordinate system of the camera to be calibrated, the third transformation matrix calculation model is solved to obtain the transformation matrix between the coordinate system of the end of the manipulator and the coordinate system of the calibration plate; The third transformation matrix calculation model expression is: in, Represents the transformation matrix from the calibration plate coordinate system to the robot end coordinate system, b represents the position point index, b = 3, 4, It represents the transformation matrix from the robot base coordinate system to the robot end coordinate system when the robot end is at the bth position point. It represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system when the end of the robot is at the bth position. Represents the transformation matrix from the calibration plate coordinate system to the camera coordinate system to be calibrated when the end of the robotic arm is at the bth position; The fourth transformation matrix calculation model expression is: in, express The inverse operation of Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, express The inverse operation of .

5. The automatic calibration method based on optimal pose generation according to claim 2, characterized in that: Solve the end position of the robot arm based on the recommended calibration pose and the current parameters of the camera to be calibrated, including: The recommended calibration pose is calculated using the PnP algorithm to obtain the transformation matrix from the pixel coordinate system to the calibration plate coordinate system. Using the relative pose calculation model, the transformation matrix from the pixel coordinate system to the calibration board coordinate system and the current parameters of the camera to be calibrated are calculated and processed to obtain the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system; Using the robot end position calculation model, the transformation matrix from the camera coordinate system to be calibrated to the calibration plate coordinate system is calculated and processed to obtain the transformation matrix from the robot end coordinate system to the robot base coordinate system; According to the transformation matrix from the robot end coordinate system to the robot base coordinate system, the position of the robot end relative to the robot base is determined.

6. The automatic calibration method based on optimal pose generation according to claim 5, characterized in that: The relative posture calculation model expression is: in, Represents the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system, K -1 It represents the inverse operation of K, K represents the current intrinsic parameter matrix of the camera to be calibrated, and [R|t] represents the transformation matrix from the pixel coordinate system to the calibration plate coordinate system.

7. The automatic calibration method based on optimal pose generation according to claim 5, characterized in that: When the first pre-data is obtained in step 1, the calculation model expression of the end position of the robot arm is: in, Represents the transformation matrix from the robot end coordinate system to the robot base coordinate system, Represents the transformation matrix from the robot end coordinate system to the camera coordinate system to be calibrated, Represents the transformation matrix from the camera coordinate system to be calibrated to the calibration board coordinate system, Represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system; When the second pre-data is obtained in step 1, the calculation model expression of the robot end position is: in, Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, express The inverse operation of Represents the transformation matrix from the robot end coordinate system to the calibration plate coordinate system.

8. The automatic calibration method based on optimal pose generation according to claim 2, characterized in that: When the first pre-data is obtained in step 1, the calibration data set is filtered based on the global geometric constraint to obtain the final data set, including: For each calibration data, the first transformation matrix is ​​used to solve the model, and the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to each calibration data acquisition process is calculated; The obtained transformation matrix from the calibration plate coordinate system to the robot base coordinate system is split into a rotation matrix and a translation vector; Calculate the mean of the rotation matrices corresponding to all calibration data to obtain the average rotation matrix; Calculate the mean of the translation vectors corresponding to all calibration data to obtain the average translation vector; Using the data filtering judgment model, the average rotation matrix, the average translation vector, the rotation matrix and the translation vector corresponding to the calibration data are processed to obtain the filtering judgment result; According to the filtering judgment result, the calibration data is deleted or retained; The first transformation matrix solves the model expression as follows: Where i represents the calibration data index, represents the transformation matrix from the calibration plate coordinate system to the robot base coordinate system corresponding to the i-th calibration data acquisition process, represents the transformation matrix from the robot end coordinate system to the robot base coordinate system corresponding to the i-th calibration data acquisition process, Represents the transformation matrix from the camera coordinate system to be calibrated to the end coordinate system of the robotic arm. Represents the transformation matrix from the calibration plate coordinate system obtained through the i-th calibration data to the camera coordinate system to be calibrated.

9. The automatic calibration method based on optimal pose generation according to claim 2, characterized in that: When the second pre-data is obtained in step 1, the calibration data set is filtered based on the global geometric constraint to obtain the final data set, including: For each calibration data, the second transformation matrix is ​​used to solve the model, and the transformation matrix from the calibration plate coordinate system to the robot end coordinate system corresponding to each calibration data acquisition process is calculated; The obtained transformation matrix from the calibration plate coordinate system to the robot end coordinate system is split into a rotation matrix and a translation vector; Calculate the mean of the rotation matrices corresponding to all calibration data to obtain the average rotation matrix; Calculate the mean of the translation vectors corresponding to all calibration data to obtain the average translation vector; Using the data filtering judgment model, the average rotation matrix, the average translation vector, the rotation matrix and the translation vector corresponding to the calibration data are processed to obtain the filtering judgment result; According to the filtering judgment result, the calibration data is deleted or retained; The second transformation matrix solves the model expression as follows: Where i represents the calibration data index, It represents the transformation matrix from the calibration plate coordinate system to the robot end coordinate system corresponding to the i-th calibration data acquisition process, represents the transformation matrix from the robot base coordinate system to the robot end coordinate system corresponding to the i-th calibration data acquisition process, Represents the transformation matrix from the camera coordinate system to be calibrated to the robot base coordinate system, Represents the transformation matrix from the calibration plate coordinate system obtained through the i-th calibration data to the camera coordinate system to be calibrated.

10. The automatic calibration method based on optimal pose generation according to claim 8 or 9, characterized in that: The data filtering judgment model expression is: Among them, Δt i Represents the difference between the translation vector corresponding to the i-th calibration data and the average translation vector, Represents the average value of the difference between the translation vector corresponding to all calibration data and the average translation vector, ΔR i Represents the difference between the rotation matrix corresponding to the i-th calibration data and the average rotation matrix, It represents the average value of the difference between the rotation matrix corresponding to all calibration data and the average rotation matrix, and scale represents the scale factor; When result = 1, the calibration data corresponding to the current rotation matrix and translation vector are retained; When result = 0, the calibration data corresponding to the current rotation matrix and translation vector are deleted.

Images