Calibration jig error elimination method, storage medium, electronic equipment and program product
By calculating and optimizing the reprojection error of the calibration fixture, the inaccurate sensor calibration caused by the calibration fixture installation error is solved, and efficient and low-cost external parameter calibration of the sensor is achieved, which is suitable for large-scale production.
Patent Information
- Application Number
- CN202510557350.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-28
- Publication Date
- 2025-07-11
AI Technical Summary
In the existing sensor calibration methods, the calibration fixture often has position and attitude deviations when installed in the reference coordinate system, resulting in inaccurate calibration results, and relying on additional equipment such as laser trackers is expensive, making it difficult to meet the needs of large-scale production.
By determining the first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system and the second transformation parameter of the sensor relative to the calibration fixture coordinate system, the reprojection error is calculated, and the first transformation parameter is optimized based on the multiple reprojection errors, reducing installation errors and improving calibration accuracy.
There is no need to rely on expensive equipment, which reduces calibration costs, improves calibration efficiency and accuracy, meets the needs of large-scale production, and has stronger optimization process robustness.
Smart Images
Figure CN120293203A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of calibration technology, and particularly to a method for eliminating errors of a calibration fixture, a storage medium, an electronic device, and a program product. Background Art
[0002] The extrinsic calibration of a sensor is a key step in determining the conversion relationship between the sensor coordinate system and the external reference coordinate system, which has a direct impact on the measurement accuracy of the sensor.
[0003] A common calibration method is to fix the sensor on a calibration fixture for extrinsic calibration. However, when the calibration fixture is installed in the reference coordinate system, there may be position and attitude deviations, resulting in incomplete coincidence between the calibration fixture coordinate system and the reference coordinate system, thereby introducing installation errors and further affecting the accuracy of the sensor calibration result. Summary of the Invention
[0004] In view of this, embodiments of this application provide a method for eliminating errors of a calibration fixture, a storage medium, an electronic device, and a program product.
[0005] In a first aspect, an embodiment of this application provides a method for eliminating errors of a calibration fixture, which is applied to a calibration fixture in a calibration field. Multiple sensors are arranged on the calibration fixture, and the calibration field further includes a calibration board. The method includes: determining a first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system; determining a second transformation parameter of each sensor relative to the calibration fixture coordinate system; based on the first transformation parameter and the second transformation parameter, calculating multiple reprojection errors. The calibration board includes target points, and the reprojection error is the reprojection error of the target points from the reference coordinate system to the sensor coordinate system; based on the multiple reprojection errors, optimizing the first transformation parameter to obtain a third transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system.
[0006] In combination with the first aspect, in some implementation manners of the first aspect, calculating multiple reprojection errors based on the first transformation parameter and the second transformation parameter includes: obtaining first coordinate data of the target points in the reference coordinate system, and second coordinate data of the target points in each sensor coordinate system; based on the product of the first transformation parameter, the second transformation parameter, and the first coordinate data, obtaining third coordinate data of the target points from the reference coordinate system to each sensor coordinate system; based on the difference between the third coordinate data and the second coordinate data, obtaining multiple reprojection errors.
[0007] In combination with the first aspect, in some implementation manners of the first aspect, obtaining first coordinate data of the target points in the reference coordinate system includes: obtaining scan data of the calibration field by a scanning device, and the reference coordinate system is the coordinate system where the scanning device is located; based on the scan data, determining the first coordinate data of the target points in the reference coordinate system.
[0008] In combination with the first aspect, in some implementations of the first aspect, the first transformation parameter is optimized based on multiple reprojection errors, including: determining the relative transformation error between any two sensors based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system; and optimizing the first transformation parameter based on the multiple reprojection errors and the relative transformation error between any two sensors.
[0009] In combination with the first aspect, in some implementations of the first aspect, determining the relative transformation error between any two sensors based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system includes: determining the fourth transformation parameter between any two sensors based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system; determining the fifth transformation parameter between any two sensors based on the co-visibility relationship between any two sensors; and determining the relative transformation error between any two sensors based on the fourth transformation parameter and the fifth transformation parameter.
[0010] In combination with the first aspect, in some implementations of the first aspect, determining the fifth transformation parameter between any two sensors based on the co-visibility relationship between any two sensors includes: obtaining the observation data of the same target point for any two sensors based on the co-visibility relationship between any two sensors; obtaining the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two sensors based on the observation data of the same target point for any two sensors; and determining the fifth transformation parameter between any two sensors based on the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two sensors.
[0011] In combination with the first aspect, in some implementations of the first aspect, the multiple sensors include a first sensor and a second sensor, the first sensor and the second sensor are configured to be disposed in a robot, and during the execution of the calibration fixture error elimination method, the relative positions of the first sensor and the second sensor are fixed.
[0012] Second aspect, an embodiment of the present application provides a calibration fixture error elimination device, which is applied to a calibration fixture in a calibration field. A plurality of sensors are arranged on the calibration fixture, and the calibration field further includes a calibration plate. The device includes: a first determination module, configured to determine a first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system; a second determination module, configured to determine a second transformation parameter of each sensor relative to the calibration fixture coordinate system; a calculation module, configured to calculate a plurality of reprojection errors based on the first transformation parameter and the second transformation parameter. The calibration plate includes target points, and the reprojection error is the reprojection error of the target points when converted from the reference coordinate system to the sensor coordinate system; an optimization module, configured to optimize the first transformation parameter based on the plurality of reprojection errors to obtain a third transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system.
[0013] Third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for executing the calibration fixture error elimination method described in the first aspect.
[0014] Fourth aspect, an embodiment of the present application provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to execute the calibration fixture error elimination method described in the first aspect.
[0015] Fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions that, when executed on an electronic device, cause the electronic device to implement the calibration fixture error elimination method described in the first aspect.
[0016] In this application, by determining the first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system, a foundation is laid for subsequent optimization steps. Then, the second transformation parameter of each sensor relative to the calibration fixture coordinate system is determined, so that an accurate conversion relationship is established between each sensor and the calibration fixture coordinate system. Based on these two transformation parameters, the reprojection error is calculated, and by analyzing the reprojection error, the deviation existing in the installation process of the calibration fixture can be identified. Finally, based on multiple reprojection errors, the first transformation parameter is optimized to obtain the third transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system. This optimization step can effectively reduce the reprojection error, thereby improving the accuracy of the conversion relationship between the calibration fixture coordinate system and the reference coordinate system, and further enhancing the overall accuracy of the external parameter calibration of the sensor. Moreover, compared with the existing methods, this application does not need to rely on additional expensive equipment, such as a laser tracker, etc., reducing the calibration cost and improving the calibration efficiency at the same time, meeting the requirements of mass production. In addition, each sensor has its own distortion, noise, and calibration error. Using a single reprojection error alone may lead to a local optimal solution. Therefore, in this embodiment, multiple reprojection errors are utilized, which can make the optimization process more robust, reduce the optimization deviation caused by a certain reprojection error, and improve the accuracy of the third transformation parameter. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] By describing the embodiments of the present application in more detail with reference to the accompanying drawings, the above and other objects, features, and advantages of the present application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application, and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0018] Figure 1 The flowchart of the calibration fixture error elimination method provided by an embodiment of the present application is shown.
[0019] Figure 2 The schematic diagram of the calibration field provided by an embodiment of the present application is shown.
[0020] Figure 3 The schematic flowchart of calculating multiple reprojection errors provided by an embodiment of the present application is shown.
[0021] Figure 4 The schematic flowchart of optimizing the first transformation parameter provided by an embodiment of the present application is shown.
[0022] Figure 5 The schematic flowchart of determining the relative transformation error between any two sensors provided by an embodiment of the present application is shown.
[0023] Figure 6The following is a schematic structural diagram of a calibration fixture error elimination device provided by an embodiment of the present application.
[0024] Figure 7 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners
[0025] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0026] The core task of external parameter calibration of a sensor is to determine the conversion relationship between the sensor coordinate system and the external reference coordinate system, and this conversion relationship directly affects the positioning and measurement accuracy of the sensor in actual applications.
[0027] However, in an actual production line, the calibration process usually relies on fixing the sensor on a calibration fixture for external parameter calibration. Although in theory, this method can provide stable calibration conditions, in actual operation, there will inevitably be position and attitude deviations when the calibration fixture is installed in the reference coordinate system. This deviation causes the calibration fixture coordinate system and the reference coordinate system to not completely coincide, thereby introducing installation errors.
[0028] Most of the existing methods for external parameter calibration of sensors ignore the influence of installation errors on the final calibration results. However, this will directly weaken the accuracy of external parameter calibration of sensors, and further affect the measurement accuracy of sensors. In addition, although some technologies attempt to compensate for the installation errors of calibration fixtures by introducing additional devices (such as laser trackers), these methods are usually costly and complex to operate, and it is difficult to meet the requirements of large-scale production. For example, laser trackers require professional personnel to operate and require a long calibration time, which makes them difficult to popularize on high-efficiency production lines.
[0029] Therefore, there is an urgent need to develop a method that can effectively remove the installation errors of calibration fixtures. This method should be efficient and low-cost in order to improve the accuracy of external parameter calibration of sensors.
[0030] Based on the above, the present application is proposed. Specifically, Figure 1 The following is a flowchart of a calibration fixture error elimination method provided by an embodiment of the present application. Exemplarily, this method is applied to Figure 2The calibration fixture in the calibration field shown, where the calibration field is an environment specifically for performing external parameter calibration of sensors. Multiple sensors are arranged on the calibration fixture, and these sensors are the objects that need to be externally parameter calibrated. By fixing the sensors on the calibration fixture, the calibration work can be carried out more stably. In addition, the calibration field also includes a calibration board, which has feature points, and the positions of these points in the calibration board are predetermined. The sensors can sense the feature points on the calibration board to provide a data basis for calculating the external parameters.
[0031] As Figure 1 shown, the method includes the following steps.
[0032] Step S110, determine the first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system.
[0033] Specifically, the calibration fixture coordinate system refers to the coordinate system where the calibration fixture is located. The reference coordinate system is a pre-determined coordinate system in the calibration field and remains unchanged during the calibration process, providing a common reference standard for the calibration fixture coordinate system and other related coordinate systems, making it possible to convert and compare different coordinate systems.
[0034] Optionally, the first transformation parameter is a parameter used to describe the transformation relationship between the calibration fixture coordinate system and the reference coordinate system, including two parts: rotation and translation. In some embodiments, the first transformation parameter is an initial transformation parameter to be optimized. Specifically, when the calibration fixture is installed in the calibration field, it has an initial first transformation parameter, but this first transformation parameter is to be optimized. As mentioned above, the reason is that there will inevitably be certain position and attitude deviations during the installation process of the calibration fixture, resulting in the first transformation parameter not being able to accurately reflect the true transformation relationship between the calibration fixture coordinate system and the reference coordinate system.
[0035] Step S120, determine the second transformation parameter of each sensor relative to the calibration fixture coordinate system.
[0036] The second transformation parameter is a parameter used to describe the transformation relationship between the sensor and the calibration fixture coordinate system, also including two parts: rotation and translation.
[0037] In some embodiments, based on the structural parameters, determine the second transformation parameter of each sensor relative to the calibration fixture coordinate system. Among them, the structural parameters refer to the physical structure-related parameters such as the installation position and installation direction of the sensor on the calibration fixture. It can be understood that the sensors are installed on the calibration fixture, and their structural parameters such as installation position and direction are known or can be measured.
[0038] In actual operation, during the design and manufacturing stage of the calibration fixture, the installation position of each sensor (such as the coordinates relative to the origin of the calibration fixture coordinate system) and the installation direction (such as the direction angles relative to the axes of the calibration fixture coordinate system) can be accurately measured and recorded. Then, based on these structural parameters, the second transformation parameters of each sensor relative to the calibration fixture coordinate system are calculated through the principle of geometric transformation (such as the combination of rotation matrix and translation vector).
[0039] Step S130, calculate multiple reprojection errors based on the first transformation parameters and the second transformation parameters.
[0040] The calibration board includes target points. The reprojection error is the reprojection error of the target points when transformed from the reference coordinate system to the sensor coordinate system. Specifically, it refers to the difference between the actual coordinate data of the target points in the calibration board in the sensor coordinate system and the theoretical coordinate data calculated according to the coordinate transformation relationship, which reflects the accuracy of the coordinate transformation parameters and various error factors during the calibration process.
[0041] Specifically, through the first transformation parameters and the second transformation parameters, the theoretical projection coordinates of the target points when transformed from the reference coordinate system to the sensor coordinate system can be obtained; then, the actual coordinates of the target points in the sensor coordinate system are acquired; based on the actual coordinates and the theoretical projection coordinates, the reprojection error is obtained.
[0042] Step S140, optimize the first transformation parameters based on multiple reprojection errors to obtain the third transformation parameters of the calibration fixture coordinate system relative to the reference coordinate system.
[0043] In one example, the first transformation parameters are used as the initial values, the reprojection error is used as the objective function, and a nonlinear optimization model is constructed; according to factors such as the characteristics of the optimization objective and the calculation efficiency, a suitable optimization algorithm is selected; using this optimization algorithm, through iterative calculation, the first transformation parameters are continuously adjusted to gradually reduce the objective function until the convergence condition is reached, and the third transformation parameters are obtained. The third transformation parameters can more accurately describe the transformation relationship between the calibration fixture coordinate system and the reference coordinate system, effectively compensating for the influence of the calibration fixture installation error and other factors on the calibration result.
[0044] In another example, the first transformation parameters and the second transformation parameters are jointly used as the initial values to construct a nonlinear optimization model. Through iterative calculation, the first transformation parameters and the second transformation parameters are continuously adjusted until the convergence condition is reached, obtaining the third transformation parameters and the optimized second transformation parameters. The optimized second transformation parameters can more accurately describe the transformation relationship of the sensor relative to the calibration fixture coordinate system, reducing calculation errors.
[0045] It should be noted that when the first transformation parameter and the second transformation parameter are jointly used as the initial values for iterative optimization, the second transformation parameter can absorb a part of the residuals that should have been borne by the first transformation parameter, making the optimized third transformation parameter more accurate.
[0046] Optionally, the convergence condition includes that the error change is less than a certain threshold or the maximum number of iterations is reached. Optionally, the optimization algorithms include the Levenberg-Marquardt algorithm, the Ceres Solver algorithm, etc.
[0047] In this embodiment, by determining the first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system, a foundation is laid for subsequent optimization steps. Then, the second transformation parameter of each sensor relative to the calibration fixture coordinate system is determined, so that an accurate conversion relationship is established between each sensor and the calibration fixture coordinate system. Based on these two transformation parameters, the reprojection error is calculated, and by analyzing the reprojection error, the deviation existing in the installation process of the calibration fixture can be identified. Finally, based on multiple reprojection errors, the first transformation parameter is optimized to obtain the third transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system. This optimization step can effectively reduce the reprojection error, thereby improving the accuracy of the conversion relationship between the calibration fixture coordinate system and the reference coordinate system, and further enhancing the overall accuracy of the external parameter calibration of the sensor. Moreover, compared with the existing methods, this application does not need to rely on additional expensive equipment such as laser trackers, reducing the calibration cost, while improving the calibration efficiency and meeting the requirements of mass production. In addition, each sensor has its own distortion, noise, and calibration error. Using only one reprojection error may lead to a local optimal solution. Therefore, in this embodiment, using multiple reprojection errors can make the optimization process more robust, reduce the optimization deviation caused by a certain reprojection error, and improve the accuracy of the third transformation parameter.
[0048] Figure 3 The following shows a schematic flowchart of calculating multiple reprojection errors provided by an embodiment of the present application. Figure 1 Based on the embodiment shown above, Figure 3 the embodiment shown below is extended. Figure 3 The differences between the embodiment shown below and Figure 1 the embodiment shown above are emphasized below, and the same parts will not be described again.
[0049] As Figure 3 shown, in this embodiment, based on the first transformation parameter and the second transformation parameter, calculating multiple reprojection errors includes the following steps.
[0050] Step S310, obtain the first coordinate data of the target point in the reference coordinate system and the second coordinate data of the target point in each sensor coordinate system.
[0051] In the calibration field, the position and orientation of the calibration board are known, and the first coordinate data of the target point in the reference coordinate system can be obtained through measurement or presetting, which reflects the position of the target point in the overall coordinate system of the calibration field.
[0052] Each sensor on the calibration fixture aligns with the target point on the calibration board for measurement. According to its own coordinate system and measurement principle, each sensor can obtain the coordinate data of the target point in the coordinate system of the sensor, that is, the second coordinate data, which reflects the position of the target point from the perspective of each sensor.
[0053] Step S320: Based on the product of the first transformation parameter, the second transformation parameter, and the first coordinate data, obtain the third coordinate data of the target point transformed from the reference coordinate system to the coordinate systems of each sensor.
[0054] Optionally, first, based on the product of the first transformation parameter and the second transformation parameter, obtain the transformation parameter of each sensor relative to the reference coordinate system. Then, based on the product of the transformation parameter of each sensor relative to the reference coordinate system and the first coordinate data, obtain the third coordinate data of the target point transformed from the reference coordinate system to the coordinate systems of each sensor.
[0055] Specifically, the calibration fixture coordinate system is associated with the reference coordinate system through the first transformation parameter, and each sensor is associated with the calibration fixture coordinate system through the second transformation parameter. According to the transitivity of coordinate transformation, by multiplying these two transformation parameters, the direct transformation relationship between the sensor coordinate system and the reference coordinate system can be obtained. Then, applying the first coordinate data of the target point in the reference coordinate system to the transformation parameter of the sensor relative to the reference coordinate system can transform it to the corresponding sensor coordinate system.
[0056] Assume that the first transformation parameter consists of a rotation matrix R1 and a translation vector T1, and the second transformation parameter consists of a rotation matrix R2 and a translation vector T2. Then, the rotation matrix R corresponding to the transformation parameter of the sensor relative to the reference coordinate system is R = R1 × R2; the translation vector T corresponding to the transformation parameter of the sensor relative to the reference coordinate system is T = R1 × T2 + T1. In this way, the transformation parameter of the sensor relative to the reference coordinate system synthesizes the transformation of the calibration fixture coordinate system relative to the reference coordinate system and the transformation of the sensor relative to the calibration fixture coordinate system, thereby directly establishing the conversion relationship between the sensor and the reference coordinate system. Assume that the first coordinate data of the target point in the reference coordinate system is point P. Then, the third coordinate data P' of the target point in the sensor coordinate system is P' = R
[0057] × P + T.
[0058] Step S330: Based on the difference between the third coordinate data and the second coordinate data, obtain multiple reprojection errors.
[0059] In this embodiment, by calculating the difference between the third coordinate data and the second coordinate data, the reprojection error is obtained. The magnitude and distribution of the reprojection error can be used to evaluate the installation error of the calibration fixture. Optionally, a smaller reprojection error indicates a smaller installation error of the calibration fixture, while a larger reprojection error indicates a larger installation error of the calibration fixture and installation calibration is required.
[0060] Assume that the first coordinate data of the target point P in the reference coordinate system is (1, 2, 3). Through transformation, the third coordinate data of the target point P in the sensor coordinate system is (3, 8, 12), while the second coordinate data of the target point P actually measured by the sensor in the sensor coordinate system is (3.1, 8.1, 12.1). Then the error in the X direction is ΔX = 3.1 - 3 = 0.1; the error in the Y direction is ΔY = 8.1 - 8 = 0.1; the error in the Z direction is ΔZ = 12.1 - 12 = 0.1. Therefore, the reprojection error of the target point P in this sensor coordinate system is (0.1, 0.1, 0.1).
[0061] In this embodiment, by obtaining the coordinate data of the target point in the reference coordinate system and the sensor coordinate system, a basis is provided for the calculation of the reprojection error. Then, based on the first transformation parameter and the second transformation parameter, the target point is transformed from the reference coordinate system to each sensor coordinate system, and the theoretical third coordinate data is obtained. In this process, the structural relationship between the calibration fixture and the sensor is fully utilized to ensure the accuracy of the transformation. Finally, by calculating the difference between the third coordinate data and the actually measured second coordinate data, the reprojection error is obtained. The reprojection error intuitively reflects the accuracy of the calibration fixture installation and provides a direct basis for optimizing the first transformation parameter.
[0062] Next, based on Figure 3 the embodiment shown below, a specific implementation method for obtaining the first coordinate data of the target point in the reference coordinate system is introduced. This implementation method includes: obtaining the scan data of the calibration field by the scanning device, where the reference coordinate system is the coordinate system where the scanning device is located; and determining the first coordinate data of the target point in the reference coordinate system based on the scan data.
[0063] The scanning device refers to an instrument with high-precision scanning functions, such as a laser scanner, a 3D scanner, etc., which can scan the calibration field and obtain a large number of data points for reconstructing the three-dimensional structure of the calibration field.
[0064] Optionally, first obtain the scanning data of the scanning device for the calibration field. The coordinate system where the scanning device is located is the reference coordinate system. Therefore, the coordinates of the scanned data points are all relative to the reference coordinate system. Then, based on this scanning data, through calibration fixture error elimination and analysis, the first coordinate data of the target points on the calibration plate in the reference coordinate system can be calibrated.
[0065] In this embodiment, the scanning device can scan the entire calibration field with high precision, thereby effectively improving the accuracy of the first coordinate data. Defining the reference coordinate system as the coordinate system where the scanning device is located enables the scanning data to directly reference this coordinate system, avoiding additional coordinate conversion steps, reducing errors caused by coordinate conversion, and improving the consistency and reliability of the data.
[0066] Figure 4 The following shows a schematic flowchart for optimizing the first transformation parameter provided by an embodiment of the present application. On the basis of Figure 1 the embodiment shown, an embodiment is extended, Figure 4 and the following focuses on Figure 4 the differences between the embodiment shown and Figure 1 the embodiment shown. The same parts will not be elaborated again.
[0067] As Figure 4 shown, in this embodiment, based on multiple reprojection errors, the first transformation parameter is optimized, including the following steps.
[0068] Step S410, based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system, determine the relative transformation error between any two sensors.
[0069] The relative transformation error refers to the deviation between the coordinate conversion relationship calculated based on the second transformation parameters of any two sensors and the actual conversion relationship between them. This error reflects the inconsistency between the two sensor coordinate systems in actual applications.
[0070] Optionally, the second transformation parameter describes the conversion relationship of each sensor relative to the calibration fixture coordinate system. Then, based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system, the transformation parameter between any two sensors can be calculated; based on the transformation parameter between any two sensors measured actually and the calculated transformation parameter between any two sensors, the relative transformation error between any two sensors is obtained.
[0071] Exemplarily, for any two sensors A and sensor B, by multiplying the inverse of the second transformation parameter of sensor A by the second transformation parameter of sensor B, the transformation parameter between sensor A and sensor B can be obtained, that is where TA and T B are the second transformation parameters of sensor A and sensor B respectively.
[0072] Step S420: Optimize the first transformation parameter based on multiple reprojection errors and the relative transformation error between any two sensors.
[0073] Optionally, add each reprojection error and the relative transformation error between any two sensors to obtain an objective function; continuously adjust the first transformation parameter in the objective function through iterative calculation, and evaluate the sum of the reprojection error and the relative transformation error under the current first transformation parameter in each iterative calculation; update the current first transformation parameter according to the rules of the optimization algorithm and the value of this sum, so that it gradually approaches the optimal solution.
[0074] Exemplarily, assuming there are two sensors A and B, then, based on multiple reprojection errors and the relative transformation error between any two sensors, the constructed objective function is:
[0075]
[0076] where represents the second transformation parameter of sensor A relative to the calibration fixture coordinate system, represents the first transformation parameter of the calibration fixture coordinate system relative to the reference coordinate system, P gt represents the first coordinate data of the target point in the reference coordinate system, P A represents the second coordinate data of the target point in the sensor coordinate system where sensor A is located, represents the second transformation parameter of sensor B relative to the calibration fixture coordinate system, P B represents the second coordinate data of the target point in the sensor coordinate system where sensor B is located, represents the transformation parameter between sensor A and sensor B measured actually, and π represents the projection function. represents the reprojection error corresponding to sensor A; represents the reprojection error corresponding to sensor B; represents the relative transformation error between sensor A and B.
[0077] In addition, it can be understood that if the values of the transformation parameters between the two actually measured sensors A and B are the same as the calculated transformation parameters between the two sensors A and B, If the value is the identity matrix, that is, 1, then by taking its Log value, zero can be obtained. Therefore, in this example, by taking the Log value of the relative transformation error between any two sensors, the error value can be more conveniently quantified. It should be noted that this example only gives an implementation method of an objective function, and those skilled in the art can make changes according to requirements in actual applications.
[0078] In this embodiment, by introducing the relative transformation error between any two sensors, the relative relationship between sensors during the optimization process is avoided from shifting, that is, the optimized third transformation parameter may cause the second transformation parameter of the sensor relative to the calibration fixture coordinate system to change, thereby maintaining the self-consistency of the entire calibration system, making the optimized third transformation parameter also conform to the calibration relationship between the original sensors, improving the stability of subsequent external parameter calibration of the sensors, and finally obtaining the reliable pose of the calibration fixture in the reference coordinate system.
[0079] Figure 5 The following shows a schematic flow chart of determining the relative transformation error between any two sensors provided by an embodiment of the present application. In Figure 4 Based on the embodiment shown, Figure 5 The following embodiment is extended, Figure 5 The differences between the embodiment shown below and Figure 4 The embodiment shown will be emphasized below, and the same parts will not be elaborated.
[0080] As Figure 5 Shown, based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system, determining the relative transformation error between any two sensors includes the following steps.
[0081] Step S510, based on the second transformation parameters of each sensor relative to the calibration fixture coordinate system, determine the fourth transformation parameter between any two sensors.
[0082] As mentioned above, for any two sensors A and B, by multiplying the inverse of the second transformation parameter of sensor A by the second transformation parameter of sensor B, the fourth transformation parameter between sensor A and sensor B can be obtained, that is Of course, the second transformation parameter of sensor A can also be multiplied by the inverse of the second transformation parameter of sensor B to obtain the fourth transformation parameter between sensor A and sensor B.
[0083] Step S520, based on the co-visibility relationship between any two sensors, determine the fifth transformation parameter between any two sensors.
[0084] The co-visibility relationship refers to the situation where two sensors can simultaneously observe the same target point or the same area. In this case, the observation data of the same target point by the two sensors can be used to calculate the relative position and attitude relationship between them.
[0085] In some embodiments, first find the common target points that can be observed by both sensors, then obtain the coordinate data of these common target points in the coordinate systems of the two sensors, and finally, use the coordinate data of the common target points collected in the sensor coordinate systems to calculate the relative transformation relationship between the two sensors, that is, the fifth transformation parameter, through mathematical methods (such as the least squares method, singular value decomposition, etc.).
[0086] Specifically, based on the co-visibility relationship between any two sensors, obtain the observation data of any two sensors for the same target point; based on the observation data of any two sensors for the same target point, obtain the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two sensors; based on the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two sensors, determine the fifth transformation parameter between any two sensors.
[0087] Optionally, if the sensor is a camera, the observation data includes the images collected by the camera; if the sensor is a lidar, the observation data includes the point cloud collected by the lidar. Then, the two-dimensional coordinates of the target point in the image, or the three-dimensional coordinates of the target point in the point cloud, can be obtained. Finally, the relative transformation relationship between the two sensors, that is, the fifth transformation parameter, is calculated through mathematical methods (such as the least squares method, etc.).
[0088] Suppose sensor A and sensor B with co-visibility relationship are lidars, and they can both observe the target point P on the calibration board. The coordinates of the target point P in the coordinate system of sensor A are (1, 2, 3), and the coordinates in the coordinate system of sensor B are (4, 5, 6). Through these observation data, a system of equations can be established to calculate the fifth transformation parameter from sensor A to sensor B. Suppose the calculated fifth transformation parameter is: rotation matrix R5 = [[0.999, -0.012, 0.023], [0.011, 0.999, -0.021], [-0.022, 0.019, 0.999]] translation vector T5 = (0.5, -0.3, 1.2).
[0089] In this embodiment, by using the co-visibility relationship to obtain the observation data of two sensors for the same target point, and then obtaining the fourth coordinate data based on these observation data, the calculation process of the fifth transformation parameter is made more reliable, improving the accuracy of the relative transformation relationship between sensors.
[0090] Step S530: Determine the relative transformation error between any two sensors based on the fourth transformation parameter and the fifth transformation parameter.
[0091] In one example, the relative transformation error between any two sensors is obtained based on the difference between the fourth transformation parameter and the fifth transformation parameter. In another example, as Figure 4 described in the embodiment shown, the relative transformation error between any two sensors can be obtained by using the product of the inverse of the fourth transformation parameter and the fifth transformation parameter; or the relative transformation error between any two sensors can be obtained by using the product of the fourth transformation parameter and the inverse of the fifth transformation parameter.
[0092] In Figure 5 the solution shown, first, the fourth transformation parameter is calculated by using the second transformation parameter, providing an estimated value of the relative position and attitude between the sensors. Second, the fifth transformation parameter determined through the co-visibility relationship directly reflects the actual conversion relationship between the sensors based on the actual observation data of the same target point by the sensors. Finally, by comparing the fourth transformation parameter and the fifth transformation parameter, the relative transformation error is determined, which can accurately quantify the deviation of the conversion relationship between the sensors and provide a direct basis for optimizing the first transformation parameter.
[0093] In some embodiments, the multiple sensors include a first sensor and a second sensor. The first sensor and the second sensor are used to be disposed in a robot, and during the process of performing the calibration jig error elimination method of the present application, the relative position of the first sensor and the second sensor is fixed, which ensures that the geometric relationship between the sensors remains stable during the entire use process, thereby improving the reliability and stability of the obtained third transformation parameter. Optionally, the robot includes an embodied robot.
[0094] As described above in combination with Figures 1 to 5 , the embodiments of the calibration jig error elimination method of the present application are described in detail. Below in combination with Figure 6 , the embodiments of the calibration jig error elimination device of the present application are described in detail. It should be understood that the description of the embodiments of the calibration jig error elimination method corresponds to the description of the embodiments of the calibration jig error elimination device. Therefore, the parts not described in detail can be referred to the previous method embodiments.
[0095] Figure 6 The following shows a schematic structural diagram of a calibration jig error elimination device provided by an embodiment of the present application. As Figure 6 shown, the calibration jig error elimination device 60 provided by the embodiment of the present application includes:
[0096] A first determination module 610, configured to determine a first transformation parameter of the calibration jig coordinate system relative to the reference coordinate system;
[0097] A second determination module 620, configured to determine second transformation parameters of each sensor relative to the calibration jig coordinate system;
[0098] A calculation module 630, configured to calculate a plurality of reprojection errors based on the first transformation parameters and the second transformation parameters. The calibration board includes target points, and the reprojection error is the reprojection error of the target points when transformed from the reference coordinate system to the sensor coordinate system;
[0099] An optimization module 640, configured to optimize the first transformation parameters based on the plurality of reprojection errors to obtain third transformation parameters of the calibration jig coordinate system relative to the reference coordinate system.
[0100] In some embodiments, the calculation module 630 is further configured to obtain first coordinate data of the target points in the reference coordinate system and second coordinate data of the target points in each sensor coordinate system; obtain third coordinate data of the target points when transformed from the reference coordinate system to each sensor coordinate system based on the product of the first transformation parameters, the second transformation parameters, and the first coordinate data; and obtain a plurality of reprojection errors based on the difference between the third coordinate data and the second coordinate data.
[0101] In some embodiments, the calculation module 630 is further configured to obtain scanning data of the calibration field by a scanning device, where the reference coordinate system is the coordinate system where the scanning device is located; and determine first coordinate data of the target points in the reference coordinate system based on the scanning data.
[0102] In some embodiments, the optimization module 640 is further configured to determine relative transformation errors between any two sensors based on the second transformation parameters of each sensor relative to the calibration jig coordinate system; and optimize the first transformation parameters based on the plurality of reprojection errors and the relative transformation errors between any two sensors.
[0103] In some embodiments, the optimization module 640 is further configured to determine fourth transformation parameters between any two sensors based on the second transformation parameters of each sensor relative to the calibration jig coordinate system; determine fifth transformation parameters between any two sensors based on the co-visibility relationship between any two sensors; and determine relative transformation errors between any two sensors based on the fourth transformation parameters and the fifth transformation parameters.
[0104] In some embodiments, the optimization module 640 is further configured to obtain observation data of the same target point by any two sensors based on the co-visibility relationship between any two sensors; obtain fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two sensors based on the observation data of the same target point by any two sensors; and determine fifth transformation parameters between any two sensors based on the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two sensors.
[0105] In some embodiments, the multiple sensors include a first sensor and a second sensor. The first sensor and the second sensor are configured to be disposed in a robot, and during the execution of the calibration jig error elimination method, the relative positions of the first sensor and the second sensor are fixed.
[0106] Next, reference is made to Figure 7 to describe the electronic device according to an embodiment of the present application. Figure 7 FIG. shows a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.
[0107] As Figure 7 shown, the electronic device 70 includes one or more processors 701 and a memory 702.
[0108] The processor 701 may be a central processing unit (CPU) or other forms of processing units having the calibration jig error elimination ability and / or instruction execution ability, and may control other components in the electronic device 70 to perform desired functions.
[0109] The memory 702 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 701 may run the program instructions to implement the calibration jig error elimination method of various embodiments of the present application described above and / or other desired functions. Various contents such as the first transformation parameter, the second transformation parameter, the reprojection error, etc. may also be stored in the computer-readable storage media.
[0110] In one example, the electronic device 70 may further include: an input device 703 and an output device 704, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).
[0111] The input device 703 may include, for example, a keyboard, a mouse, etc.
[0112] The output device 704 may output various information to the outside, including the first transformation parameter, the second transformation parameter, the reprojection error, etc. The output device 704 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.
[0113] Of course, for simplicity, Figure 7Only some of the components in the electronic device 70 related to this application are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 70 may further include any other appropriate components.
[0114] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps in the calibration fixture error elimination method according to various embodiments of the present application described above in this specification.
[0115] The computer program product can be written in any combination of one or more programming languages for the program code to perform the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0116] In addition, an embodiment of the present application may also be a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run by a processor, the processor is caused to execute the steps in the calibration fixture error elimination method according to various embodiments of the present application described above in this specification.
[0117] The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0118] The basic principles of the present application have been described in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present application. Additionally, the specific details disclosed above are only for illustrative and easy-to-understand purposes and not limitations. The above details do not limit the present application to necessarily implement using the above specific details.
[0119] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present application are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms meaning "including but not limited to" and can be used interchangeably with each other. The word "or" and "and" used herein refer to the word "and / or" and can be used interchangeably with each other unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with each other.
[0120] It should also be noted that in the devices, equipment, and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present application.
[0121] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.
[0122] The above description has been given for purposes of illustration and description. In addition, this description does not intend to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.
Claims
1. A method for eliminating fixture errors, characterized in that A calibration jig applied to a calibration field, with multiple sensors provided on the calibration jig, and the calibration field further includes a calibration board. The method includes: Determine the first transformation parameter of the calibration jig coordinate system relative to the reference coordinate system; Determine the second transformation parameter of each of the sensors relative to the calibration jig coordinate system; Based on the first transformation parameter and the second transformation parameter, calculate multiple reprojection errors. The calibration board includes target points, and the reprojection error is the reprojection error of the target points when transformed from the reference coordinate system to the sensor coordinate system; Based on the multiple reprojection errors, optimize the first transformation parameter to obtain the third transformation parameter of the calibration jig coordinate system relative to the reference coordinate system.
2. The calibration fixture error elimination method according to claim 1, wherein The step of calculating multiple reprojection errors based on the first transformation parameter and the second transformation parameter includes: Obtain the first coordinate data of the target points in the reference coordinate system and the second coordinate data of the target points in each of the sensor coordinate systems; Based on the product of the first transformation parameter, the second transformation parameter, and the first coordinate data, obtain the third coordinate data of the target points when transformed from the reference coordinate system to each of the sensor coordinate systems; Based on the difference between the third coordinate data and the second coordinate data, obtain multiple reprojection errors.
3. The calibration fixture error elimination method according to claim 2, wherein The step of obtaining the first coordinate data of the target points in the reference coordinate system includes: Obtain the scanning data of the scanning device for the calibration field. The reference coordinate system is the coordinate system where the scanning device is located; Based on the scanning data, determine the first coordinate data of the target points in the reference coordinate system.
4. The calibration fixture error elimination method according to any one of claims 1 to 3, characterized in that, The step of optimizing the first transformation parameter based on the multiple reprojection errors includes: Based on the second transformation parameter of each of the sensors relative to the calibration jig coordinate system, determine the relative transformation error between any two of the sensors; Based on the multiple reprojection errors and the relative transformation error between any two of the sensors, optimize the first transformation parameter.
5. The error elimination method of the calibration fixture according to claim 4, characterized in that, The step of determining the relative transformation error between any two of the sensors based on the second transformation parameter of each of the sensors relative to the calibration jig coordinate system includes: Based on the second transformation parameter of each of the sensors relative to the calibration jig coordinate system, determine the fourth transformation parameter between any two of the sensors; Based on the co-visibility relationship between any two of the sensors, determine the fifth transformation parameter between any two of the sensors; Based on the fourth transformation parameter and the fifth transformation parameter, determine the relative transformation error between any two of the sensors.
6. The calibration fixture error elimination method according to claim 5, characterized in that The step of determining the fifth transformation parameter between any two of the sensors based on the co-visibility relationship between any two of the sensors includes: Based on the co-visibility relationship between any two of the sensors, obtain the observation data of any two of the sensors for the same target point; Based on the observation data of the same target point by any two of the sensors, obtain the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two of the sensors. Based on the fourth coordinate data of the target point in the sensor coordinate systems corresponding to any two of the sensors, determine the fifth transformation parameter between any two of the sensors.
7. The calibration fixture error elimination method according to any one of claims 1 to 3, characterized in that, The multiple sensors include a first sensor and a second sensor. The first sensor and the second sensor are configured to be disposed in a robot, and during the execution of the calibration jig error elimination method, the relative positions of the first sensor and the second sensor are fixed.
8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the calibration jig error elimination method according to any one of claims 1 to 7 above.
9. An electronic device, characterized in that, Comprising: A processor; A memory for storing instructions executable by the processor; The processor is configured to execute the calibration jig error elimination method according to any one of claims 1 to 7 above.
10. A computer program product, characterized in that, The computer program product includes instructions, and when the instructions are executed on an electronic device, the electronic device is caused to implement the calibration jig error elimination method according to any one of claims 1 to 7.