Low-cost and high-precision robot multi-component calibration method based on consumer camera

Abstract

The invention discloses a robot multi-component low-cost high-precision calibration method based on a consumer-level camera, and the method comprises the steps: photographing a calibration plate through the consumer-level camera, and optimizing the internal parameters of the camera through corner detection and nonlinear optimization; a camera is installed at the tail end of the mechanical arm, about 50 sets of calibration plate images are shot, the pose is obtained by means of forward kinematics of the mechanical arm, and the pose relation between the camera and the mechanical arm is analyzed through the formula AX = XB; then the chassis and the camera on the mechanical arm are used for joint calibration, the calibration relation between the mechanical arm and the trolley is solved by moving the trolley and the mechanical arm observation calibration plate, and finally the calibration precision is verified by calculating the re-projection error. The method does not need to depend on high-precision equipment and a professional calibration field, calibration can be completed by using a common consumer camera and a simple calibration board, the method can adapt to various cameras, the requirement for computing equipment is not high, and the requirement can be met by a common workstation. The method can be carried out in a dynamic scene, the calibration time is within 1 hour, and the precision is within 5 mm.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a low-cost and high-precision calibration method for multiple components of a robot based on a consumer-grade camera. Background Art

[0002] The evolution of robot multi-component calibration technology:

[0003] Traditional geometric modeling stage (1990s to 2000s)

[0004] Static benchmark calibration: Using equipment such as laser trackers and coordinate measuring machines, a fixed reference point is established between the base of the robotic arm and the chassis, allowing direct measurement of their relative position. For example, NASA's 2002 Mars rover, Sojourner, used a combination of a laser rangefinder at the end of the robotic arm and an IMU on the chassis to achieve static positioning on the Martian surface. However, this method is not adaptable to dynamic scenarios.

[0005] Motion constraint methods utilize geometric constraints on the robot's trajectory and chassis movement to establish a pose transformation model. For example, a variation of the Tsai two-step method is used to move the robot. This method uses multi-point pose data of the robot's end in the chassis coordinate system to solve the relative transformation matrix between the base and chassis. This method is suitable for AGV-mounted robot arms, but relies on fixed markers and has poor dynamic adaptability.

[0006] Dynamic parameter identification stage (2000s~2010s)

[0007] Algebraic matrix decomposition: This method converts the pose relationship between the manipulator and chassis into a homogeneous transformation equation and uses SVD decomposition to find a linear solution. This method is generally used for dynamic pose synchronization between the manipulator and the mobile platform, such as in satellite in-orbit maintenance scenarios. However, this method requires equation establishment and the assumption that the axes of rotation are non-parallel, resulting in non-unique solutions.

[0008] Nonlinear optimization: The LM method is used to jointly model the end-of-arm error and the chassis movement trajectory error, and the global optimal parameters are solved through the iterative residual optimization method. However, the computational complexity is high and the real-time performance is poor.

[0009] Existing static benchmark calibration methods rely on fixed equipment and calibration sites, which are expensive, incapable of adapting to dynamic scenarios, require specialized personnel, and suffer from low productivity. Motion constraint methods require fixed markers in the environment and use the relative positions of the robotic arm and these markers to establish a pose transformation model. However, in some practical applications, setting fixed markers may not be convenient, or they may be subject to interference from environmental factors, affecting calibration accuracy. For example, in complex outdoor environments or frequently changing industrial scenarios, fixed markers may be obscured, damaged, or moved, resulting in inaccurate calibration. Algebraic matrix decomposition, which transforms the pose relationship between the robotic arm and chassis into a homogeneous transformation matrix equation and linearizes it, is a simplified model that may ignore some real-world nonlinear factors and errors. In some applications requiring high precision, these ignored factors may lead to significant deviations between the calibration results and the actual pose. Nonlinear optimization: Linear optimization algorithms are often sensitive to initial values. Different initial values ​​may cause the algorithm to converge to different local optimal solutions or even fail to converge to the global optimal solution. In practical applications, determining appropriate initial values ​​is often difficult. Improper initialization can lead to inaccurate calibration results, requiring multiple attempts at different initial values ​​to find the optimal solution. This increases the complexity and time required for calibration. Therefore, a low-cost, high-precision calibration method for multiple robot components based on consumer-grade cameras was designed. Summary of the Invention

[0010] The purpose of the present invention is to provide a low-cost and high-precision calibration method for multiple components of a robot based on a consumer-grade camera, so as to solve the problems raised in the above-mentioned background technology.

[0011] To achieve the above objectives, the present invention provides the following technical solution: a low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera, comprising the following steps:

[0012] A. In-camera parameter optimization;

[0013] B. Analysis of the pose relationship between the camera and the robotic arm;

[0014] C. Calibration of robotic arm and chassis;

[0015] D. Accuracy verification.

[0016] Preferably, the step A specifically comprises: using a normal optical camera to photograph the calibration plate, detecting key points using a corner point detection method in computer vision, and then optimizing the camera internal parameters using a nonlinear optimization method.

[0017] Preferably, the specific working method of step B is: after the camera optimization is completed, the camera is installed on the flange at the end of the robotic arm, and the camera is used to shoot the calibration plate by operating the robotic arm to shoot 50 sets of images. The calibration plate should occupy 40% to 80% of the entire image, so that each joint of the robotic arm can move, so that its pitch angle, roll angle and yaw angle are all rotated by a certain angle, and then the corresponding posture is obtained through the forward kinematics of the robotic arm, and then the posture relationship between the camera and the robotic arm is analyzed using the formula AX=XB.

[0018] Preferably, the AX=XB formula is as follows:

[0019]

[0020] The numbers represent the number of shots. The transformation matrix from the base of the manipulator to the end of the manipulator for the i-th shot is obtained from the forward kinematics of the manipulator; The matrix to be calculated is the transformation matrix from the end of the robotic arm to the camera; The transformation matrix between the camera and the calibration plate is obtained by calibration.

[0021] Preferably, the step C is specifically as follows: after the calibration of the camera and the robotic arm is analyzed and completed, the robotic arm and the chassis are calibrated, and the visual or radar equipment installed on the chassis is used to perform joint calibration with the camera on the robotic arm.

[0022] Preferably, the specific method is as follows:

[0023] 1) Place the calibration plate in a fixed position in the room;

[0024] 2) Move the car to observe the calibration plate from multiple angles. The calibration plate should be observed by the robot arm camera and the chassis camera, occupying 40% to 80% of the image. Then move the robot arm to record multiple sets of data. While driving the car to observe the calibration plate from multiple positions, the calibration relationship between the robot arm and the car is solved by the equation, and the error is optimized through multiple sets. Equation:

[0025]

[0026] in, The transformation relationship between the car's built-in base coordinate system and the robot's base coordinate system;

[0027] The transformation matrix from the car to the car's built-in camera;

[0028] Use the calibration plate to calculate the transformation matrix from the car camera to the calibration plate;

[0029] Use the calibration plate to calculate the transformation matrix from the calibration plate to the camera on the robotic arm;

[0030] Equal to the transformation matrix obtained by hand-eye calibration in the previous step, that is, the transformation matrix from the robotic arm camera to the end of the robotic arm;

[0031] The transformation matrix from the end of the manipulator to the base of the manipulator is obtained through the forward kinematics of the manipulator. Finally, the resulting transformation matrix is ​​obtained by multiple measurements and error smoothing.

[0032] Preferably, the verification method in step D is to calculate the reprojection error, use a calibration plate, change the scene and perspective, add multiple movements of the robotic arm, and use the points on the calibration plate to calculate:

[0033]

[0034] Beneficial effects: The present invention can directly use consumer-grade cameras to complete the task without the use of high-precision equipment; there is no need to fix the calibration object, and the calibration can be completed using a simple calibration plate and a cabinet; the camera only uses a consumer-grade camera, and can be adapted to multiple cameras without changing the algorithm; Realsense, hp60 cameras, etc. can be used for calibration; the present invention does not need to use a fixed calibration field to avoid being restricted by the site, but uses SLAM technology to calculate the position and posture of the car, and then the calibration results of each structure can be settled; and the car and the calibration plate do not need to be stationary, the car can run in a dynamic scene, as long as "the same calibration plate is observed" at the same time; no professional engineer is required for calibration, only the calibration plate needs to appear in the middle, and complete shooting from multiple angles can be done, which effectively reduces labor; no powerful computing equipment is required, and a normal working station can complete the calibration; the calibration time is within 1 hour, the accuracy is within 5mm, and the camera intrinsic calibration accuracy is within 1 pixel. In actual applications, users can complete the calibration process without human intervention, which is particularly suitable for mobile platform scenarios such as unmanned driving, robot navigation, AR / VR, etc. Compared with traditional calibration processes that rely on specific patterns and static environments, this method is more flexible and versatile, and can significantly reduce deployment costs and technical barriers.

[0035] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to more clearly understand the technical means of the embodiments of the present application, they can be implemented according to the contents of the specification. In addition, in order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of the present invention;

[0037] Figure 2Schematic diagram of the analytical relationship between the camera and the robotic arm of the present invention;

[0038] Figure 3 This is a schematic diagram of the calibration of the robotic arm and chassis of the present invention. DETAILED DESCRIPTION

[0039] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification, claims and drawings of this application are intended to cover non-exclusive inclusions.

[0041] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0042] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0043] See also Figure 1-Figure 3 The present invention provides the following technical solution: a low-cost and high-precision calibration method for multiple components of a robot based on a consumer-grade camera, comprising the following steps:

[0044] A. In-camera parameter optimization;

[0045] B. Analysis of the pose relationship between the camera and the robotic arm;

[0046] C. Calibration of robotic arm and chassis;

[0047] D. Accuracy verification.

[0048] In the present invention, step A specifically includes: using a normal optical camera to shoot a calibration plate, detecting key points using a corner point detection method in computer vision, and then optimizing the camera internal parameters using a nonlinear optimization method.

[0049] In the present invention, the specific working method of step B is: after the camera optimization is completed, the camera is installed on the flange at the end of the robotic arm, and the camera is used to shoot the calibration plate by operating the robotic arm to shoot 50 sets of images. The calibration plate should occupy 40% to 80% of the entire image, so that each joint of the robotic arm can move, so that its pitch angle, roll angle and yaw angle are all rotated by a certain angle, and then the corresponding posture is obtained through the forward kinematics of the robotic arm, and then the posture relationship between the camera and the robotic arm is analyzed using the formula AX=XB.

[0050] Preferably, the AX=XB formula is as follows:

[0051]

[0052] The numbers represent the number of shots. The transformation matrix from the base of the manipulator to the end of the manipulator for the i-th shot is obtained from the forward kinematics of the manipulator; The matrix to be calculated is the transformation matrix from the end of the robotic arm to the camera; The transformation matrix between the camera and the calibration plate is obtained by calibration.

[0053] In the present invention, step C is specifically as follows: after the calibration of the camera and the robotic arm is completed, the robotic arm and the chassis are calibrated, and the visual or radar equipment installed on the chassis is used to perform joint calibration with the camera on the robotic arm.

[0054] The specific method is as follows:

[0055] 1) Place the calibration plate in a fixed position in the room;

[0056] 2) Move the car to observe the calibration plate from multiple angles. The calibration plate should be observed by the robot arm camera and the chassis camera, occupying 40% to 80% of the image. Then, move the robot arm to record multiple sets of data. While driving the car to observe the calibration plate from multiple positions, the calibration relationship between the robot arm and the car is solved by the equation, and the error is optimized through multiple sets.

[0057] equation:

[0058]

[0059] in, The transformation relationship between the car's built-in base coordinate system and the robot's base coordinate system;

[0060] The transformation matrix from the car to the car's built-in camera;

[0061] Use the calibration plate to calculate the transformation matrix from the car camera to the calibration plate;

[0062] Use the calibration plate to calculate the transformation matrix from the calibration plate to the camera on the robotic arm;

[0063] Equal to the transformation matrix obtained by hand-eye calibration in the previous step, that is, the transformation matrix from the robotic arm camera to the end of the robotic arm;

[0064] The transformation matrix from the end of the manipulator to the base of the manipulator is obtained through the forward kinematics of the manipulator. Finally, the resulting transformation matrix is ​​obtained by multiple measurements and error smoothing.

[0065] In the present invention, the verification method in step D is to calculate the reprojection error, use a calibration plate, change the scene and perspective, add multiple movements of the robotic arm, and use the points on the calibration plate to calculate:

[0066]

[0067] Example 1:

[0068] Indoor calibration experiment:

[0069] Prepare equipment: consumer-grade camera, robotic arm, mobile car, and ordinary checkerboard calibration plate.

[0070] Camera intrinsic parameter optimization: Use the camera to capture images of the checkerboard calibration plate at different angles, extract the corner points using a computer vision corner detection algorithm, and use a nonlinear optimization algorithm to optimize the camera intrinsic parameters to obtain the camera's intrinsic parameter matrix.

[0071] Analyzing the pose relationship between the camera and the robotic arm: Mount the optimized camera on the flange at the end of the robotic arm. Operate the robotic arm so that the camera captures the calibration plate. Capture 50 images, ensuring that the calibration plate occupies 40% to 80% of the image and that the robotic arm joints have a certain degree of motion. The transformation matrix from the base of the robotic arm to the end of the arm for each image is obtained through the forward kinematics of the robotic arm. Using the transformation matrix between the camera and the calibration plate obtained through calibration, the pose relationship between the camera and the robotic arm is analyzed using the formula AX = XB to obtain the transformation matrix from the robotic arm end to the camera.

[0072] Robotic arm and chassis calibration: Fix the calibration plate in a certain location in the room and move the robot so that both the robot arm camera and the chassis camera can see the calibration plate, and the calibration plate occupies 40% to 80% of the image. Move the robot arm and record multiple sets of data while simultaneously driving the robot to observe the calibration plate from multiple positions. Using the above formula and multiple measurement data, solve the transformation matrix from the robot's built-in base coordinate system to the robot arm's base coordinate system by optimizing the error.

[0073] Accuracy verification: Using a calibration plate, changing the scene and perspective, moving the robotic arm, and calculating the reprojection error, the result was 0.853 pixels.

[0074] Example 2:

[0075] Outdoor calibration experiment:

[0076] Prepare equipment: consumer-grade camera, robotic arm, mobile car, and ordinary checkerboard calibration plate.

[0077] Follow the steps in Example 1 to optimize the camera's internal parameters and analyze the position relationship between the camera and the robotic arm.

[0078] Robotic arm and chassis calibration: Fix the calibration plate at a specific location outdoors and move the robot so that both the robot arm camera and the chassis camera can observe the calibration plate. Move the robot arm and record multiple sets of data, while driving the robot to observe the calibration plate from multiple positions. Use the formula to solve the calibration relationship between the robot and the robot arm.

[0079] Accuracy verification: In an outdoor environment, the reprojection error was calculated and the result was 0.865 pixels.

[0080] In summary, the present invention can directly use consumer-grade cameras to complete the task without the use of high-precision equipment; there is no need to fix the calibration object, and calibration can be completed using a simple calibration plate and a cabinet; the camera only uses a consumer-grade camera, can be adapted to multiple cameras, and the algorithm does not need to be changed; Realsense, hp60 cameras, etc. can be used for calibration; the present invention does not need to use a fixed calibration field to avoid being restricted by the site, but uses SLAM technology to calculate the position and posture of the car, and then the calibration results of each structure can be settled; and the car and the calibration plate do not need to be stationary, the car can run in a dynamic scene, as long as "the same calibration plate is observed" at the same time; no professional engineer is required for calibration, only the calibration plate needs to appear in the middle, and complete shooting from multiple angles can be done, which effectively reduces labor; no powerful computing equipment is required, and a normal working station can complete the calibration; the calibration time is within 1 hour, the accuracy is within 5mm, and the camera intrinsic calibration accuracy is within 1 pixel. In actual application, users can complete the calibration process without human intervention, which is particularly suitable for mobile platform scenarios such as unmanned driving, robot navigation, AR / VR, etc. Compared with traditional calibration processes that rely on specific patterns and static environments, this method is more flexible and versatile, and can significantly reduce deployment costs and technical barriers.

[0081] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A low-cost, high-precision calibration method for multiple robot components based on consumer-grade cameras, characterized by: The following steps are involved: A. In-camera parameter optimization; B. Analysis of the pose relationship between the camera and the robotic arm; C. Calibration of robotic arm and chassis; D. Accuracy verification.

2. The low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera according to claim 1, characterized in that: The step A specifically includes: using a normal optical camera to photograph the calibration plate, detecting key points using a corner detection method in computer vision, and then optimizing the camera internal parameters using a nonlinear optimization method.

3. The low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera according to claim 1, characterized in that: The specific working method of step B is as follows: after the camera optimization is completed, the camera is installed on the flange at the end of the robotic arm, and the camera is used to shoot the calibration plate by operating the robotic arm. 50 sets of images are taken. The calibration plate should occupy 40% to 80% of the entire image, so that each joint of the robotic arm can move, and its pitch angle, roll angle and yaw angle are all rotated by a certain angle. Then, the corresponding posture is obtained through the forward kinematics of the robotic arm, and then the posture relationship between the camera and the robotic arm is analyzed using the formula AX=XB.

4. The low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera according to claim 3, characterized in that: The AX=XB formula is as follows: The numbers represent the number of shots. The transformation matrix from the base of the manipulator to the end of the manipulator for the i-th shot is obtained from the forward kinematics of the manipulator; The matrix to be calculated is the transformation matrix from the end of the robotic arm to the camera; The transformation matrix between the camera and the calibration plate is obtained by calibration.

5. The low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera according to claim 1, characterized in that: The specific step C is as follows: after the calibration of the camera and the robotic arm is completed, the robotic arm and the chassis are calibrated, and the visual or radar equipment installed on the chassis is used to perform joint calibration with the camera on the robotic arm.

6. The low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera according to claim 5, characterized in that: The specific method is as follows: 1) Place the calibration plate in a fixed position in the room; 2) Move the car to observe the calibration plate from multiple angles. The calibration plate should be observed by the robot arm camera and the chassis camera, occupying 40% to 80% of the image. Then move the robot arm to record multiple sets of data. While driving the car to observe the calibration plate from multiple positions, the calibration relationship between the robot arm and the car is solved by the equation. Through multiple sets of optimization errors, the equation is: in, The transformation relationship between the car's built-in base coordinate system and the robot's base coordinate system; The transformation matrix from the car to the car's built-in camera; Use the calibration plate to calculate the transformation matrix from the car camera to the calibration plate; Use the calibration plate to calculate the transformation matrix from the calibration plate to the camera on the robotic arm; Equal to the transformation matrix obtained by hand-eye calibration in the previous step, that is, the transformation matrix from the robotic arm camera to the end of the robotic arm; The transformation matrix from the end of the manipulator to the base of the manipulator is obtained through the forward kinematics of the manipulator. Finally, the resulting transformation matrix is ​​obtained by multiple measurements and error smoothing. The low-cost, high-precision calibration method for multiple components of a robot based on a consumer-grade camera according to claim 1 is characterized in that: The verification method in step D is to calculate the reprojection error, use a calibration plate, transform the scene and perspective, add multiple movements of the robotic arm, and use the points on the calibration plate to calculate: Proj_Error: reprojection error; Board_point: calibration point; Transformation matrix from board to carcamera, transformation from calibration board to camera on car The transformation matrix from the carcamera car camera to the carbase vehicle coordinate system; The transformation matrix from the carbase vehicle-based coordinate system to the armbase robotic arm-based coordinate system; The transformation matrix of the armbase robot base coordinate system to the armcamera robot arm camera; The transformation matrix of the armcamera robotic arm camera to the transformation matrix of the calibration plate.

Images