Calibration board, multi-sensor joint calibration method, system, and application

Abstract

The present invention designs a calibration plate, as well as a calibration method and system using the calibration plate. The calibration plate includes an infrared calibration area, a visible light calibration area, and a strong laser reflection area. The strong laser reflection area is composed of two long straight laser reflection areas arranged at right angles, and the infrared calibration area and the visible light calibration area are located within the right-angled area enclosed by the two long straight laser reflection areas. The infrared calibration area is marked with a heat-generating QR code, and the visible light calibration area is marked with a visible light QR code. The present invention designs a calibration plate that can be simultaneously recognized by lidar, visible light cameras, and infrared cameras. The two long straight areas are fitted with the X and Y axes, and the Z axis is determined using the right-hand rule, which can then quickly determine the spatial coordinate system. During the calibration process, using a single calibration plate can link the coordinate systems of multiple sensors.

Description

Technical Field

[0001] The present invention relates to the field of robot perception application technology, and in particular to a calibration plate, a multi-sensor joint calibration method, a system, and an application. Background Art

[0002] Multi-source heterogeneous sensor fusion is common in today's robotics perception applications. Accurate multi-sensor temporal and spatial alignment is a prerequisite for multi-sensor fusion perception and localization. Temporal alignment is primarily achieved through hard or soft time synchronization. However, in real-world applications, sensor latency often varies, and differences in latency between different sensors can lead to time offsets. Therefore, maintaining a constant offset between measurement instances and timestamps—that is, time calibration—is crucial. Spatial calibration is essential for fusing data from different sensors into a single reference frame.

[0003] Visible light cameras, infrared cameras, lidars, and inertial measurement units (IMUs) are widely used in humanoid robots. They are usually equipped with multiple cameras and lidars to improve the overall situational awareness of the system. These sensors are usually concentrated in the head. Humanoid robots need to integrate head and torso sensors for overall perception, so the external parameters of the head and torso also need to be calibrated. The calibration of these sensors is very cumbersome. The main reasons are: (1) Existing calibration schemes usually only use a single calibration target, so it is difficult to calibrate all sensors at the same time, especially when the sensors have non-overlapping fields of view. Hand-eye calibration is a common solution for calibrating non-overlapping field of view sensors. Its accuracy is lower than that of direct feature matching methods and the error is larger. (2) Existing calibration schemes provide calibration for infrared camera-IMU, lidar-IMU, and camera-IMU, but few frameworks can jointly calibrate lidar, visible light camera, infrared camera, and IMU, and there is no external parameter calibration for the head and torso of humanoid robots.

[0004] Chinese patent publication CN117630892A proposes a joint calibration method for visible light cameras, infrared cameras, and lidar. This method involves setting up a rectangular calibration plate and refrigerating it. The system then collects the 2D coordinates of the plate's corner points in the visible light image, the 2D coordinates in the infrared image, and the 3D coordinates in the lidar point cloud data at different positions and angles. The coordinate system rotation and translation matrices are then calculated to complete the joint calibration. This method can simultaneously calibrate visible light cameras, infrared cameras, and lidar, but it is cumbersome, requires the calibration plate to be refrigerated in advance, and cannot calibrate an IMU.

[0005] Chinese patent publication CN105701827A proposes a joint calibration method for visible light and infrared cameras. This method performs edge detection on visible light and infrared images to obtain visible edge images and infrared edge images. The scale-invariant feature transform (SIFT) algorithm is then used to obtain matching point pairs between the visible and infrared edge images. The extrinsic parameter matrix is ​​then determined based on the matching point pairs, the intrinsic parameter matrix of the visible light camera, and the intrinsic parameter matrix of the infrared camera. This scheme obtains extrinsic parameters by pairing feature points in the visible and infrared edge images. However, calibration is impossible when the visible light and infrared cameras do not have overlapping fields of view, and feature points are prone to mismatching, resulting in large calibration errors. Summary of the Invention

[0006] The technical problem to be solved by the present invention is the joint spatiotemporal calibration of multi-source heterogeneous sensors in a robot with a large execution end span. The existing calibration board cannot quickly determine the spatial coordinate system.

[0007] The present invention solves the above technical problems through the following technical means:

[0008] The calibration plate includes an infrared calibration area, a visible light calibration area, and a laser strong reflection area; the laser strong reflection area is two long straight laser reflection areas arranged at right angles, and the infrared calibration area and the visible light calibration area are located within the right-angle area surrounded by the two long straight laser reflection areas; the infrared calibration area is marked with a heat-generating QR code. The present invention designs a calibration plate that can be simultaneously recognized by a lidar, a visible light camera, and an infrared camera. The two long straight areas are fitted with the X and Y axes, and the Z axis is determined using the right-hand rule, thereby quickly determining the spatial coordinate system. During the calibration process, a single calibration plate can be used to link multiple sensor coordinate systems.

[0009] Furthermore, the laser strong reflection area adopts laser reflection sticker.

[0010] The present invention also provides a multi-sensor joint calibration method using the above-mentioned calibration plate, comprising the following steps:

[0011] Visible light-infrared camera external parameter calibration: The infrared camera and visible light camera establish relative pose relationships with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area;

[0012] The laser radar extracts a strongly reflected laser point cloud from the laser strong radiation area, extracts the strong reflection points of the calibration plate from the laser point cloud, and fits the straight line to obtain the Cartesian coordinate system X and Y axes. The Z axis is determined by the right-hand rule, thereby establishing the position relationship between the laser radar and the calibration plate;

[0013] Then, the calibration plate information observed by the robot at different positions during movement is extracted, the current posture state is calculated, and the angular velocity and acceleration are compared with the angular velocity and acceleration measured by the IMU. By linking the IMU with other sensors, the external parameters of the robot's head and torso can be obtained.

[0014] Furthermore, the visible light and infrared camera calibration method is as follows: when the visible light camera and the infrared camera observe the calibration plate, the relative position relationship between the calibration plate and the camera coordinate system is expressed as:

[0015]

[0016] Where W is the world coordinate system, B is the body coordinate system, A is the calibration plate coordinate system, and C is the infrared / visible light camera coordinate system. In this paper, the body coordinate system B is set to coincide with the visible light camera coordinate system; T is the pose relationship between the coordinate systems, R is the rotation matrix, and t is the translation vector; is the relative pose between the camera and the body coordinate system, is the relative pose between the world coordinate system and the body coordinate system, is the camera time delay, is the relative pose between the world coordinate system and the calibration plate coordinate system. The visible light camera and infrared camera can obtain the pose relationship with the calibration plate respectively. Therefore, the relative pose T of the visible light camera and infrared camera can be calculated. By collecting multiple frames of data during the motion process, the external parameter residual term can be constructed:

[0017]

[0018] Among them, C1 and C2 represent visible light camera and infrared camera respectively. With R, and t represent the relative poses solved at different times, log(·) is the operation of mapping the rotation matrix to its corresponding rotation vector, and λ is a scaling factor that balances the rotation and translation errors.

[0019] Furthermore, the visible light camera and IMU calibration method is as follows: the visible light camera calculates its own pose by identifying the features on the calibration plate. However, the camera can only perform discrete sampling during continuous motion. It is necessary to fit the B-spline curve to obtain the continuous-time camera pose, and then obtain the camera motion angular velocity and acceleration by differentiation. Therefore, the extrinsic parameter residual term of the camera and IMU is constructed by the angular velocity and acceleration:

[0020]

[0021]

[0022] Where I is the IMU coordinate system, 、 Measure angular velocity and acceleration for the IMU, is the relative pose between IMU and camera, is the IMU angular velocity bias, is the IMU acceleration bias, is the IMU time delay, is the acceleration due to gravity.

[0023] Furthermore, the visible light camera and lidar calibration method is as follows: the external parameter calibration of the lidar and visible light camera is connected to the calibration QR code through a strong reflective tape. The pose relationship between the calibration plate and the lidar is obtained by fitting a straight line through the point cloud with high reflection intensity. Since the straight line fitting has errors, it is only used for coarse calibration to provide initial values ​​for fine calibration. The pose relationship between the lidar and camera can be expressed as:

[0024]

[0025] Where L is the laser radar coordinate system. Multiple sets of relative poses T are calculated at different times during the robot's motion. The residual term is constructed based on the pose error to obtain the optimal initial extrinsic parameter. After solving the initial pose relationship, the point cloud data reflected by the calibration plate is extracted from the point cloud. The residual is constructed based on the point cloud depth. The point cloud depth calculation formula is:

[0026]

[0027] in is the normal vector of the calibration plate, is the laser point cloud coordinate, is the time delay of the laser radar. The depth of the laser point cloud is calculated by the above method, and the residual term is constructed with the point cloud depth error output by the radar.

[0028] Furthermore, the method for calibrating the head sensor extrinsic parameters is as follows: combining the residual constraints of the above-mentioned visible light-infrared camera extrinsic parameters, IMU-visible light camera extrinsic parameters, and lidar-visible light camera extrinsic parameters, constructing a least squares optimization problem:

[0029]

[0030] 、 、 They are the residual terms of the external parameters of the visible light camera, infrared camera, IMU, and lidar, respectively. Using the LM optimization method to solve them, we can get the optimized external parameters between each sensor.

[0031] Furthermore, the head and torso external parameter calibration method is as follows: the humanoid robot torso is calculated by fusing IMU, encoder, etc., and the head camera pose is calculated by the calibration board. kWith t k+1 The head and torso postures at the moment are T c,k 、T c,k+1 and T I,k 、T I,k+1 , by solving the equation:

[0032]

[0033] By obtaining the pose matrix X of the head and torso, the external parameters of the head and torso of the humanoid robot can be obtained.

[0034] The present invention also provides a multi-sensor joint calibration system, comprising:

[0035] Visible light-infrared camera external parameter calibration module, the infrared camera and visible light camera establish relative posture relationship with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area;

[0036] The LiDAR and calibration plate pose calibration module extracts a strongly reflected laser point cloud from the laser strong radiation area, extracts the calibration plate's strongly reflected points from the laser point cloud, and fits a straight line to obtain the Cartesian coordinate system's X and Y axes. The Z axis is determined by the right-hand rule, thereby establishing the pose relationship between the LiDAR and the calibration plate.

[0037] The robot external parameter calibration module obtains the calibration plate information observed at different positions of the robot during movement, calculates the current posture state, and compares the calculated angular velocity and acceleration with the angular velocity and acceleration measured by the IMU. By linking the IMU with other sensors, the external parameters of the robot's head and torso can be obtained.

[0038] Furthermore, the present invention also provides an application of a multi-sensor joint calibration system.

[0039] The advantages of the present invention are:

[0040] (1) Special calibration plate: A calibration plate that can be recognized by lidar, visible light camera and infrared camera at the same time is designed. Two long straight areas are fitted to the X and Y axes, and the Z axis is determined by the right-hand rule, so that the spatial coordinate system can be quickly determined. During the calibration process, a single calibration plate can be used to link the coordinate systems of multiple sensors.

[0041] (2) Multi-sensor calibration scheme: Using the visible light camera as a benchmark, design the residual constraints between other sensors on the humanoid robot head and the camera, construct a least squares optimization problem, and solve the multi-sensor optimization extrinsic parameters;

[0042] (3) Based on the idea of ​​hand-eye calibration, the head pose and torso pose of the humanoid robot are combined to solve the external parameters of the head and torso.

[0043] (4) The proposed calibration scheme can calibrate all sensors of the humanoid robot at one time, greatly reducing the calibration steps and reducing the data sampling time. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a design drawing of the calibration plate in Example 1 of the present invention;

[0045] Figure 2 This is a flow chart of the method of Example 2 of the present invention. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0047] Example 1

[0048] This embodiment introduces a special calibration plate: Figure 1 As shown in , the calibration plate is square and contains three areas: a visible cursor calibration area, an infrared calibration area, and a laser strong reflection area. The visible cursor calibration area uses an ordinary QR code marker, the infrared calibration area uses a heat-generating QR code marker, and the laser strong reflection area uses two long straight areas at right angles, and the two long straight areas are covered with laser reflective stickers. The two long straight areas fit the X and Y axes of the coordinate system, and the right-angled area enclosed can be regarded as the first quadrant of the coordinate system. The visible cursor calibration area and the infrared calibration area are located in the first quadrant. In this embodiment, the infrared calibration area is located above the visible cursor calibration area. The visible cursor calibration area is used to extract features from images taken by the visible light camera, the infrared calibration area is used to extract features from infrared images, and the laser strong reflection area is used to extract strong reflection laser point clouds. The calibration plate contains the feature recognition parts of the three sensors at the same time, which can link the sensor coordinate systems.

[0049] Example 2

[0050] Based on the calibration plate of Example 1, this embodiment provides a method for multi-sensor joint calibration: Multi-sensor calibration requires linking different sensors. The infrared camera and visible light camera establish a relative pose relationship with the calibration plate by recognizing the QR code. The strong reflection points of the calibration plate are extracted from the laser point cloud and fitted with straight lines to obtain the Cartesian coordinate system X and Y axes. The Z axis is determined by the right-hand rule, with the origin being the intersection of the X and Y axes, thereby establishing the pose relationship between the lidar and the calibration plate. The calibration plate information observed by the humanoid robot at different positions during movement is extracted, the current pose state is calculated, and the angular velocity and acceleration are calculated and compared with the angular velocity and acceleration measured by the IMU, thereby linking the IMU with other sensors.

[0051] The method of this embodiment is suitable for robots with multiple actuators with large distances and spans, such as humanoid robots with a head and torso, or mobile robots with mechanical arms and mobile parts. This embodiment is described using a humanoid robot as an example.

[0052] The above sensors are generally located on the head of a humanoid robot. The external parameters of the head and torso are calibrated using the hand-eye calibration method. The head sensor calculates its own position and posture through the calibration board. The torso obtains its own position and posture through the fusion positioning of other sensors such as the IMU and encoder. The extrinsic parameters between the two are calculated by establishing the equation AX=XB.

[0053] The following is a detailed introduction to the specific calibration method:

[0054] First, define W as the world coordinate system, B as the body coordinate system, A as the calibration plate coordinate system, L as the lidar coordinate system, I as the IMU coordinate system, and C as the (infrared / visible light) camera coordinate system. In this article, the body coordinate system B is assumed to coincide with the visible light camera coordinate system. T is the pose relationship between the coordinate systems, R is the rotation matrix, and t is the translation vector.

[0055] (1) Calibration of visible light and infrared cameras: When the visible light camera and infrared camera observe the calibration plate, the relative position relationship between the calibration plate and the camera coordinate system can be expressed as:

[0056]

[0057] in is the relative pose between the camera and the body coordinate system, is the relative pose between the world coordinate system and the body coordinate system, is the camera time delay, is the relative pose between the world coordinate system and the calibration plate coordinate system. The visible light camera and infrared camera can obtain the pose relationship with the calibration plate respectively. Therefore, the relative pose T of the visible light camera and infrared camera can be calculated. By collecting multiple frames of data during the motion process, the external parameter residual term can be constructed:

[0058]

[0059] Among them, C1 and C2 represent visible light camera and infrared camera respectively. With R, and t represent the relative poses solved at different times, log(·) is the operation of mapping the rotation matrix to its corresponding rotation vector, and λ is a scaling factor that balances the rotation and translation errors.

[0060] (2) Calibration of visible light camera and IMU: The visible light camera calculates its own pose by identifying the features on the calibration plate. However, the camera can only perform discrete sampling during continuous motion. It is necessary to fit the B-spline curve to obtain the continuous-time camera pose, and then obtain the camera motion angular velocity and acceleration by differentiation. Therefore, the extrinsic parameter residual term of the camera and IMU is constructed by the angular velocity and acceleration:

[0061]

[0062]

[0063] in 、 Measure angular velocity and acceleration for the IMU, is the relative pose between IMU and camera, is the IMU angular velocity bias, is the IMU acceleration bias, is the IMU time delay, is the acceleration due to gravity.

[0064] (3) Calibration of visible light camera and lidar: The external parameter calibration of lidar and camera is linked to the calibration QR code through a strong reflective tape. The pose relationship between the calibration plate and lidar is obtained by fitting a straight line through the point cloud with high reflection intensity. Since the straight line fitting has errors, it is only used for coarse calibration and provides initial values ​​for fine calibration. The pose relationship between lidar and camera can be expressed as:

[0065]

[0066] During the motion of the humanoid robot, multiple sets of relative poses T are calculated at different times. The residual term is constructed based on the pose error to obtain the optimal initial extrinsic parameter. After solving the initial pose relationship, the point cloud data reflected by the calibration plate is extracted from the point cloud, and the residual is constructed based on the point cloud depth. The point cloud depth calculation formula is:

[0067]

[0068] in is the normal vector of the calibration plate, is the laser point cloud coordinate, is the time delay of the laser radar. The depth of the laser point cloud is calculated by the above method, and the residual term is constructed with the point cloud depth error output by the radar.

[0069] (4) Head sensor extrinsic parameter calibration: Based on the residual constraints of the visible light-infrared camera extrinsic parameters, IMU-visible light camera extrinsic parameters, and lidar-visible light camera extrinsic parameters, a least squares optimization problem is constructed:

[0070]

[0071] 、 、 They are the residual terms of the external parameters of the visible light camera, infrared camera, IMU, and lidar, respectively. Using the LM optimization method to solve them, we can get the optimized external parameters between each sensor.

[0072] (5) Calibration of external parameters of head and torso (spatial position relationship of several sensors): The humanoid robot torso pose is calculated by fusion of IMU, encoder, etc., and the head camera pose is calculated by calibration board. k With t k+1 The head and torso postures at the moment are T c,k 、T c,k+1 and T I,k 、T I,k+1 , by solving the equation:

[0073]

[0074] By obtaining the pose matrix X of the head and torso, the external parameters of the head and torso of the humanoid robot can be obtained.

[0075] The design principles of this embodiment are as follows:

[0076] (1) Special calibration plate: A calibration plate is designed that can be recognized by lidar, visible light camera and infrared camera at the same time. During the calibration process, one calibration plate can be used to link the coordinate systems of multiple sensors.

[0077] (2) Multi-sensor calibration scheme: Using the visible light camera as a benchmark, design the residual constraints between other sensors on the humanoid robot head and the camera, construct a least squares optimization problem, and solve the multi-sensor optimization extrinsic parameters;

[0078] (3) Based on the idea of ​​hand-eye calibration, the head pose and torso pose of the humanoid robot are combined to solve the external parameters of the head and torso.

[0079] (4) The proposed calibration scheme can calibrate all sensors of the humanoid robot at one time, greatly reducing the calibration steps and reducing the data sampling time.

[0080] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. Calibration plate, characterized in that, It includes an infrared calibration area, a visible light calibration area, and a laser strong reflection area; the laser strong reflection area is two long straight laser reflection areas arranged at right angles, and the infrared calibration area and the visible light calibration area are located in the right angle area surrounded by the two long straight laser reflection areas; the infrared calibration area is marked with a heat-generating QR code, and the visible light calibration area is marked with a visible light QR code.

2. The calibration plate according to claim 1, characterized in that: The laser strong reflection area adopts laser reflection sticker.

3. A multi-sensor joint calibration method using the calibration plate according to claim 1 or 2, characterized in that: The following steps are involved: Visible light-infrared camera external parameter calibration: The infrared camera and visible light camera establish relative pose relationships with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area; The laser radar extracts a strongly reflected laser point cloud from the laser strong radiation area, extracts the strong reflection points of the calibration plate from the laser point cloud, and fits the straight line to obtain the Cartesian coordinate system X and Y axes. The Z axis is determined by the right-hand rule, thereby establishing the position relationship between the laser radar and the calibration plate; Then, the calibration plate information observed by the robot at different positions during movement is extracted, the current posture state is calculated, and the angular velocity and acceleration are compared with the angular velocity and acceleration measured by the IMU. The IMU is then linked to the visible light-infrared camera and lidar to obtain the external parameters of the robot's head and torso.

4. The calibration method according to claim 3, characterized in that: The calibration method for visible light and infrared cameras is as follows: when the visible light camera and infrared camera observe the calibration plate, the relative position relationship between the calibration plate and the camera coordinate system is expressed as: Where W is the world coordinate system, B is the body coordinate system, A is the calibration plate coordinate system, and C is the infrared / visible light camera coordinate system. In this paper, the body coordinate system B is set to coincide with the visible light camera coordinate system; T is the pose relationship between the coordinate systems, R is the rotation matrix, and t is the translation vector; is the relative pose between the camera and the body coordinate system, is the relative pose between the world coordinate system and the body coordinate system, is the camera time delay, is the relative pose between the world coordinate system and the calibration plate coordinate system. The visible light camera and infrared camera can obtain the pose relationship with the calibration plate respectively. Therefore, the relative pose T of the visible light camera and infrared camera can be calculated. By collecting multiple frames of data during the motion process, the external parameter residual term can be constructed: Among them, C1 and C2 represent visible light camera and infrared camera respectively. With R, and t represent the relative poses solved at different times, log(·) is the operation of mapping the rotation matrix to its corresponding rotation vector, and λ is a scaling factor that balances the rotation and translation errors.

5. The calibration method according to claim 4, characterized in that: The visible light camera and IMU calibration method is as follows: the visible light camera calculates its own pose by identifying features on the calibration plate. However, the camera can only perform discrete sampling during continuous motion. It is necessary to fit the B-spline curve to obtain the continuous-time camera pose, and then obtain the camera motion angular velocity and acceleration through differentiation. Therefore, the extrinsic parameter residual term of the camera and IMU is constructed by the angular velocity and acceleration: Where I is the IMU coordinate system, 、 Measure angular velocity and acceleration for the IMU, is the relative pose between IMU and camera, is the IMU angular velocity bias, is the IMU acceleration bias, is the IMU time delay, is the acceleration due to gravity.

6. The calibration method according to claim 5, characterized in that: The calibration method for the visible light camera and lidar is as follows: the external parameter calibration of the lidar and visible light camera is connected to the calibration QR code through a highly reflective tape. A straight line is fitted through the point cloud with high reflection intensity to obtain the pose relationship between the calibration plate and the lidar. Since the straight line fitting has errors, it is only used for coarse calibration to provide initial values ​​for fine calibration. The pose relationship between the lidar and camera can be expressed as: Where L is the laser radar coordinate system. Multiple sets of relative poses T are calculated at different times during the robot's motion. The residual term is constructed based on the pose error to obtain the optimal initial extrinsic parameter. After solving the initial pose relationship, the point cloud data reflected by the calibration plate is extracted from the point cloud. The residual is constructed based on the point cloud depth. The point cloud depth calculation formula is: in is the normal vector of the calibration plate, is the laser point cloud coordinate, is the time delay of the laser radar. The depth of the laser point cloud is calculated by the above method, and the residual term is constructed with the point cloud depth error output by the radar.

7. The calibration method according to claim 6, characterized in that: The method for calibrating the head sensor extrinsic parameters is to combine the residual constraints of the visible light-infrared camera extrinsic parameters, IMU-visible light camera extrinsic parameters, and lidar-visible light camera extrinsic parameters to construct a least squares optimization problem: 、 、 They are the residual terms of the external parameters of the visible light camera, infrared camera, IMU, and lidar, respectively. Using the LM optimization method to solve them, we can get the optimized external parameters between each sensor.

8. The calibration method according to claim 7, characterized in that: The head and torso external parameter calibration method is as follows: the humanoid robot torso is calculated by IMU and encoder fusion pose, and the head camera pose is calculated by the calibration board. k With t k+1 The head and torso postures at the moment are T c,k 、T c,k+1 and T I,k 、T I,k+1 , by solving the equation: By obtaining the pose matrix X of the head and torso, the external parameters of the head and torso of the humanoid robot can be obtained.

9. Multi-sensor joint calibration system, characterized by: include: Visible light-infrared camera external parameter calibration module, the infrared camera and visible light camera establish relative posture relationship with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area; The LiDAR and calibration plate pose calibration module extracts a strongly reflected laser point cloud from the laser strong radiation area, extracts the calibration plate's strongly reflected points from the laser point cloud, and fits a straight line to obtain the Cartesian coordinate system's X and Y axes. The Z axis is determined by the right-hand rule, thereby establishing the pose relationship between the LiDAR and the calibration plate. The robot's external parameter calibration module obtains the calibration plate information observed at different positions during the robot's movement, calculates the current posture state, and compares the calculated angular velocity and acceleration with the angular velocity and acceleration measured by the IMU. It then links the IMU with the visible light-infrared camera and lidar to obtain the external parameters of the robot's head and torso.

10. Application of the multi-sensor joint calibration system according to claim 9.

Images