Calibration board, multi-sensor joint calibration method, system and application

Abstract

The invention designs a calibration plate, and a calibration method and system using the calibration plate. The calibration plate comprises an infrared calibration area, a visible light calibration area and a laser strong reflection area. The laser strong reflection area is formed by arranging two long and straight laser reflection areas in a right angle, and the infrared calibration area and the visible light calibration area are located in a right-angle area defined by the two long and straight laser reflection areas; the infrared calibration area is marked by a heating two-dimensional code, and the visible light calibration area is marked by a visible light two-dimensional code. The invention designs a calibration plate which can be identified by a laser radar, a visible light camera and an infrared camera at the same time, two long and straight areas fit an X axis and a Y axis, a Z axis is determined by using a right hand rule, then a space coordinate system can be rapidly determined, and multiple sensor coordinate systems can be linked by using one calibration plate in the calibration process.

Description

Technical Field

[0001] The present invention relates to the field of robot perception application technology, and in particular to a calibration board, a multi-sensor joint calibration method, a system, and an application. Background Art

[0002] Multi-source heterogeneous sensor fusion is common in today's robot perception application scenarios. Accurate multi-sensor time and space unification is a prerequisite for multi-sensor fusion perception and positioning. Among them, time unification is mainly achieved through time hard synchronization or time soft synchronization, but in practical applications, sensors often have delays. The delay difference between different sensors will cause time offset. It is important to determine the constant offset between the measurement instance and the timestamp, that is, time calibration. If you want to fuse data from different sensors into a reference frame, spatial calibration is essential.

[0003] Visible light cameras, infrared cameras, lidars, and inertial measurement units (IMUs) are widely used in humanoid robots. They are usually equipped with multiple cameras and lidars to improve the overall situational awareness of the system. The above sensors are usually concentrated in the head. The overall perception of humanoid robots requires the fusion of head and torso sensors, so the extrinsic parameters of the head and torso also need to be calibrated. The calibration of these sensors is very cumbersome. The main reasons are: (1) Existing calibration schemes usually only use a single calibration target, so it is difficult to calibrate all sensors at the same time, especially when the sensors have non-overlapping fields of view. Hand-eye calibration is a common scheme for calibrating non-overlapping field of view sensors. Its accuracy is lower than that of direct feature matching methods and has large errors. (2) Existing calibration schemes provide calibration for infrared camera-IMU, lidar-IMU, and camera-IMU, but few frameworks can jointly calibrate lidar, visible light camera, infrared camera, and IMU, and there is no extrinsic calibration for the head and torso of humanoid robots.

[0004] Chinese patent publication number CN117630892A proposes a joint calibration method for visible light cameras, infrared cameras and laser radars. By setting a rectangular calibration plate and refrigerating it, the two-dimensional coordinates of the corner points of the calibration plate in the visible light image, the two-dimensional coordinates in the infrared image and the three-dimensional coordinates in the laser radar point cloud data at different positions and angles are collected, and the rotation and translation matrix of the coordinate system is solved to complete the joint calibration. This solution can calibrate visible light cameras, infrared cameras and laser radars at the same time, but the operation is cumbersome, the calibration plate needs to be refrigerated in advance, and the IMU cannot be calibrated.

[0005] Chinese patent publication number CN105701827A proposes a joint calibration method for a visible light camera and an infrared camera. The method performs edge detection on visible light images and infrared images to obtain visible light edge images and infrared edge images, uses the scale-invariant feature transformation algorithm SIFT to obtain matching point pairs of the visible light edge image and the infrared edge image, and determines the external parameter matrix based on the matching point pairs, the internal parameter matrix of the visible light camera, and the internal parameter matrix of the infrared camera. This scheme obtains external parameters by pairing the feature points of the visible light and infrared edge images, but when the visible light camera and the infrared camera do not have overlapping fields of view, calibration cannot be performed, and feature points are prone to mismatch, resulting in large calibration errors. Summary of the invention

[0006] The technical problem to be solved by the present invention is the joint spatiotemporal calibration of multi-source heterogeneous sensors in a robot with a large execution end span, and the existing calibration board cannot quickly determine the spatial coordinate system.

[0007] The present invention solves the above technical problems through the following technical means:

[0008] The calibration plate comprises an infrared calibration area, a visible light calibration area and a laser strong reflection area; the laser strong reflection area is two long straight laser reflection areas arranged at right angles, and the infrared calibration area and the visible light calibration area are located in the right angle area surrounded by the two long straight laser reflection areas; the infrared calibration area is marked with a heating two-dimensional code. The present invention designs a calibration plate that can be recognized by a laser radar, a visible light camera and an infrared camera at the same time, the two long straight areas are fitted with X and Y axes, the Z axis is determined by the right-hand rule, and then the spatial coordinate system can be quickly determined, and a plurality of sensor coordinate systems can be linked using one calibration plate during the calibration process.

[0009] Furthermore, the ribbon is made of laser reflective sticker.

[0010] The present invention also provides a multi-sensor joint calibration method using the above calibration plate, comprising the following steps:

[0011] Visible light-infrared camera external parameter calibration: The infrared camera and the visible light camera establish a relative posture relationship with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area;

[0012] The laser radar extracts the strong reflection laser point cloud from the laser strong radiation area, extracts the strong reflection point of the calibration plate from the laser point cloud, and fits the straight line to obtain the X and Y axes of the Cartesian coordinate system. The Z axis is determined by the right-hand rule, thereby establishing the position and posture relationship between the laser radar and the calibration plate.

[0013] Then, the calibration plate information observed by the robot at different positions during the movement is extracted, the current posture state is calculated, the angular velocity and acceleration are calculated and compared with the angular velocity and acceleration measured by the IMU, so as to connect the IMU with other sensors to obtain the external parameters of the robot's head and torso.

[0014] Furthermore, the visible light and infrared camera calibration method is: when the visible light camera and the infrared camera observe the calibration plate, the relative position relationship between the calibration plate and the camera coordinate system is expressed as:

[0015]

[0016] Where W is the world coordinate system, B is the body coordinate system, A is the calibration plate coordinate system, and C is the infrared / visible light camera coordinate system. In this paper, the body coordinate system B is set to coincide with the visible light camera coordinate system; T is the position relationship between the coordinate systems, R is the rotation matrix, and t is the translation vector; T C,B is the relative position of the camera and the body coordinate system, T w,B is the relative position between the world coordinate system and the body coordinate system, d c is the camera time delay, T W,A is the relative pose between the world coordinate system and the calibration plate coordinate system. The visible light camera and the infrared camera can obtain the pose relationship with the calibration plate respectively. Therefore, the relative pose T of the visible light camera and the infrared camera can be calculated. By collecting multiple frames of data during the motion process, the external parameter residual term can be constructed:

[0017]

[0018] Among them, C1 and C2 represent visible light camera and infrared camera respectively. With R, and t represent the relative poses solved at different times, log(·) is the operation of mapping the rotation matrix to its corresponding rotation vector, and λ is a proportional factor that balances the rotation and translation errors.

[0019] Furthermore, the calibration method of the visible light camera and IMU is as follows: the visible light camera calculates its own posture by identifying the features on the calibration plate. However, the camera can only perform discrete sampling during continuous motion, and it is necessary to fit the B-spline curve to obtain the continuous-time camera posture, and then obtain the camera motion angular velocity and acceleration by differentiation. Therefore, the extrinsic parameter residual term of the camera and IMU is constructed by the angular velocity and acceleration:

[0020]

[0021] Where I is the IMU coordinate system, The IMU measures angular velocity and acceleration. is the relative position of IMU and camera, bω is the IMU angular velocity bias, b a is the IMU acceleration bias, d I is the IMU time delay, and g is the gravitational acceleration.

[0022] Furthermore, the calibration method of the visible light camera and the lidar is as follows: the external parameter calibration of the lidar and the visible light camera is linked to the calibration QR code through a strong reflective tape, and the straight line is fitted through the point cloud with high reflection intensity to obtain the pose relationship between the calibration plate and the lidar. Since the straight line fitting has errors, it is only used for rough calibration to provide initial values ​​for fine calibration. The pose relationship between the lidar and the camera can be expressed as:

[0023]

[0024] Among them, L is the laser radar coordinate system. Multiple sets of relative postures T are calculated at different times during the robot movement. The residual term is constructed according to the posture error to obtain the optimal initial external parameter. After solving the initial posture relationship, the point cloud data reflected by the calibration plate is extracted from the point cloud, and the residual is constructed according to the point cloud depth. The point cloud depth calculation formula is:

[0025]

[0026] in is the normal vector of the calibration plate, p i is the laser point cloud coordinate, d L is the time delay of the laser radar. The depth of the laser point cloud is calculated by the above method, and the residual term is constructed with the point cloud depth error output by the radar.

[0027] Furthermore, the method for calibrating the head sensor extrinsic parameters is as follows: combining the above visible light-infrared camera extrinsic parameters, IMU-visible light camera extrinsic parameters, and lidar-visible light camera extrinsic parameter residual constraints to construct a least squares optimization problem:

[0028] y=argmin{∑r I +∑r C +∑r L}

[0029] r I 、r C 、r L They are the residual terms of the external parameters of the visible light camera, infrared camera, IMU, and lidar. They are solved using the LM optimization method to obtain the optimal external parameters between each sensor.

[0030] Furthermore, the head and torso external parameter calibration method is as follows: the humanoid robot torso calculates the pose through the fusion of IMU, encoder, etc., and the head camera pose is calculated through the calibration board. k With t k+1The head and torso positions at the moment are T c,k , T c,k+1 and T I,k , T I,k+1 , by solving the equation:

[0031]

[0032] By obtaining the pose matrix X of the head and torso, the external parameters of the head and torso of the humanoid robot can be obtained.

[0033] The present invention also provides a multi-sensor joint calibration system, comprising:

[0034] Visible light-infrared camera external parameter calibration module, the infrared camera and the visible light camera establish a relative posture relationship with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area;

[0035] The position and posture calibration module of the laser radar and the calibration plate. The laser radar extracts the strong reflection laser point cloud from the laser strong radiation area, extracts the strong reflection point of the calibration plate from the laser point cloud, and fits the straight line to obtain the Cartesian coordinate system X and Y axes. The Z axis is determined by the right-hand rule, so as to establish the position and posture relationship between the laser radar and the calibration plate.

[0036] The robot external parameter calibration module obtains the calibration plate information observed at different positions of the robot during movement, calculates the current posture state, calculates the angular velocity and acceleration, and compares them with the angular velocity and acceleration measured by the IMU, thereby connecting the IMU with other sensors to obtain the external parameters of the robot's head and torso.

[0037] Furthermore, the present invention also provides application of a multi-sensor joint calibration system.

[0038] The advantages of the present invention are:

[0039] (1) Special calibration plate: A calibration plate that can be recognized by lidar, visible light camera, and infrared camera at the same time is designed. Two long straight areas fit the X and Y axes, and the Z axis is determined by the right-hand rule, and then the spatial coordinate system can be quickly determined. During the calibration process, one calibration plate can be used to link the coordinate systems of multiple sensors.

[0040] (2) Multi-sensor calibration scheme: Taking the visible light camera as the benchmark, the residual constraints between other sensors on the humanoid robot head and the camera are designed, the least squares optimization problem is constructed, and the multi-sensor optimization extrinsic parameters are solved;

[0041] (3) Based on the idea of ​​hand-eye calibration, the head pose and torso pose of the humanoid robot are combined to solve the external parameters of the head and torso.

[0042] (4) The proposed calibration scheme can calibrate all sensors of the humanoid robot at one time, greatly reducing the calibration steps and data sampling time. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The design drawing of the calibration plate in Example 1 of the present invention;

[0044] Figure 2 This is a flow chart of the method of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Example 1

[0047] This embodiment introduces a special calibration plate: Figure 1 As shown in , the calibration board is square and contains three areas: a visible light calibration area, an infrared calibration area, and a laser strong reflection area. The visible light calibration area is marked with an ordinary QR code, the infrared calibration area is marked with a heating QR code, and the laser strong reflection area uses two long straight areas at right angles, and the two long straight areas are covered with laser reflective stickers. The two long straight areas fit the X and Y axes of the coordinate system, and the right-angle area enclosed can be regarded as the first quadrant of the coordinate system. The visible light calibration area and the infrared calibration area are located in the first quadrant. In this embodiment, the infrared calibration area is located above the visible light calibration area. The visible light calibration area is used to extract features from images taken by a visible light camera, the infrared calibration area is used to extract features from infrared images, and the laser strong reflection area is used to extract strong reflection laser point clouds. The calibration board contains the feature recognition parts of the three sensors at the same time, which can link the sensor coordinate systems.

[0048] Example 2

[0049] Based on the calibration plate of Example 1, this embodiment provides a method for joint calibration of multiple sensors: multi-sensor calibration requires linking different sensors, the infrared camera and the visible light camera establish a relative posture relationship with the calibration plate by recognizing the QR code, extract the strong reflection point of the calibration plate from the laser point cloud to fit the straight line to obtain the X and Y axes of the Cartesian coordinate system, determine the Z axis by the right-hand rule, and the origin is the intersection of the X and Y axes, thereby establishing the posture relationship between the laser radar and the calibration plate. Extract the calibration plate information observed by the humanoid robot at different positions during the movement, calculate the current posture state, calculate the angular velocity and acceleration, and compare them with the angular velocity and acceleration measured by the IMU, so as to link the IMU with other sensors.

[0050] The method of this embodiment is suitable for a robot having multiple execution ends with a large distance and span, such as a humanoid robot including a head and a torso, or a mobile robot with a mechanical arm and a mobile part. This embodiment is described by taking a humanoid robot as an example.

[0051] The above sensors are generally located on the head of the humanoid robot. The external parameters of the head and torso are calibrated through the hand-eye calibration method. The head sensor calculates its own posture through the calibration board, and the torso obtains its own posture through the fusion positioning of other sensors such as IMU and encoder. The AX=XB equation is established to solve the external parameters between the two.

[0052] The following is a detailed introduction to the specific calibration method:

[0053] First, define W as the world coordinate system, B as the body coordinate system, A as the calibration plate coordinate system, L as the lidar coordinate system, I as the IMU coordinate system, and C as the (infrared / visible light) camera coordinate system. In this paper, the body coordinate system B is set to coincide with the visible light camera coordinate system. T is the pose relationship between the coordinate systems, R is the rotation matrix, and t is the translation vector.

[0054] (1) Calibration of visible light and infrared cameras: When the visible light camera and infrared camera observe the calibration plate, the relative position relationship between the calibration plate and the camera coordinate system can be expressed as:

[0055]

[0056] Where T C,B is the relative position of the camera and the body coordinate system, T w,B is the relative position between the world coordinate system and the body coordinate system, d c is the camera time delay, T W,A is the relative pose between the world coordinate system and the calibration plate coordinate system. The visible light camera and the infrared camera can obtain the pose relationship with the calibration plate respectively. Therefore, the relative pose T of the visible light camera and the infrared camera can be calculated. By collecting multiple frames of data during the motion process, the external parameter residual term can be constructed:

[0057]

[0058] Among them, C1 and C2 represent visible light camera and infrared camera respectively. With R, and t represent the relative poses solved at different times, log(·) is the operation of mapping the rotation matrix to its corresponding rotation vector, and λ is a scaling factor that balances the rotation and translation errors.

[0059] (2) Calibration of visible light camera and IMU: The visible light camera calculates its own pose by identifying the features on the calibration plate. However, the camera can only perform discrete sampling during continuous motion. It is necessary to fit the B-spline curve to obtain the continuous-time camera pose, and then obtain the camera motion angular velocity and acceleration by differentiation. Therefore, the extrinsic parameter residual term of the camera and IMU is constructed by the angular velocity and acceleration:

[0060]

[0061] in The IMU measures angular velocity and acceleration. is the relative position of IMU and camera, b ω is the IMU angular velocity bias, b a is the IMU acceleration bias, d I is the IMU time delay, and g is the gravitational acceleration.

[0062] (3) Calibration of visible light camera and lidar: The external parameter calibration of the lidar and camera is linked to the calibration QR code through a strong reflective tape. The pose relationship between the calibration plate and the lidar is obtained by fitting a straight line through the point cloud with high reflection intensity. Since the straight line fitting has errors, it is only used for rough calibration to provide initial values ​​for fine calibration. The pose relationship between the lidar and the camera can be expressed as:

[0063]

[0064] During the movement of the humanoid robot, multiple sets of relative postures T are calculated at different times, and the residual term is constructed based on the posture error to obtain the optimal initial external parameter. After solving the initial posture relationship, the point cloud data reflected by the calibration plate is extracted from the point cloud, and the residual is constructed based on the point cloud depth. The point cloud depth calculation formula is:

[0065]

[0066] in is the normal vector of the calibration plate, p i is the laser point cloud coordinate, d Lis the time delay of the laser radar. The depth of the laser point cloud is calculated by the above method, and the residual term is constructed with the point cloud depth error output by the radar.

[0067] (4) Head sensor extrinsic calibration: Based on the residual constraints of the above visible light-infrared camera extrinsic parameters, IMU-visible light camera extrinsic parameters, and lidar-visible light camera extrinsic parameters, a least squares optimization problem is constructed:

[0068] y=argmin{∑r I +∑r C +∑r L}

[0069] r I 、r C 、r L They are the residual terms of the external parameters of the visible light camera, infrared camera, IMU, and lidar. They are solved using the LM optimization method to obtain the optimal external parameters between each sensor.

[0070] (5) Calibration of head and torso external parameters (spatial position relationship of several sensors): The humanoid robot torso position is calculated by fusion of IMU, encoder, etc., and the head camera position is calculated by calibration board. k With t k+1 The head and torso positions at the moment are T c,k , T c,k+1 and T I,k , T I,k+1 , by solving the equation:

[0071]

[0072] By obtaining the pose matrix X of the head and torso, the external parameters of the head and torso of the humanoid robot can be obtained.

[0073] The design principle of this embodiment is as follows:

[0074] (1) Special calibration plate: A calibration plate is designed that can be recognized by lidar, visible light camera, and infrared camera at the same time. During the calibration process, one calibration plate can be used to link the coordinate systems of multiple sensors.

[0075] (2) Multi-sensor calibration scheme: Taking the visible light camera as the benchmark, the residual constraints between other sensors on the humanoid robot head and the camera are designed, the least squares optimization problem is constructed, and the multi-sensor optimization extrinsic parameters are solved;

[0076] (3) Based on the idea of ​​hand-eye calibration, the head pose and torso pose of the humanoid robot are combined to solve the external parameters of the head and torso.

[0077] (4) The proposed calibration scheme can calibrate all sensors of the humanoid robot at one time, greatly reducing the calibration steps and data sampling time.

[0078] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. Calibration plate, characterized in that: It includes an infrared calibration area, a visible light calibration area, and a laser strong reflection area; the laser strong reflection area is two long straight laser reflection areas arranged at right angles, and the infrared calibration area and the visible light calibration area are located in the right angle area surrounded by the two long straight laser reflection areas; the infrared calibration area is marked with a heat-generating two-dimensional code, and the visible light calibration area is marked with a visible light two-dimensional code.

2. The calibration plate according to claim 1, characterized in that: The ribbon adopts laser reflective sticker.

3. A multi-sensor joint calibration method using the calibration plate according to claim 1 or 2, characterized in that: The following steps are involved: Visible light-infrared camera external parameter calibration: The infrared camera and the visible light camera establish a relative posture relationship with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area; The laser radar extracts the strong reflection laser point cloud from the laser strong radiation area, extracts the strong reflection point of the calibration plate from the laser point cloud, and fits the straight line to obtain the X and Y axes of the Cartesian coordinate system. The Z axis is determined by the right-hand rule, thereby establishing the position and posture relationship between the laser radar and the calibration plate. Then, the calibration plate information observed by the robot at different positions during the movement is extracted, the current posture state is calculated, the angular velocity and acceleration are calculated and compared with the angular velocity and acceleration measured by the IMU, so as to connect the IMU with other sensors to obtain the external parameters of the robot's head and torso.

4. The calibration method according to claim 3, characterized in that: The calibration method for visible light and infrared cameras is as follows: when the visible light camera and infrared camera observe the calibration plate, the relative position relationship between the calibration plate and the camera coordinate system is expressed as: Where W is the world coordinate system, B is the body coordinate system, A is the calibration plate coordinate system, and C is the infrared / visible light camera coordinate system. In this paper, the body coordinate system B is set to coincide with the visible light camera coordinate system; T is the position relationship between the coordinate systems, R is the rotation matrix, and t is the translation vector; T C,B is the relative position of the camera and the body coordinate system, T w,B is the relative position between the world coordinate system and the body coordinate system, d c is the camera time delay, T W,A is the relative pose between the world coordinate system and the calibration plate coordinate system. The visible light camera and the infrared camera can obtain the pose relationship with the calibration plate respectively. Therefore, the relative pose T of the visible light camera and the infrared camera can be calculated. By collecting multiple frames of data during the motion process, the external parameter residual term can be constructed: Among them, C1 and C2 represent visible light camera and infrared camera respectively. With R, and t represent the relative poses solved at different times, log(·) is the operation of mapping the rotation matrix to its corresponding rotation vector, and λ is a proportional factor that balances the rotation and translation errors.

5. The calibration method according to claim 4, characterized in that: The calibration method of the visible light camera and IMU is as follows: the visible light camera calculates its own posture by identifying the features on the calibration plate. However, the camera can only perform discrete sampling during continuous motion, so it is necessary to fit the B-spline curve to obtain the continuous-time camera posture, and then obtain the camera motion angular velocity and acceleration by differentiation. Therefore, the extrinsic parameter residual term of the camera and IMU is constructed by the angular velocity and acceleration: Where I is the IMU coordinate system, The IMU measures angular velocity and acceleration. is the relative position of IMU and camera, b ω is the IMU angular velocity bias, b a is the IMU acceleration bias, d I is the IMU time delay, and g is the gravitational acceleration.

6. The calibration method according to claim 5, characterized in that: The calibration method of visible light camera and lidar is as follows: the external parameter calibration of lidar and visible light camera is linked to the calibration QR code through strong reflective tape, and the straight line is fitted through the point cloud with high reflection intensity to obtain the pose relationship between the calibration plate and the lidar. Since the straight line fitting has errors, it is only used for rough calibration to provide initial values ​​for fine calibration. The pose relationship between the lidar and the camera can be expressed as: Among them, L is the laser radar coordinate system. Multiple sets of relative postures T are calculated at different times during the robot movement. The residual term is constructed according to the posture error to obtain the optimal initial external parameter. After solving the initial posture relationship, the point cloud data reflected by the calibration plate is extracted from the point cloud, and the residual is constructed according to the point cloud depth. The point cloud depth calculation formula is: in is the normal vector of the calibration plate, p i is the laser point cloud coordinate, d L is the time delay of the laser radar. The depth of the laser point cloud is calculated by the above method, and the residual term is constructed with the point cloud depth error output by the radar.

7. The calibration method according to claim 6, characterized in that: The method for calibrating the head sensor extrinsic parameters is as follows: Combining the above visible light-infrared camera extrinsic parameters, IMU-visible light camera extrinsic parameters, and lidar-visible light camera extrinsic parameter residual constraints, construct a least squares optimization problem: y=argmin{∑r I +∑r C +∑r L } r I 、r C 、r L They are the residual terms of the external parameters of the visible light camera, infrared camera, IMU, and lidar. They are solved using the LM optimization method to obtain the optimal external parameters between each sensor.

8. The calibration method according to claim 7, characterized in that: The head and torso external parameter calibration method is as follows: the humanoid robot torso calculates the pose through the fusion of IMU, encoder, etc., and the head camera pose is calculated through the calibration board. k With t k+1 The head and torso positions at the moment are T c,k , T c,k+1 and T I,k , T I,k+1 , by solving the equation: By obtaining the pose matrix X of the head and torso, the external parameters of the head and torso of the humanoid robot can be obtained. 9.Multi-sensor joint calibration system, characterized in that: include: Visible light-infrared camera external parameter calibration module, the infrared camera and the visible light camera establish a relative posture relationship with the calibration plate by respectively identifying the QR codes of the infrared calibration area and the visible light calibration area; The position and posture calibration module of the laser radar and the calibration plate. The laser radar extracts the strong reflection laser point cloud from the laser strong radiation area, extracts the strong reflection point of the calibration plate from the laser point cloud, and fits the straight line to obtain the Cartesian coordinate system X and Y axes. The Z axis is determined by the right-hand rule, so as to establish the position and posture relationship between the laser radar and the calibration plate. The robot external parameter calibration module obtains the calibration plate information observed at different positions of the robot during movement, calculates the current posture state, calculates the angular velocity and acceleration, and compares them with the angular velocity and acceleration measured by the IMU, thereby connecting the IMU with other sensors to obtain the external parameters of the robot's head and torso.

10. Application of the multi-sensor joint calibration system as claimed in claim 9.

Images