Multi-sensor calibration method for mobile robots

Abstract

The application discloses a kind of multi-sensor calibration methods for mobile robot, mobile robot includes 2D laser radar, upper view depth camera and lower view depth camera, it is the calibration method for upper view depth camera and lower view depth camera, it specifically includes the following steps: S1, setting calibration environment;S2, setting mobile robot;S3, obtaining raw data;S4, upper view depth camera calibration;S5, lower view depth camera calibration.The beneficial effects of the application are: not dependent on the precision and installation precision of fixture, to calibrate the depth camera of mobile robot, align the point cloud of depth camera with laser radar point cloud and ensure the time efficiency of calculation.

Description

Technical Field

[0001] This invention belongs to the field of robotics technology, specifically relating to a multi-sensor calibration method for mobile robots. Background Technology

[0002] Most commercial mobile robots are equipped with several depth cameras to perceive 3D information in their environment. Depth cameras measure the position of objects in the environment by emitting and receiving infrared beams. In practice, some sensors are used to detect suspended obstacles, i.e., tabletops, while others are used to detect low obstacles on the ground. Existing calibration methods require fixing the robot to a fixture and using infrared images to calibrate the depth camera's pose, which introduces many disadvantages to calibration and places high demands on the manufacturing accuracy of the fixture and the installation accuracy of the robot on the fixture. The technical problem this invention aims to solve is to design a fixtureless depth camera calibration method for mobile robots using a low-cost embedded computing unit, aligning the depth camera's point cloud with the LiDAR point cloud while ensuring computational time efficiency. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-sensor calibration method for mobile robots that solves or partially solves the above-mentioned problems.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A multi-sensor calibration method for a mobile robot, the mobile robot including a 2D LiDAR, an upward-looking depth camera, and a downward-looking depth camera, comprising the following steps:

[0006] S1. Set up the calibration environment: including the ground, a middle calibration plate perpendicular to the ground, and two side calibration plates perpendicular to the ground; both side calibration plates are perpendicular to the middle calibration plate; the middle calibration plate and the two side calibration plates together form a U-shaped structure;

[0007] S2. Setting up the mobile robot: Place the mobile robot on the ground, between the intermediate calibration plate and the two side calibration plates, ensuring that the 2D LiDAR can see the intermediate calibration plate and the two side calibration plates, and that the upper-view depth camera and the lower-view depth camera can both see the intermediate calibration plate and at least one of the side calibration plates; create a robot coordinate system with the mobile robot as the origin, and camera coordinate systems with the upper-view depth camera and the lower-view depth camera as the origins respectively;

[0008] S3. Obtain raw data: The upper-view depth camera and the lower-view depth camera take pictures to obtain raw point clouds, and the 2D lidar scans the middle calibration plate and the two side calibration plates and extracts the contours therein;

[0009] S4. Calibration of the upward-looking depth camera: The upward-looking depth camera is calibrated according to the following steps:

[0010] S4-1. Obtain calibration board point cloud: For the original point cloud obtained by the upward-looking depth camera, first identify and remove the ground point cloud to obtain the calibration board point cloud about the middle calibration board and the side calibration board.

[0011] S4-2, Roll and Pitch Angle Calibration: The roll and pitch angles of the calibration plate point cloud are successively fine-tuned to generate multiple first sample point clouds; all first sample point clouds are compared to obtain the first sample point cloud that is most perpendicular to the ground, which is used as the first standard sample point cloud.

[0012] S4-3. Contour Alignment: The yaw angle and the offsets of the X and Y axes of the first standard sample point cloud are successively fine-tuned to obtain multiple second sample point clouds. The contours of the second sample point clouds are aligned with the contours obtained by the 2D LiDAR. The second sample point cloud with the highest contour matching degree is selected as the second standard sample point cloud, and its roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes are used as the calibrated roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes.

[0013] S5. Calibration of the downward-looking depth camera: The upward-looking depth camera is calibrated according to the following steps:

[0014] S5-1. Obtaining ground point cloud: For the original point cloud obtained by the downward-looking depth camera, first identify the ground point cloud in it to obtain the ground point cloud.

[0015] S5-2, Roll and Pitch Angle Calibration: The roll and pitch angles of the ground point cloud are successively fine-tuned to generate multiple third-sample point clouds; the normal vector V of each third-sample point cloud is calculated. c Select the one that is closest to the ground normal vector V. g The third sample point cloud is used as the third standard sample point cloud, where V g = (0, 0, 1) T T denotes matrix transpose;

[0016] S5-3, Contour Alignment: The yaw angle and the offsets of the X and Y axes of the third standard sample point cloud are successively fine-tuned to obtain multiple fourth sample point clouds. The contours of the fourth sample point clouds are aligned with the contours obtained by the 2D LiDAR. The fourth sample point cloud with the highest contour matching degree is selected as the fourth standard sample point cloud, and its roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes are used as the calibrated roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes.

[0017] Preferably, in step S3, a high-resolution mesh map is first created in the robot coordinate system, and then the outlines of the intermediate calibration plate and the two side calibration plates obtained by the 2D LiDAR scan are depicted in the high-resolution mesh map.

[0018] Preferably, in step S4-2, a planar grid parallel to the ground is first created, then the first sample point cloud is projected onto the planar grid, and then the following steps are used to select the first sample point cloud with the fewest grids that have the projection of the first sample point cloud as the first standard sample point cloud:

[0019] The z-component is extracted from the first sample point cloud, and the first sample point cloud is projected onto the planar grid. The first sample point cloud is then scored by calculating the number of grid indices occupied; point p = [x, y]. T The grid index I occupied g For: I g =ceil(y-y0) / res)×num x +ceil((xx o ) / res), where the number of grid cells in the x-direction is num. x The number of grid cells in the y direction is num y The origin of the planar grid is Origin. m =[x o y o ] T res represents the resolution, and ceil represents rounding up the numerical value.

[0020] Preferably, in step S3, the original point cloud is obtained from the depth images captured by the upward-looking depth camera and the downward-looking depth camera, and is transformed through projection transformation:

[0021] p d =(x d y d , z d ) T p s =(xs y s 1 ) T

[0022] Where p d It is a point transformed into the camera coordinate system of the upward-viewing depth camera and the downward-viewing depth camera, where D represents the depth of that point, and f x It is the focal length of the upward-viewing depth camera and the downward-viewing depth camera on the x-axis, f y It is the focal length of the upward-viewing depth camera and the downward-viewing depth camera on the y-axis, p s It is the original point in the depth image, x d It is the x-axis coordinate in the camera coordinate system, y d It is the y-axis coordinate in the camera coordinate system, z d It is the z-axis coordinate in the camera coordinate system, T is, x S It is the x-axis index value within the depth image, y s It is the y-axis index value within the depth image.

[0023] Preferably, in step S3, the original point cloud obtained by the upward-looking depth camera and the downward-looking depth camera is first converted into a point cloud set in the robot coordinate system, which is as follows:

[0024] Where S d This refers to either the upward-viewing depth camera or the downward-viewing depth camera. This is a homogeneous transformation from the upward-viewing depth camera or the downward-viewing depth camera to the robot; To transform the points into the robot's coordinate system.

[0025] Preferably, in step S4-2, the first sample point cloud includes a roll sample and a pitch sample, and the point cloud set S of the roll sample... roll for:

[0026] S roll ={r|r∈[-r max r max ]}, re s r = 2r max / n r

[0027] Where, r max It is the maximum offset of the roll, res r It is the resolution of the sample, n r It represents the number of samples, where r is a tumbling sample;

[0028] The rotation matrix R of the tumbling sample r for:

[0029] Preferably, the point cloud set S of the pitch sample pitch for:

[0030] S pitc h={p|p∈[-p max p max ]}, re s p = 2p max / n p

[0031] Where, p max It is the maximum pitch offset, res p It is the resolution of the sample, n p The number of samples is p, where p is a pitch angle sample; the rotation matrix R of the pitch sample is... p for:

[0032]

[0033] Preferably, the roll samples and pitch samples constitute the first sample point cloud, and the candidate pose is transformed by left multiplication T. c Transform the point cloud set in the robot coordinate system to obtain point cloud T on the first sample point cloud. C : t = (0, 0, 0) T Where t is the point cloud translation amount;

[0034] The point cloud set S on the first sample point cloud C For: S c ={p c |p c =Tc×p r p r ∈Sr}, where p C It is S C Inner point cloud, p r It is the sensor coordinate system, S r It is a collection of point clouds of the original robot.

[0035] Preferably, in step S5-2, the largest plane detected by the region growing method is taken as the ground surface, and the normal vector V of the ground surface is obtained. cp Then by calculating V cp and V g The angle difference θ between them c To score the pose candidates, θ c The smaller the value, the closer it is to the ground normal vector V. g This yields the roll and pitch angles adjusted by the downward-facing camera, where...

[0036]

[0037] Preferably, in step S4-3, after obtaining the first standard sample point cloud, the point cloud set S of the first standard sample point cloud is... rp Adjust to be perpendicular to the ground; for point cloud set S rp For each point in the graph, remove the z-component to create a two-dimensional point cloud; adjust the yaw and position of the two-dimensional point cloud to match the high-resolution mesh graph.

[0038] A second sample point cloud T y From yaw samples and location sample t p =(x p y p ) T Composition, where y is the yaw angle, x p It is the x-axis offset, y p It is the y-axis offset;

[0039] The yaw sample set is: S yaw ={y|y∈[-y max y max ]}, res y =2y max / n y ;y max It is the maximum yaw angle, n y It represents the number of yaw angle samples, res y This is the yaw angle sample resolution;

[0040] The second sample point cloud T y Transform into a new set S yp :

[0041] S yp ={p yp |p yp =T yp ×p rp p rp ∈S rp}, Where, p yp It is the point for adjusting the heading and translation, T yp It is a homogeneous transformation of the heading angle and translation, p rp It is s rp The point s in rp It is the point cloud set of the first standard sample point cloud, R y It is the heading angle rotation matrix, t p It is the translation amount;

[0042] The second sample point cloud T y It contains one rotated sample and one translated sample; the set of rotated samples is S.rot ;

[0043] p rot =[x rot y rot ] T , where p rot It is a point that is rotated only, x rot It is p rot The X-axis component, y rot It is p rot The Y-axis component;

[0044] S rot The point cloud in the image is projected onto the high-resolution mesh map, corresponding to the index. The set is:

[0045] in, It is the index value within the map, and res is the resolution;

[0046] A set of point cloud indices is displayed in the high-resolution mesh map;

[0047] Translate samples by moving to Add an integer to the index to translate and rotate the point cloud of the sample;

[0048] The translation set of rotation indices is:

[0049] Among them, S trans It is a translation set, p trans It is the translation point, x offset It is p trans X-axis component, y offset It is p trans Y-axis component, Z is a set of integers, x max It is the maximum offset on the X-axis, and the y-axis is the maximum offset on the X-axis. max It is the maximum offset on the Y-axis;

[0050] The set of sample indices translated by a rotation is:

[0051]

[0052] The second sample point cloud T y The corresponding index set in the high-resolution mesh graph;

[0053] The second sample point cloud T y The score is the gray value of the corresponding index i of the point cloud projected in the high-resolution mesh map. The sum of; therefore, the score s of the second sample point cloud.c for:

[0054] Select the highest score s c The second sample point cloud is used as the second standard sample point cloud.

[0055] The beneficial effects of this invention are: it calibrates the depth camera of a mobile robot without relying on the precision of the fixture and the installation precision, aligns the point cloud of the depth camera with the point cloud of the lidar, and ensures the time efficiency of the calculation. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the principle of the present invention;

[0057] Figure 2 This is a schematic diagram illustrating the principle of using a high-resolution mesh graph. Detailed Implementation

[0058] The technical solution of this patent will be further described in detail below with reference to specific embodiments.

[0059] In the description of this invention, it should be noted that the terms "inner", "outer", "upper", "lower", "horizontal", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0060] like Figure 1 and Figure 2 As shown, this invention provides a multi-sensor calibration method for a mobile robot, which includes a 2D LiDAR, an upward-looking depth camera, and a downward-looking depth camera. The method is specifically designed for calibrating the upward-looking depth camera and the downward-looking depth camera, and includes the following steps:

[0061] S1. Set up the calibration environment: including the ground, a middle calibration plate perpendicular to the ground, and two side calibration plates perpendicular to the ground; both side calibration plates are perpendicular to the middle calibration plate; the middle calibration plate and the two side calibration plates together form a U-shaped structure;

[0062] S2. Set up the mobile robot: Place the mobile robot on the ground between the central calibration plate and the two side calibration plates, ensuring that the 2D LiDAR can see the central calibration plate and the two side calibration plates, and that both the top-view depth camera and the bottom-view depth camera can see the central calibration plate and at least one side calibration plate; create a robot coordinate system with the mobile robot as the origin, and camera coordinate systems with the top-view depth camera and the bottom-view depth camera as the origins respectively;

[0063] S3. Obtain raw data: Take pictures with the top-view depth camera and the bottom-view depth camera to obtain raw point cloud, and use 2D LiDAR to scan the middle calibration plate and two side calibration plates and extract their contours.

[0064] S4. Calibrate the upward-looking depth camera. Follow these steps to calibrate the upward-looking depth camera:

[0065] S4-1. Obtain the calibration board point cloud: For the original point cloud obtained by the upward-looking depth camera, first identify the ground point cloud and remove it to obtain the calibration board point cloud about the middle calibration board and the side calibration board.

[0066] S4-2, Roll and Pitch Angle Calibration: The roll and pitch angles of the calibration board point cloud are successively fine-tuned to generate multiple first sample point clouds; all first sample point clouds are compared to obtain the first sample point cloud that is most perpendicular to the ground, which is used as the first standard sample point cloud.

[0067] S4-3. Contour Alignment: Successively fine-tune the yaw angle and the offsets of the X and Y axes of the first standard sample point cloud to obtain multiple second sample point clouds. Align the contours of the second sample point clouds with the contours obtained by the 2D LiDAR. Select the second sample point cloud with the highest contour matching degree and use it as the second standard sample point cloud. Use its roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes as the calibrated roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes.

[0068] S5. Calibrate the downward-looking depth camera. Follow the steps below to calibrate the upward-looking depth camera:

[0069] S5-1. Obtaining Ground Point Cloud: For the raw point cloud obtained by the downward-looking depth camera, first identify the ground point cloud to obtain the ground point cloud.

[0070] S5-2, Roll and Pitch Angle Calibration: The roll and pitch angles of the ground point cloud are successively fine-tuned to generate multiple third-sample point clouds; the normal vector V of each third-sample point cloud is calculated. c Select the one that is closest to the ground normal vector V. g The third sample point cloud is used as the third standard sample point cloud, where V g = (0, 0, 1) T T denotes matrix transpose;

[0071] S5-3, Contour Alignment: Successively fine-tune the yaw angle and the offsets of the X and Y axes of the third standard sample point cloud to obtain multiple fourth sample point clouds. Align the contours of the fourth sample point clouds with the contours obtained by the 2D LiDAR. Select the fourth sample point cloud with the highest contour matching degree and use it as the fourth standard sample point cloud. Use its roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes as the calibrated roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes.

[0072] In step S3, a high-resolution mesh map (also known as a high-precision map) is first created in the robot coordinate system. Then, the outlines of the intermediate calibration plate and two side calibration plates obtained from the 2D LiDAR scan are depicted in the high-resolution mesh map. The resolution of the high-resolution mesh map is res (meters) / grid. The map origin is Origin = (x o y o ) T The grayscale value of the grid hit by the LiDAR is 100, and the grayscale value of the remaining grids is... It is 0.

[0073] In step S4-2, a planar grid parallel to the ground is first created. Then, the first sample point cloud is projected onto the planar grid. The following steps are then used to select the first sample point cloud with the fewest grid cells as the first standard sample point cloud:

[0074] For mobile robots, the z-component in point cloud information has little impact on obstacle avoidance. Adjustments to the point cloud position are limited to the x and y axes. Therefore, the z-component is extracted from the first sample point cloud, projected onto a planar grid, and then scored by calculating the number of grid indices occupied. Point p = [x, y] T The grid index I occupied g For: I g =ceil((y-y0) / res)×num x +ceil((xx o ) / res), where the number of grid cells in the x-direction is num. x The number of grid cells in the y direction is num y The origin of the planar grid is Origin. m =[x o y o ] T res represents the resolution, and ceil represents rounding up the numerical value.

[0075] In step S3, the original point cloud is obtained from the depth images captured by the top-view depth camera and the bottom-view depth camera, and is transformed through projection transformation:

[0076] p d =(x d y d , z d ) T p s =(x s y s ,1) T

[0077] Where p d It is a point transformed into the camera coordinate system of the upward-viewing depth camera and the downward-viewing depth camera, where D represents the depth of that point, and f x It is the focal length of the upward-viewing depth camera and the downward-viewing depth camera on the x-axis, f y It is the focal length of the upward-viewing depth camera and the downward-viewing depth camera on the y-axis, p s It is the original point in the depth image, x d It is the x-axis coordinate in the camera coordinate system, y d It is the y-axis coordinate in the camera coordinate system, z d It is the z-axis coordinate in the camera coordinate system, T is, x S It is the x-axis index value within the depth image, y s It is the y-axis index value within the depth image.

[0078] In step S3, the original point cloud obtained by the upward-looking depth camera and the downward-looking depth camera is first converted into a point cloud set in the robot coordinate system, which is as follows:

[0079] Where S d It is either a top-view depth camera or a bottom-view depth camera. This is a homogeneous transformation from an upward-looking depth camera or a downward-looking depth camera to the robot. To transform the points into the robot's coordinate system.

[0080] In step S4-2, the first sample point cloud includes a roll sample and a pitch sample, and the point cloud set S of the roll sample... roll for:

[0081] S roll ={r|r∈[-r max r max ]}, res r =2r mmax / n r

[0082] Where, r max It is the maximum offset of the roll, res r It is the resolution of the sample, n rIt represents the number of samples, where r is a tumbling sample;

[0083] Rotation matrix R of the tumbling sample r for:

[0084] Point cloud set S of pitch samples pitch for:

[0085] S pitch ={p|p∈[-p]} max p max ]}, res p =2p max / n p

[0086] Where, p max It is the maximum pitch offset, res p It is the resolution of the sample, n p It represents the number of samples, where p is a pitch angle sample; R is the rotation matrix of the pitch sample. p for:

[0087]

[0088] The roll and pitch samples form the first sample point cloud, and the candidate poses are transformed by left multiplication T. c Transform the point cloud set in the robot coordinate system to obtain the point cloud Tc on the first sample point cloud: t = (0, 0, 0) T Where t is the point cloud translation amount;

[0089] The point cloud set S on the first sample point cloud c For: S c ={p c |p c =T c ×p r p r ∈S r}, where p c It is S c Inner point cloud, p r It is the sensor coordinate system, S r It is a collection of point clouds of the original robot.

[0090] In step S5-2, the largest plane detected by the region growing method is taken as the ground, and the normal vector V of the ground is obtained. cp Then by calculating V cp and V g The angle difference θ between them c To score the pose candidates, θ c The smaller the value, the closer it is to the ground normal vector V.g This yields the roll and pitch angles adjusted by the downward-facing camera, where...

[0091] In step S4-3, after obtaining the first standard sample point cloud, the point cloud set S of the first standard sample point cloud is... rp Adjust to be perpendicular to the ground; for point cloud set S rp For each point in the graph, remove the z-component to create a 2D point cloud; adjust the yaw and position of the 2D point cloud to match the high-resolution mesh graph.

[0092] A second sample point cloud T y From yaw samples and location sample t p =(x p y p ) T Composition, where y is the yaw angle, x p It is the x-axis offset, y p It is the y-axis offset;

[0093] The yaw sample set is: S yaw ={y|y∈[-y max y max ]}, res y =2y max / n y ;y max It is the maximum yaw angle, n y It is the number of yaw samples, res y This is the yaw angle sample resolution;

[0094] The second sample point cloud T y Transform into a new set S yp :

[0095] S yp ={p yp |p yp =T yp ×p rp p rp ∈S rp}, Where, p ，p It is the point for adjusting the heading and translation, T yp It is a homogeneous transformation of the heading angle and translation, p rp It is s rp The point s in rp It is the point cloud set of the first standard sample point cloud, R y It is the heading angle rotation matrix, t p It is the translation amount;

[0096] Second sample point cloud Ty It contains one rotated sample and one translated sample; the set of rotated samples is S. rot ;

[0097] p rot =[x rot y rot ] T , where p rot It is a point that is rotated only, x rot It is p rot The X-axis component, y rot It is p ro The Y-axis component in t;

[0098] S rot The point cloud in the image is projected onto a high-resolution mesh map, corresponding to the index. The set is: in, It is the index value within the map, and res is the resolution;

[0099] A set of point cloud indices is displayed in the high-resolution mesh map;

[0100] Translate samples by moving to Adding an integer to the index to translate the point cloud of the sample is computationally much more efficient than translating floating-point numbers; the translation set for the rotation index is:

[0101] Among them, S trans It is a translation set, p trans It is the translation point, x offset It is p trans X-axis component, y offset It is p trans Y-axis component, Z is a set of integers, x max It is the maximum offset on the X-axis, and the y-axis is the maximum offset on the X-axis. max It is the maximum offset on the Y-axis;

[0102] The set of sample indices translated by a rotation is:

[0103]

[0104] It is the second sample point cloud T y The corresponding index set in the high-resolution grid map;

[0105] Second sample point cloud T y The score is the gray value of the corresponding index i in the projection of the point cloud in the high-resolution mesh map. The sum of; therefore, the score s of the second sample point cloud. cfor:

[0106]

[0107] Select the highest score s c The second sample point cloud is used as the second standard sample point cloud.

[0108] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multi-sensor calibration method for a mobile robot, the mobile robot including a 2D LiDAR, an upward-looking depth camera, and a downward-looking depth camera, characterized in that, It includes the following steps: S1. Set up the calibration environment: including the ground, a middle calibration plate perpendicular to the ground, and two side calibration plates perpendicular to the ground; both side calibration plates are perpendicular to the middle calibration plate; the middle calibration plate and the two side calibration plates together form a U-shaped structure; S2. Setting up the mobile robot: Place the mobile robot on the ground, between the intermediate calibration plate and the two side calibration plates, ensuring that the 2D LiDAR can see the intermediate calibration plate and the two side calibration plates, and that the upper-view depth camera and the lower-view depth camera can both see the intermediate calibration plate and at least one of the side calibration plates; create a robot coordinate system with the mobile robot as the origin, and camera coordinate systems with the upper-view depth camera and the lower-view depth camera as the origins respectively; S3. Obtain raw data: The upper-view depth camera and the lower-view depth camera take pictures to obtain raw point clouds, and the 2D lidar scans the middle calibration plate and the two side calibration plates and extracts the contours therein; S4. Calibration of the upward-looking depth camera: The upward-looking depth camera is calibrated according to the following steps: S4-1. Obtain calibration board point cloud: For the original point cloud obtained by the upward-looking depth camera, first identify and remove the ground point cloud to obtain the calibration board point cloud about the middle calibration board and the side calibration board. S4-2, Roll and Pitch Angle Calibration: The roll and pitch angles of the calibration plate point cloud are successively fine-tuned to generate multiple first sample point clouds; all first sample point clouds are compared to obtain the first sample point cloud that is most perpendicular to the ground, which is used as the first standard sample point cloud. S4-3, Contour Alignment: The yaw angle and the offset of the X-axis and Y-axis of the first standard sample point cloud are successively fine-tuned to obtain multiple second sample point clouds. The contour of the second sample point cloud is aligned with the contour obtained by the 2D LiDAR. The second sample point cloud with the highest contour matching degree is selected as the second standard sample point cloud, and its roll angle, pitch angle, yaw angle, and the offset of the X-axis and Y-axis are used as the calibrated roll angle, pitch angle, yaw angle, and the offset of the X-axis and Y-axis. S5. Calibration of the downward-looking depth camera: The upward-looking depth camera is calibrated according to the following steps: S5-1. Obtaining ground point cloud: For the original point cloud obtained by the downward-looking depth camera, first identify the ground point cloud in it to obtain the ground point cloud. S5-2, Roll and Pitch Angle Calibration: The roll and pitch angles of the ground point cloud are successively fine-tuned to generate multiple third-sample point clouds; the normal vector of each third-sample point cloud is calculated. Select the one that is closest to the ground normal vector. The third sample point cloud is used as the third standard sample point cloud, wherein T denotes matrix transpose; S5-3, Contour Alignment: The yaw angle and the offsets of the X and Y axes of the third standard sample point cloud are successively fine-tuned to obtain multiple fourth sample point clouds. The contours of the fourth sample point clouds are aligned with the contours obtained by the 2D LiDAR. The fourth sample point cloud with the highest contour matching degree is selected as the fourth standard sample point cloud, and its roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes are used as the calibrated roll angle, pitch angle, yaw angle, and the offsets of the X and Y axes.

2. The multi-sensor calibration method for mobile robots according to claim 1, characterized in that: In step S3, a high-resolution mesh map is first created in the robot coordinate system, and then the outlines of the intermediate calibration plate and the two side calibration plates obtained by the 2D LiDAR scan are depicted in the high-resolution mesh map.

3. The multi-sensor calibration method for mobile robots according to claim 1, characterized in that: In step S4-2, firstly, a planar grid parallel to the ground is created. Then, the first sample point cloud is projected onto the planar grid. Then, the following steps are used to select the first sample point cloud with the fewest grid cells that have the first sample point cloud projection as the first standard sample point cloud: The z-component is extracted from the first sample point cloud, and the first sample point cloud is projected onto the planar grid. Then, the first sample point cloud is scored by calculating the number of grid indices occupied. The grid index occupied for: Where the number of grids in the x-direction is The number of grid cells in the y direction is The origin of the planar grid is... res represents the resolution, and ceil represents rounding up the numerical value.

4. The multi-sensor calibration method for mobile robots according to claim 1, characterized in that: In step S3, the original point cloud is obtained from the depth images captured by the upward-looking depth camera and the downward-looking depth camera, and is transformed through projection transformation: in It is a point transformed into the camera coordinate system of the upward-viewing depth camera and the downward-viewing depth camera. Indicates the depth of that point. It refers to the focal lengths of the upward-viewing depth camera and the downward-viewing depth camera on the x-axis. It refers to the focal lengths of the upward-viewing depth camera and the downward-viewing depth camera on the y-axis. It is the original point in the depth image, x d These are the x-axis coordinates and y-axis coordinates in the camera coordinate system. d It is the y-axis coordinate in the camera coordinate system, z d It is the z-axis coordinate in the camera coordinate system, T is, x S It is the x-axis index value within the depth image, y s It is the y-axis index value within the depth image.

5. The multi-sensor calibration method for mobile robots according to claim 4, characterized in that: In step S3, the original point cloud obtained by the upward-looking depth camera and the downward-looking depth camera is first converted into the point cloud set in the robot coordinate system, which is as follows: ,in This refers to either the upward-viewing depth camera or the downward-viewing depth camera. This is a homogeneous transformation from the upward-viewing depth camera or the downward-viewing depth camera to the robot; To transform the points into the robot's coordinate system.

6. The multi-sensor calibration method for mobile robots according to claim 5, characterized in that: In step S4-2, the first sample point cloud includes a roll sample and a pitch sample, and the point cloud set of the roll sample... for: in, It is the maximum offset of the roll. It is the resolution of the sample. It represents the number of samples, where r is a tumbling sample. The rotation matrix of the tumbling sample for: .

7. The multi-sensor calibration method for mobile robots according to claim 6, characterized in that: The point cloud set of the pitch samples for: in, It is the maximum pitch offset. It is the resolution of the sample. The number of samples is p, where p is a pitch angle sample; the rotation matrix R of the pitch sample is... p for: 。 8. The multi-sensor calibration method for mobile robots according to claim 7, characterized in that: The roll and pitch samples constitute the first sample point cloud, and the candidate poses are transformed by left multiplication. Transform the point cloud set in the robot coordinate system to obtain point cloud T on the first sample point cloud. C : Where t is the point cloud translation amount; The point cloud set S on the first sample point cloud C for: , where p C It is S C Inner point cloud, p r It is the sensor coordinate system, S r It is a collection of point clouds of the original robot.

9. The multi-sensor calibration method for mobile robots according to claim 1, characterized in that: In step S5-2, the largest plane detected by the region growing method is taken as the ground surface, and the normal vector of the ground surface is obtained. Then through calculation and The angle difference between To score the posture candidates. The smaller the value, the closer it is to the ground normal vector. This yields the roll and pitch angles adjusted by the downward-facing camera, where... .

10. The multi-sensor calibration method for mobile robots according to claim 2, characterized in that: In step S4-3, after obtaining the first standard sample point cloud, the point cloud set of the first standard sample point cloud is... Adjust to be perpendicular to the ground; for point cloud ensembles For each point in the graph, remove the z-component to create a two-dimensional point cloud; adjust the yaw and position of the two-dimensional point cloud to match the high-resolution mesh graph. A second sample point cloud From yaw samples and location samples Composition, where y is the yaw angle, x p It is the x-axis offset, y p It is the y-axis offset; The yaw sample set is: ;y max It is the maximum yaw angle, n y It is the number of yaw samples, res y It is the yaw resolution; The second sample point cloud Transform into a new set : , where p yp It is the point for adjusting the heading and translation, T yp It is a homogeneous transformation of the heading angle and translation, p rp It is s rp The point s in rp It is the point cloud set of the first standard sample point cloud, R y It is the heading angle rotation matrix, t p It is the translation amount; The second sample point cloud It contains one rotated sample and one translated sample; the set of rotated samples is... ; , , where p rot It is a point that is rotated only, x rot It is p rot The X-axis component, y rot It is p rot The Y-axis component; Will The point cloud in the image is projected onto the high-resolution mesh map, corresponding to the index. The set is: ;in, It is the index value within the map, and res is the resolution; A set of point cloud indices is displayed in the high-resolution mesh map; Translate samples by moving to Add an integer to the index to translate the point cloud of the sample; the translation set for the rotation index is: ;in, It is a translation set. It is a translation point. yes X-axis component yes Y-axis components, Z is a set of integers. It is the maximum offset on the X-axis. It is the maximum offset on the Y-axis; The set of sample indices translated by a rotation is: ； It is the second sample point cloud The corresponding index set in the high-resolution mesh graph; The second sample point cloud The score is the gray value of the corresponding index i of the point cloud projected in the high-resolution mesh map. The sum of the points; therefore, the score of the second sample point cloud. for: ; The second sample point cloud with the highest score Sc is selected as the second standard sample point cloud.

Images