A robot upper deep camera mounting pose calibration method and mobile robot

Abstract

The application discloses a kind of robot upper deep camera installation pose calibration method and mobile robot, belong to robot navigation and barrier avoidance technical field, deep camera is installed in robot in oblique downward look-down, obtains the point cloud of the ground in front of robot, and the ground point cloud obtained is processed with downsampling, plane point cloud is segmented and extracted;The plane equation and normal vector of the plane point cloud are obtained, the projection of normal vector in three coordinate planes is obtained in depth camera coordinate system relative to the angle of three coordinate axes of depth camera coordinate system, the installation angle of depth camera is obtained by converting the angle to robot coordinate system;According to the distance of depth camera coordinate system origin to ground, the installation height of depth camera is obtained;The installation angle and installation height are the pose of the required depth camera.The application can detect the deviation of installation after the installation of depth camera is completed, adjust the position of depth camera, and the position of depth camera can be found in time and corrected in time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot navigation and obstacle avoidance, in particular to a robot depth camera installation pose calibration method and mobile robot. BACKGROUND

[0002] The depth camera has dozens of functions such as face recognition, gesture recognition, human skeleton recognition, three-dimensional measurement, environment perception, three-dimensional map reconstruction, and can be widely used in television, mobile phone, robot, unmanned aerial vehicle, logistics, VR / AR, smart home, security, automobile driving assistance and other fields. In the process of mobile robot navigation and obstacle avoidance, a depth camera installed at a top-down angle is often used to supplement the visual angle blind area of the laser radar. Since the depth camera has installation errors, the installation parameters need to be calibrated, but there is currently a lack of a fast and effective depth camera installation pose automatic calibration method. SUMMARY

[0003] The technical task of the present application is to solve the above problems, and provide a robot depth camera installation pose calibration method and mobile robot, which can detect the installation deviation after the installation of the depth camera is completed, adjust the position of the depth camera, and timely discover and correct the incorrect position of the depth camera.

[0004] The technical solution adopted by the present application to solve its technical problems is:

[0005] A robot depth camera installation pose calibration method, which calibrates by ground point cloud segmentation, obtains a translation vector and Euler angles in the installation parameters of the depth camera, and calibrates the installation pose of the depth camera by the translation vector and Euler angles.

[0006] The depth camera is installed on the robot at a downward angle, ground point clouds in front of the robot are obtained, and the obtained ground point clouds are processed by downsampling, segmented and extracted as plane point clouds.

[0007] The plane equation and normal vector of the plane point clouds are obtained, the projection of the normal vector in three coordinate planes relative to the three coordinate axes of the depth camera coordinate system is obtained in the depth camera coordinate system, the installation angle of the depth camera is obtained by converting the included angle to the robot coordinate system, and the installation height of the depth camera is obtained according to the distance from the origin of the depth camera coordinate system to the ground.

[0008] The installation angle and installation height are the required pose of the depth camera.

[0009] The installation parameter of the depth camera is a transformation from the IR camera coordinate system of the depth camera to the base_link coordinate system of the robot, wherein the origin of the base_link coordinate system is located at the intersection of the central axis of the robot and the ground. The installation parameter of the depth camera can be represented by a translation matrix and a rotation matrix, or by three translation amounts x, y and z and three Euler angles r, p and y. The method can obtain three elements z, r and p through ground point cloud segmentation and calibration. In the prior art, the method for calibrating the installation pose of the depth camera is generally limited in space, such as a fixed scene and specific textures. Generally, the depth camera needs to be equipped with an RGB camera to calibrate the installation pose through texture feature information. The method only needs a flat plane and can be calibrated through the plane point cloud without the need of an RGB camera.

[0010] Preferably, the specific process implemented by the method is as follows:

[0011] Step S101: A depth camera is installed on the robot to look downward along the dip direction, and a point cloud containing the ground is obtained through the depth camera;

[0012] Step S102: The ground point cloud obtained is subjected to voxel filtering and downsampling to improve the processing speed;

[0013] Step S103: The ground point cloud is segmented into a plane, and the point cloud of the plane P is obtained by removing the point cloud of the non-ground. The plane equation of the point cloud P in the depth camera coordinate system is represented by a mathematical model Ax+By+Cz+D=0;

[0014] Step S104: The normal vector of the ground point cloud P is V(A, B, C);

[0015] Step S105: The projection of the normal vector V in the depth camera coordinate system on the three coordinate planes is calculated, and the included angle of the projection with respect to the three coordinate axes of the depth camera coordinate system is calculated. The installation angles r and p are calculated by converting the included angle to the robot coordinate system;

[0016] Step S106: The distance from the origin of the depth camera coordinate system to the point cloud P of the plane is the installation height H of the depth camera.

[0017] Further, in step S103, the plane is segmented by a random sample consensus method, and the parameters A, B and C of the plane model are estimated.

[0018] Further, in step S105, the installation angles r and p are calculated as follows: The results are converted to the robot coordinate system.

[0019] Further, in step S106, the installation height H is calculated as follows:

[0020]

[0021] The application also claims a mobile robot, wherein a depth camera is installed on the mobile robot, the depth camera is installed on the mobile robot to look downward obliquely;

[0022] The depth camera acquires a point cloud of the ground in front of the mobile robot, and performs down-sampling processing on the acquired ground point cloud, segments and extracts a planar point cloud; a plane equation and a normal vector of the planar point cloud are obtained according to the extracted planar point cloud, the projection of the normal vector in three coordinate planes is obtained in a depth camera coordinate system, and the included angle of the projection relative to three coordinate axes of the depth camera coordinate system is obtained, the included angle is converted into a mobile robot coordinate system to obtain the installation angle of the depth camera; and the installation height of the depth camera is obtained according to the distance from the origin of the depth camera coordinate system to the ground.

[0023] The installation angle and the installation height are the pose of the depth camera, and the installation error of the depth camera on the mobile robot is calibrated according to the pose of the depth camera.

[0024] Preferably, the depth camera installation pose correction process is as follows:

[0025] The depth camera is installed on the mobile robot to look downward obliquely, and the depth camera acquires a point cloud containing the ground;

[0026] The acquired ground point cloud is subjected to voxel filtering and down-sampling processing to improve processing speed;

[0027] The ground point cloud is segmented into a plane, and the point cloud of the non-ground is removed to obtain a planar point cloud P, and the planar point cloud P is represented by a mathematical model as a plane equation Ax+By+Cz+D=0 in a depth camera coordinate system;

[0028] The normal vector of the ground point cloud P is V(A, B, C);

[0029] The projection of the normal vector V in three coordinate planes is obtained in a depth camera coordinate system, and the included angle of the projection relative to three coordinate axes of the depth camera coordinate system is obtained, the included angle is converted into a mobile robot coordinate system to obtain installation angles r and p;

[0030] The distance from the origin of the depth camera coordinate system to the planar point cloud P is the installation height H of the depth camera;

[0031] The depth camera installation pose is corrected according to the installation angles r and p and the installation height H.

[0032] Further, the plane is segmented by a random sample consensus method, and the parameters A, B, and C of the plane model are estimated.

[0033] Further, the installation angle and the result is converted into

[0034]

[0035] Further, the installation height H is calculated in the following way:

[0036]

[0037] Compared with the prior art, the robot upper depth camera installation pose calibration method and the mobile robot have the following beneficial effects:

[0038] The installation pose of the depth camera is calibrated by segmenting the ground point cloud to obtain a plane normal vector, and the projection of the normal vector in three coordinate planes relative to the three coordinate axes of the depth camera is obtained in the depth camera coordinate system, the Euler angle is converted to the robot coordinate system to obtain the installation angle r, p, and the installation pose of the depth camera can be automatically calibrated by the method, the installation and use efficiency and the data accuracy are greatly improved, and the navigation and obstacle avoidance accuracy of the robot is improved.

[0039] The method can reduce the installation error of the depth camera, effectively calibrate the accurate pose relationship between the mobile robot and the depth camera, and make the subsequent use of the depth camera more accurate.

[0040] The method only needs a flat plane and does not need to carry an RGB camera to realize the installation pose calibration. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is the implementation flowchart of the robot upper depth camera installation pose calibration method provided by the embodiment of the application;

[0042] Figure 2 is a structural diagram of the robot upper depth camera installation pose calibration provided by the embodiment of the application.

[0043] In the figure, 1 is a robot / mobile robot, 2 is a robot coordinate system, 3 is a depth camera, 4 is a ground, and V is a ground normal vector. DETAILED DESCRIPTION

[0044] The application will be further described below in combination with specific embodiments.

[0045] The embodiment of the application provides a robot upper depth camera installation pose calibration method, the method calibrates by ground point cloud segmentation, obtains a translation vector and Euler angle in a depth camera installation parameter, and calibrates the installation pose of the depth camera by the translation vector and the Euler angle.

[0046] REFERENCE Figure 2As shown, the depth camera 3 is installed on the robot 1 to look downward obliquely, acquires the point cloud of the ground in front of the robot 1, and performs down-sampling processing on the acquired ground point cloud, segments and extracts the planar point cloud;

[0047] The plane equation and the normal vector V of the planar point cloud are obtained, the projection of the normal vector V on the XYZ three coordinate planes is obtained in the depth camera coordinate system, the included angle of the projection with respect to the xyz three coordinate axes of the depth camera coordinate system is obtained, the included angle is converted to the robot coordinate system 2 to obtain the installation angle of the depth camera 3, and the installation height of the depth camera 3 is obtained according to the distance from the origin of the depth camera coordinate system to the ground.

[0048] The installation angle and the installation height are the required pose of the depth camera.

[0049] The installation parameters of the depth camera 3 are the transformation from the depth camera IR camera coordinate system to the robot coordinate system base_link, wherein the origin of the base_link is located at the intersection of the central axis of the robot 1 and the ground. The installation parameters of the depth camera 3 can be represented by a translation matrix and a rotation matrix, or by three translation amounts x, y, and z and three Euler angles r, p, and y. The method can obtain three elements z, r, and p through ground point cloud segmentation calibration.

[0050] Reference Figure 1 As shown, the specific process of the method is as follows:

[0051] Step S101: A depth camera looking downward obliquely is installed on a robot, and a point cloud containing the ground is acquired by the depth camera;

[0052] Step S102: The acquired ground point cloud is subjected to voxel filtering and down-sampling processing to improve the processing speed;

[0053] Step S103: The ground point cloud is segmented into a plane, the point cloud of the ground is removed, a planar point cloud P is obtained, and the planar equation of the planar point cloud P in the depth camera coordinate system is represented by a mathematical model Ax+By+Cz+D=0;

[0054] Step S104: The normal vector of the ground point cloud P is V(A, B, C);

[0055] Step S105: The projection of the normal vector V on the XYZ three coordinate planes is obtained in the depth camera coordinate system, the included angle of the projection with respect to the xyz three coordinate axes of the depth camera coordinate system is obtained, the included angle is converted to the robot coordinate system 2 to obtain the installation angles r and p;

[0056] Step S106: The distance from the origin of the depth camera coordinate system to the planar point cloud P is the installation height H of the depth camera 3.

[0057] The step S103 is to segment a plane and estimate parameters A, B and C of a plane model by using a random sample consensus method.

[0058] In the step S105, the installation angle and the result is transformed to

[0059] In the step S106, the installation height H is calculated according to the following formula:

[0060]

[0061] The method for calibrating the installation pose of the depth camera in the prior art is generally limited in space, such as a fixed scene and specific textures, and generally requires that the depth camera be equipped with an RGB camera to calibrate the installation pose by using texture feature information, but the method only needs a flat plane and can be calibrated by using only the plane point cloud without the need of an RGB camera.

[0062] The embodiment of the application further provides a mobile robot, wherein a depth camera 3 is installed on the mobile robot 1, and the depth camera 3 is installed on the mobile robot 1 in a downward oblique view;

[0063] The depth camera 3 acquires the point cloud of the ground in front of the mobile robot 1, and performs down-sampling processing on the acquired ground point cloud, segments and extracts the plane point cloud; the plane equation and the normal vector V of the plane point cloud are obtained according to the extracted plane point cloud, the projection of the normal vector V in the XYZ three coordinate planes is obtained in the depth camera coordinate system, the included angle of the projection relative to the xyz three coordinate axes of the depth camera coordinate system is obtained, the included angle is converted into the mobile robot coordinate system 2 to obtain the installation angle of the depth camera 3, and the installation height of the depth camera 3 is obtained according to the distance from the origin of the depth camera coordinate system to the ground.

[0064] The installation angle and the installation height are the pose of the depth camera, and the installation error of the depth camera 3 on the mobile robot 1 is calibrated according to the pose of the depth camera.

[0065] Reference Figures 1-2 As shown in the figure, the installation pose correction process of the depth camera 3 is as follows:

[0066] The depth camera 3 is installed on the mobile robot 1 in a downward oblique view, and the point cloud containing the ground is acquired by using the depth camera 3;

[0067] The acquired ground point cloud is subjected to voxel filtering and down-sampling processing to improve the processing speed;

[0068] The ground point cloud is segmented to remove the point cloud other than the ground to obtain the plane point cloud P, and the plane equation Ax+By+Cz+D=0 of the plane point cloud P in the depth camera coordinate system is represented by using a mathematical model.

[0069] The normal vector of the ground point cloud P is V(A, B, C);

[0070] The projection of the normal vector V in the XYZ coordinate planes in the depth camera coordinate system is obtained, and the included angle of the projection with respect to the three coordinate axes of the depth camera coordinate system is converted into the mobile robot coordinate system 2 to obtain the installation angles r and p;

[0071] The distance from the origin of the depth camera coordinate system to the plane point cloud P is the installation height H of the depth camera 3;

[0072] According to the installation angles r and p, and the installation height H, the installation pose correction of the depth camera 3 is performed.

[0073] The plane is segmented by the random sample consensus method, and the parameters A, B, and C of the plane model are estimated.

[0074] The installation angles and the results are transformed into

[0075] The installation height H is calculated as follows:

[0076]

[0077] The mobile robot does not need to carry an RGB camera, only needs a flat plane, and can calibrate the installation pose of the depth camera through the plane point cloud only, realizes the automatic calibration of the installation pose of the depth camera, greatly improves the installation and use efficiency and data precision, and is beneficial to improve the navigation and obstacle avoidance precision of the mobile robot.

[0078] Through the above specific embodiments, the skilled in the art can easily realize the present application. However, it should be understood that the present application is not limited to the above specific embodiments. On the basis of the disclosed embodiments, the skilled in the art can arbitrarily combine different technical features to realize different technical solutions.

[0079] In addition to the technical features described in the specification, all are known technologies of the skilled in the art.

Claims

1. A method for calibrating a mounting pose of a depth camera on a robot, characterized in that, The method calibrates by ground point cloud segmentation, obtains a translation vector and Euler angles in installation parameters of the depth camera, and calibrates the installation pose of the depth camera by the translation vector and the Euler angles: The depth camera is installed on the robot to look downward obliquely, obtains the point cloud of the ground in front of the robot, and performs down-sampling processing on the obtained ground point cloud, segments and extracts the planar point cloud; The plane equation and the normal vector of the planar point cloud are obtained, the projection of the normal vector in three coordinate planes is obtained in the depth camera coordinate system, the included angle of the projection relative to three coordinate axes of the depth camera coordinate system is obtained, the installation angle of the depth camera is obtained by converting the included angle to the robot coordinate system, and the installation height of the depth camera is obtained according to the distance from the origin of the depth camera coordinate system to the ground; The installation angle and the installation height are the required pose of the depth camera; The specific process implemented by the method is as follows: Step S101: a depth camera looking downward obliquely is installed on the robot, and the point cloud containing the ground is obtained by the depth camera; Step S102: the obtained ground point cloud is subjected to voxel filtering and down-sampling processing; Step S103: the ground point cloud is segmented into a plane, the point cloud of the non-ground is removed, the planar point cloud P is obtained, and the plane equation of the planar point cloud P in the depth camera coordinate system is represented by a mathematical model, wherein A, B, and C are parameters of the plane model; Step S104: the normal vector of the ground point cloud P is V(A, B, C); Step S105: the projection of the normal vector V in three coordinate planes is obtained in the depth camera coordinate system, the included angle of the projection relative to three coordinate axes of the depth camera coordinate system is obtained, the installation angle r and p are obtained by converting the included angle to the robot coordinate system; Step S106: the distance from the origin of the depth camera coordinate system to the planar point cloud P is the installation height H of the depth camera.

2. The method of claim 1, wherein, In the step S103, the plane is segmented by the random sample consensus method, and the parameters A, B, and C of the plane model are estimated.

3. The method of claim 1 or 2, wherein, In step S105, the installation angle , ; and transforms the result of the solution to .

4. The method of claim 1 or 2, wherein, In the step S106, the installation height H is calculated in the following manner: 。 5. A mobile robot, characterized by A depth camera is installed on the mobile robot, and the depth camera is installed on the mobile robot to look downward obliquely; The depth camera obtains the point cloud of the ground in front of the mobile robot, and performs down-sampling processing on the obtained ground point cloud, segments and extracts the planar point cloud; the plane equation and the normal vector of the planar point cloud are obtained according to the extracted planar point cloud, the projection of the normal vector in three coordinate planes is obtained in the depth camera coordinate system, the installation angle of the depth camera is obtained by converting the included angle to the mobile robot coordinate system, and the installation height of the depth camera is obtained according to the distance from the origin of the depth camera coordinate system to the ground; The installation angle and the installation height are the pose of the depth camera, and the installation error of the depth camera on the mobile robot is calibrated according to the pose of the depth camera; The depth camera installation pose correction process is as follows: The depth camera is installed on the mobile robot to look downward obliquely, and the point cloud containing the ground is obtained by the depth camera; The obtained ground point cloud is subjected to voxel filtering and down-sampling processing; The ground point cloud is segmented, non-ground point clouds are removed, a planar point cloud P is obtained, and a planar equation Ax+By+Cz+D=0 of the planar point cloud P in a depth camera coordinate system is expressed by a mathematical model, wherein A, B, and C are parameters of the planar model; A normal vector of the ground point cloud P is V(A, B, C); An angle between a projection of the normal vector V on three coordinate planes and three coordinate axes of the depth camera coordinate system is obtained, and the angle is converted to a mobile robot coordinate system to obtain installation angles r and p; A distance from an origin of the depth camera coordinate system to the planar point cloud P is an installation height H of the depth camera; A depth camera installation pose is corrected according to the installation angles r and p and the installation height H.

6. The mobile robot of claim 5, wherein, The planar model parameters A, B, and C are estimated by using a random sample consensus method to segment a plane.

7. The mobile robot according to claim 5 or 6, characterized in that The mounting angle , ; and transforming the results of the solution to .

8. The mobile robot of claim 5 or 6, wherein, The installation height H is calculated in the following manner: 。

Images