A method, system, and medium for extrinsic calibration of a 3D lidar and depth camera

Abstract

The application discloses a 3D laser radar and a depth camera external parameter calibration method, a system and a medium, wherein the method comprises the following steps: based on a preset position, a depth camera of a robot acquires first coordinate information of a preset three-dimensional obstacle, which is set as cloud; the robot walks around the preset three-dimensional obstacle, acquires second coordinate information of the preset three-dimensional obstacle through a 3D laser radar, and constructs a point cloud map Map based on a preset first algorithm; the Map is set as a source point cloud, and the cloud is set as a target point cloud; the target point cloud is registered to the source point cloud through a preset second algorithm to obtain a registration value; a minimum value in the registration value is extracted, and (R, T) corresponding to the minimum registration value is set as an external parameter of the 3D laser radar and the depth camera on the mobile robot. The application does not need a calibration board, and the depth point cloud information output by the depth camera and the point cloud map information obtained through a slam algorithm of the 3D laser radar are used for external parameter calibration, so that the speed of external parameter calibration is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot autonomous navigation, and more particularly, to a 3D laser radar and depth camera extrinsic calibration method, system and medium. BACKGROUND

[0002] In robot navigation technology, the outside environment is generally measured by sensors such as laser radars, depth cameras, and RGB cameras to identify obstacles for obstacle avoidance. Considering that the measurement data of each sensor is based on its own sensor coordinate system, only by converting to a common coordinate system can global information be obtained to make global strategies and judgments. Therefore, in order to convert the measurement data of each sensor to a unified robot coordinate system, the relative position relationship of each sensor needs to be calibrated. At present, the main method for extrinsic calibration is the calibration board method. For sensor layouts with small common view area, the traditional calibration board scheme has certain difficulty in implementation.

[0003] Therefore, the prior art has defects and needs to be improved. SUMMARY

[0004] In view of the above problems, the purpose of the present application is to provide a 3D laser radar and depth camera extrinsic calibration method, system and medium, which can more conveniently and quickly realize the extrinsic calibration of 3D laser radar and depth camera on a robot.

[0005] The first aspect of the present application provides a 3D laser radar and depth camera extrinsic calibration method, comprising:

[0006] Based on the preset position, the depth camera of the robot acquires first coordinate information of a preset three-dimensional obstacle, which is set as cloud;

[0007] The robot walks around the preset three-dimensional obstacle, acquires second coordinate information of the preset three-dimensional obstacle through the 3D laser radar, and constructs a point cloud map Map based on a preset first algorithm;

[0008] The Map is set as a source point cloud, and the cloud is set as a target point cloud. The target point cloud is registered to the source point cloud through a preset second algorithm to obtain a registration value;

[0009] The minimum value in the registration value is extracted, and the (R, T) corresponding to the minimum registration value is set as the extrinsic calibration of the 3D laser radar and the depth camera on the mobile robot.

[0010] In the present scheme, the robot walking around the preset three-dimensional obstacle comprises: the speed of the robot walking is less than or equal to a preset speed threshold.

[0011] In the scheme, the robot walks around the preset three-dimensional obstacle, and further comprises that the turning angle of the robot per unit time is less than or equal to a preset angle threshold.

[0012] In the scheme, further comprising:

[0013] Obtain 3D laser radar field of view area information;

[0014] The 3D laser radar field of view area is divided into a plurality of sub-regions from outside to inside, which are a first warning area, a first edge area and a first core area.

[0015] In the scheme, further comprising:

[0016] Obtain the 3D laser radar field of view area information of the preset three-dimensional obstacle;

[0017] Determine whether the preset three-dimensional obstacle is in the first edge area or the first core area, if not, trigger the 3D laser radar photography adjustment information, if yes, display normally.

[0018] In the scheme, further comprising:

[0019] The registration value is set as err(R, T), and the formula is:

[0020]

[0021] Where n is the number of the nearest point pair, Map i is a point in the target point cloud Map, cloud i is the nearest point corresponding to Map i in the source point cloud cloud, R represents a rotation matrix, and T represents a displacement vector.

[0022] The second aspect of the application provides a 3D laser radar and depth camera external parameter calibration system, comprising a memory and a processor, the memory stores a 3D laser radar and depth camera external parameter calibration method program, the 3D laser radar and depth camera external parameter calibration method program is executed by the processor to realize the following steps:

[0023] Based on the preset position, the depth camera of the robot obtains the first coordinate information of the preset three-dimensional obstacle, which is set as cloud;

[0024] The robot walks around the preset three-dimensional obstacle, and obtains the second coordinate information of the preset three-dimensional obstacle through the 3D laser radar, and constructs a point cloud map Map based on a preset first algorithm;

[0025] Set Map as the source point cloud, and set cloud as the target point cloud, and register the target point cloud to the source point cloud through a preset second algorithm to obtain a registration value;

[0026] Extract the minimum value in the registration value, and set the (R, T) corresponding to the minimum registration value as the external parameter of the 3D laser radar and the depth camera on the mobile robot.

[0027] In the scheme, the robot walks around the preset three-dimensional obstacle, and the speed of the robot walking is less than or equal to a preset speed threshold.

[0028] In the scheme, the robot walks around the preset three-dimensional obstacle, and the speed of the robot walking is less than or equal to a preset speed threshold.

[0029] In the scheme, the robot walks around the preset three-dimensional obstacle, and the speed of the robot walking is less than or equal to a preset speed threshold.

[0030] Obtain the 3D laser radar field of view area information.

[0031] The 3D laser radar field of view area is divided into a plurality of sub-areas from outside to inside, which are a first warning area, a first edge area and a first core area.

[0032] In the scheme, the robot walks around the preset three-dimensional obstacle, and the speed of the robot walking is less than or equal to a preset speed threshold.

[0033] Obtain the 3D laser radar field of view area information.

[0034] Determine whether the preset three-dimensional obstacle is in the first edge area or the first core area, if not, trigger the 3D laser radar photography adjustment information, if yes, display normally.

[0035] In the scheme, the robot walks around the preset three-dimensional obstacle, and the speed of the robot walking is less than or equal to a preset speed threshold.

[0036] The registration value is set as err (R, T), and the formula is:

[0037]

[0038] Where n is the number of the nearest point pair, Map i is a point in the target point cloud Map, cloud i is the nearest point corresponding to Map i in the source point cloud cloud, R represents a rotation matrix, and T represents a displacement vector.

[0039] The third aspect of the application provides a computer medium, and the medium stores a 3D laser radar and depth camera external parameter calibration method program.

[0040] The application discloses a 3D laser radar and depth camera external parameter calibration method, system and medium, which does not need a calibration board, utilizes depth point cloud information output by a depth camera and point cloud map information obtained by a 3D laser radar through a slam algorithm to perform external parameter calibration, and the speed of external parameter calibration is improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of a 3D laser radar and depth camera external parameter calibration method is shown.

[0042] Figure 2 A block diagram of a 3D laser radar and depth camera external parameter calibration system is shown. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned purposes, features and advantages of the application, the application will be further described in detail below with reference to the drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other without conflict.

[0044] In the following description, many specific details are set forth in order to provide a thorough understanding of the application, but the application can also be practiced without the other ways different from those described herein, therefore, the scope of protection of the application is not limited by the specific embodiments disclosed below.

[0045] Figure 1 A flowchart of a 3D laser radar and depth camera external parameter calibration method is shown.

[0046] As shown in Figure 1 The application discloses a 3D laser radar and depth camera external parameter calibration method, which comprises the following steps:

[0047] S102, based on a preset position, a depth camera of a robot acquires first coordinate information of a preset three-dimensional obstacle, which is set as cloud;

[0048] S104, the robot walks around the preset three-dimensional obstacle, acquires second coordinate information of the preset three-dimensional obstacle through a 3D laser radar, and constructs a point cloud map Map based on a preset first algorithm;

[0049] S106, the Map is set as a source point cloud, the cloud is set as a target point cloud, the target point cloud is registered to the source point cloud through a preset second algorithm, and a registration value is obtained;

[0050] S108, the minimum value in the registration value is extracted, and (R, T) corresponding to the minimum registration value is set as the external parameter of the 3D laser radar and the depth camera on the mobile robot.

[0051] It should be noted that the coordinate system concept used in the embodiment of the present application is as follows: depth camera coordinate system T camera and 3D laser radar coordinate system T lidar , depth camera coordinate system to 3D laser radar coordinate system conversion That is, the six-degree-of-freedom relative spatial pose relationship between the 3D laser radar coordinate system and the depth sensor data, that is, the parameter required by the calibration method in the present application. The preset position can ensure that the preset three-dimensional obstacle is within the depth camera field of view, and the preset three-dimensional obstacle is a common object in life, such as a paper box. After the depth camera of the robot obtains the first coordinate information of the preset three-dimensional obstacle, all coordinate points form a depth point cloud, which is set as cloud. Then the robot is controlled to walk around the preset three-dimensional obstacle, ensuring that the preset three-dimensional obstacle is within the 3D laser radar field of view, and a preset first algorithm is started to build a point cloud map Map. Set Map as the source point cloud and cloud as the target point cloud, and the initial value is set to 0. The target point cloud is registered to the source point cloud by a preset second algorithm to obtain a registration value, and the (R, T) corresponding to the minimum registration value is set as the extrinsic parameter of the 3D laser radar and the depth camera on the mobile robot. The preset first algorithm is a Simultaneous Localization and Mapping (SLAM) algorithm, and the preset second algorithm is an Iterative Closest Point (ICP) algorithm.

[0052] According to the embodiment of the present application, the robot walking around the preset three-dimensional obstacle comprises: the speed of the robot walking is less than or equal to a preset speed threshold.

[0053] It should be noted that a speed sensor is preset on the robot to measure the speed information of the robot in real time. When the speed of the robot walking is greater than the preset speed threshold, the robot brake device is triggered to reduce the speed of the robot walking. For example, the preset speed threshold is 1 m / s, and the speed of the robot walking is controlled to be less than or equal to 1 m / s to ensure that the robot works normally.

[0054] According to the embodiment of the present application, the robot walking around the preset three-dimensional obstacle further comprises: the turning angle of the robot in a unit time is less than or equal to a preset angle threshold.

[0055] It should be noted that the azimuth angle detection device and the time sensing device are preset on the robot, the front direction information of the robot is monitored in real time through the preset azimuth angle detection device, the time information of the front direction of the robot is monitored in real time through the time sensing device, for example, the unit time is set as t0, the front direction θ1 of the robot at t1 time and the front direction θ2 of the robot at (t1+t0) time are recorded respectively, the turning angle of the robot in unit time is (θ2-θ1) when walking, the preset angle threshold is set as θ0, when (θ2-θ1)≤θ0, it indicates that the turning angle of the robot in unit time is normal when walking, and the robot works normally; if (θ2-θ1)>θ0, it indicates that the turning angle of the robot in unit time is too large when walking, a prompt is triggered and the corresponding turning angle is adjusted. The front direction θ1 of the robot at t1 time is set as a positive direction, for example, the front direction of the robot at t1 time is right to south by 3 degrees, and the direction of the robot when walking at (t1+t0) time is left to south by 5 degrees, which is set as a positive angle; if the direction of the robot when walking at (t1+t0) time is left to south, which is set as a negative angle. For example: the front direction of the robot at t1 time is right to south by 3 degrees, and the direction of the robot when walking at (t1+t0) time is left to south by 5 degrees, so the turning angle of the robot in unit time is (θ2-θ1)=(-5-3)=-8 degrees, and the direction of the robot at t1 time is opposite, that is, the rotation angle of the robot in unit time is 8 degrees in the opposite direction.

[0056] According to the embodiment of the application, the method further comprises:

[0057] Obtaining 3D laser radar field of view region information;

[0058] The 3D laser radar field of view region is divided into a plurality of sub-regions from outside to inside, which are a first warning region, a first edge region and a first core region.

[0059] It should be noted that the 3D laser radar field of view region is divided into three sub-regions, wherein the first warning region is distributed at the outermost end of the 3D laser radar field of view region and is connected with the region outside the 3D laser radar field of view; the first edge region and the first core region are wrapped inside by the first warning region.

[0060] According to the embodiment of the application, the method further comprises:

[0061] Obtaining 3D laser radar field of view region information;

[0062] Determining whether the preset three-dimensional obstacle is in the first edge region or the first core region, if not, triggering 3D laser radar photography adjustment information; if yes, displaying normally.

[0063] It should be noted that when controlling the robot to walk around a preset 3D obstacle, the preset 3D obstacle must be kept within the 3D LiDAR's field of view. When the preset 3D obstacle is in the first warning area of ​​the 3D LiDAR's field of view, a warning message is triggered, indicating that the preset 3D obstacle is at the outermost edge of the 3D LiDAR's field of view. The preset 3D obstacle may move out of the 3D LiDAR's field of view at any time, so the robot's movement needs to be adjusted, such as adjusting the robot's walking angle or walking speed.

[0064] According to an embodiment of the present invention, it further includes:

[0065] Set the registration value to err(R,T), and its formula is:

[0066]

[0067] Where n is the number of nearest neighbor pairs, Map i For a point in the target point cloud Map, cloud i For the source point cloud and Map i For the corresponding nearest point, R represents the rotation matrix and T represents the translation vector.

[0068] It should be noted that Map is used as the source point cloud and cloud as the target point cloud, with the initial value set to 0. The ICP method is used to register the target point cloud to the source point cloud, obtaining the registration value err(R,T). Then, the above formula is iteratively optimized until the registration value err reaches a minimum value. The (R,T) corresponding to the minimum registration value is output. External reference.

[0069] According to an embodiment of the present invention, it further includes:

[0070] Obtain information about the field of view of the depth camera;

[0071] The field of view of the depth camera is divided into multiple sub-regions from the outside to the inside, namely the second warning region, the second edge region, and the second core region.

[0072] It should be noted that the field of view of the depth camera is also divided into three sub-regions. The second warning region is located at the outermost edge of the field of view of the depth camera and is connected to the area outside the field of view of the depth camera. The second edge region and the second core region are wrapped inside the second warning region.

[0073] It should be noted that the 3D LiDAR field of view is divided into 3 sub-regions. The first warning region is located at the outermost edge of the 3D LiDAR field of view and is connected to the area outside the 3D LiDAR field of view. The first edge region and the first core region are located within the first warning region.

[0074] According to the embodiment of the present application, further comprising:

[0075] Obtaining picture information of the first warning area and the second warning area;

[0076] Comparing and analyzing the picture of the first warning area and the picture of the second warning area to obtain overlapping picture information of the first warning area and the second warning area;

[0077] Extracting an overlapping boundary length value in the overlapping picture of the first warning area and the second warning area;

[0078] Judging whether the overlapping boundary length value in the overlapping picture of the first warning area and the second warning area is greater than or equal to a preset overlapping boundary length threshold value, if yes, the 3D laser radar field of view and the depth camera field of view of the robot are normal; otherwise, they are abnormal.

[0079] It should be noted that the 3D laser radar on the robot is horizontally installed and is responsible for long-distance measurement; the depth camera is installed obliquely and is responsible for short-distance measurement, and the two complement each other to eliminate the measurement blind area of the robot. The preset overlapping boundary length threshold value is the short side length value of the intersection boundary of the 3D laser radar field of view and the depth camera field of view, and when the overlapping boundary length value in the overlapping picture of the first warning area and the second warning area is greater than or equal to the preset overlapping boundary length threshold value, it indicates that the robot does not have a blind area.

[0080] According to the embodiment of the present application, further comprising:

[0081] Extracting an overlapping area value in the overlapping picture of the first warning area and the second warning area;

[0082] Judging whether the overlapping area value in the overlapping picture of the first warning area and the second warning area is less than or equal to a preset overlapping area threshold value, if yes, the 3D laser radar field of view and the depth camera field of view of the robot are normal; otherwise, they are abnormal.

[0083] It should be noted that when the robot does not have a blind area, the smaller the overlapping area of the first warning area and the second warning area, the smaller the repeated work of the 3D laser radar and the depth camera on the robot, and the higher the work efficiency of the corresponding 3D laser radar and depth camera. If the preset overlapping area threshold value is S, when the overlapping area value in the overlapping picture of the first warning area and the second warning area is greater than S, it indicates that there is too much redundant work of the corresponding 3D laser radar and depth camera on the robot, and the position installation of the 3D laser radar or the depth camera needs to be adjusted to reduce the overlapping area of the first warning area and the second warning area, so as to improve the work efficiency of the 3D laser radar or the depth camera on the robot.

[0084] Figure 2A block diagram of a 3D laser radar and depth camera extrinsic parameter calibration system is shown.

[0085] As shown in Figure 2 , the second aspect of the present application provides a 3D laser radar and depth camera extrinsic parameter calibration system 2, comprising a memory 21 and a processor 22, the memory stores a 3D laser radar and depth camera extrinsic parameter calibration method program, the 3D laser radar and depth camera extrinsic parameter calibration method program is executed by the processor to realize the following steps:

[0086] Based on the preset position, the depth camera of the robot acquires the first coordinate information of the preset three-dimensional obstacle, which is set as cloud;

[0087] The robot walks around the preset three-dimensional obstacle, acquires the second coordinate information of the preset three-dimensional obstacle through the 3D laser radar, and constructs a point cloud map Map based on a preset first algorithm;

[0088] Set Map as the source point cloud and cloud as the target point cloud, register the target point cloud to the source point cloud through a preset second algorithm to obtain a registration value;

[0089] Extract the minimum value in the registration value, and set (R, T) corresponding to the minimum registration value as the extrinsic parameter of the 3D laser radar and the depth camera on the mobile robot.

[0090] It should be noted that the coordinate system concept used in the embodiments of the present application includes: a depth camera coordinate system T camera and a 3D laser radar coordinate system T lidar , the depth camera coordinate system to the 3D laser radar coordinate system conversion The six-degree-of-freedom relative spatial pose relationship between the 3D laser radar coordinate system and the depth sensor data, i.e., the parameter required by the calibration method in the application. The preset position can ensure that the preset three-dimensional obstacle is within the field of view of the depth camera. The preset three-dimensional obstacle is a common object in life, such as a paper box. After the depth camera of the robot obtains the first coordinate information of the preset three-dimensional obstacle, all coordinate points are combined into a depth point cloud, which is set as cloud. Then the robot is controlled to walk around the preset three-dimensional obstacle, ensuring that the preset three-dimensional obstacle is within the field of view of the 3D laser radar, and at the same time, a preset first algorithm is started to construct a point cloud map Map. Set Map as the source point cloud and cloud as the target point cloud, where the initial value is set as 0. A preset second algorithm is used to register the target point cloud to the source point cloud to obtain a registration value, and the (R, T) corresponding to the minimum registration value is set as the extrinsic parameter of the 3D laser radar and the depth camera on the mobile robot. The preset first algorithm is a Simultaneous Localization and Mapping (SLAM) algorithm, and the preset second algorithm is an Iterative Closest Point (ICP) algorithm.

[0091] According to the embodiment of the application, the robot walking around the preset three-dimensional obstacle comprises: the speed of the robot walking is less than or equal to a preset speed threshold.

[0092] It should be noted that a speed sensor is preset on the robot to measure the speed information of the robot in real time. When the speed of the robot walking is greater than the preset speed threshold, the robot brake device is triggered to reduce the speed of the robot walking. For example, the preset speed threshold is 1 m / s, and the speed of the robot walking is controlled to be less than or equal to 1 m / s to ensure that the robot works normally.

[0093] According to the embodiment of the application, the robot walking around the preset three-dimensional obstacle further comprises: the turning angle of the robot in a unit time is less than or equal to a preset angle threshold.

[0094] It should be noted that the azimuth angle detection device and the time sensing device are preset on the robot, the front direction information of the robot is monitored in real time through the preset azimuth angle detection device, the time information of the front direction of the robot is monitored in real time through the time sensing device, for example, the unit time is set as t0, the front direction of the robot at t1 time is recorded as θ1, and the front direction of the robot at (t1+t0) time is recorded as θ2, then the turning angle of the robot in unit time is (θ2-θ1), and the preset angle threshold is set as θ0, then when (θ2-θ1)≤θ0, it indicates that the turning angle of the robot in unit time is normal, and the robot works normally; if (θ2-θ1)>θ0, it indicates that the turning angle of the robot in unit time is too large, a prompt is triggered and the corresponding turning angle is adjusted. The front direction of the robot at t1 time is set as a positive direction, for example, the front direction of the robot at t1 time is right to south by 3 degrees, and the direction of the robot when walking at (t1+t0) time is left to south by 5 degrees, which is set as a positive angle; if the direction of the robot when walking at (t1+t0) time is left to south by 5 degrees, which is set as a negative angle. For example: the front direction of the robot at t1 time is right to south by 3 degrees, and the direction of the robot when walking at (t1+t0) time is left to south by 5 degrees, then the turning angle of the robot in unit time is (θ2-θ1)=(-5-3)=-8 degrees, and the direction of the robot is opposite to that at t1 time, that is, the rotation angle of the robot in unit time is 8 degrees in the opposite direction.

[0095] According to the embodiment of the application, the method further comprises:

[0096] Obtaining 3D laser radar field of view region information;

[0097] The 3D laser radar field of view region is divided into a plurality of sub-regions from outside to inside, which are a first warning region, a first edge region and a first core region.

[0098] It should be noted that the 3D laser radar field of view region is divided into three sub-regions, wherein the first warning region is distributed at the outermost end of the 3D laser radar field of view region and is connected with the region outside the 3D laser radar field of view, and the first edge region and the first core region are wrapped inside by the first warning region.

[0099] According to the embodiment of the application, the method further comprises:

[0100] Obtaining 3D laser radar field of view region information;

[0101] Determining whether the preset three-dimensional obstacle is in the first edge region or the first core region, if not, triggering 3D laser radar photography adjustment information; if yes, displaying normally.

[0102] It should be noted that when the robot is controlled to walk around the preset three-dimensional obstacle, it is necessary to ensure that the preset three-dimensional obstacle is within the 3D laser radar field of view. When the preset three-dimensional obstacle is in the first warning area in the 3D laser radar field of view area, the warning information is triggered, indicating that the preset three-dimensional obstacle is at the outermost edge of the 3D laser radar field of view area, and the preset three-dimensional obstacle may run out of the 3D laser radar field of view at any time, and therefore the robot walking needs to be adjusted, such as adjusting the angle or speed of the robot walking.

[0103] According to the embodiment of the application, the method further comprises:

[0104] The registration value is set as err(R, T), and the formula is:

[0105]

[0106] Wherein n is the number of the nearest point pairs, Map i is a point in the target point cloud Map i is the corresponding nearest point in the source point cloud cloud i R represents a rotation matrix, and T represents a displacement vector.

[0107] It should be noted that Map is used as the source point cloud, cloud is used as the target point cloud, the initial value is set as 0, the target point cloud is registered to the source point cloud by using the ICP method, the registration value err(R, T) is obtained, and the formula is iteratively optimized until the registration value err reaches a minimum value, and the (R, T) corresponding to the minimum registration value is output as the external parameter.

[0108] According to the embodiment of the application, the method further comprises:

[0109] Obtaining the field of view area information of the depth camera;

[0110] The field of view area of the depth camera is divided into a plurality of sub-areas from outside to inside, which are a second warning area, a second edge area and a second core area.

[0111] It should be noted that the field of view area of the depth camera is also divided into three sub-areas, wherein the second warning area is at the outermost end of the field of view area of the depth camera and is connected with the area outside the field of view of the depth camera, and the second edge area and the second core area are wrapped inside the second warning area.

[0112] It should be noted that the 3D laser radar field of view area is divided into three sub-areas, wherein the first warning area is distributed at the outermost end of the 3D laser radar field of view area and is connected with the area outside the 3D laser radar field of view; the first edge area and the first core area are within the first warning area.

[0113] According to the embodiment of the present application, further comprising:

[0114] Obtaining picture information of the first warning area and the second warning area;

[0115] Comparatively analyzing the picture of the first warning area and the picture of the second warning area to obtain overlapping picture information of the first warning area and the second warning area;

[0116] Extracting an overlapping boundary length value in the overlapping picture of the first warning area and the second warning area;

[0117] Judging whether the overlapping boundary length value in the overlapping picture of the first warning area and the second warning area is greater than or equal to a preset overlapping boundary length threshold value, if yes, the 3D laser radar field of view and the depth camera field of view of the robot are normal; otherwise, they are abnormal.

[0118] It should be noted that the 3D laser radar on the robot is horizontally installed and is responsible for long-distance measurement; the depth camera is installed obliquely and is responsible for short-distance measurement, and the two complement each other to eliminate the measurement blind area of the robot. The preset overlapping boundary length threshold value is the short side length value of the joint boundary of the 3D laser radar field of view and the depth camera field of view, and when the overlapping boundary length value in the overlapping picture of the first warning area and the second warning area is greater than or equal to the preset overlapping boundary length threshold value, it indicates that the robot does not have a blind area.

[0119] According to the embodiment of the present application, further comprising:

[0120] Extracting an overlapping area value in the overlapping picture of the first warning area and the second warning area;

[0121] Judging whether the overlapping area value in the overlapping picture of the first warning area and the second warning area is less than or equal to a preset overlapping area threshold value, if yes, the 3D laser radar field of view and the depth camera field of view of the robot are normal; otherwise, they are abnormal.

[0122] It should be noted that when the robot does not have a blind area, the smaller the overlapping area of the first warning area and the second warning area, the smaller the repeated work of the 3D laser radar and the depth camera on the robot, and the higher the working efficiency of the corresponding 3D laser radar and depth camera. If the preset overlapping area threshold value is S, when the overlapping area value in the overlapping picture of the first warning area and the second warning area is greater than S, it indicates that there is too much redundant work of the corresponding 3D laser radar and depth camera on the robot, and the position installation of the 3D laser radar or the depth camera needs to be adjusted to reduce the overlapping area of the first warning area and the second warning area, so as to improve the working efficiency of the 3D laser radar or the depth camera on the robot.

[0123] The third aspect of the present application provides a computer medium, wherein a 3D laser radar and depth camera extrinsic parameter calibration method program is stored in the medium, and the 3D laser radar and depth camera extrinsic parameter calibration method program is executed by a processor to realize the steps of the 3D laser radar and depth camera extrinsic parameter calibration method according to any one of the above.

[0124] The present application discloses a 3D laser radar and depth camera extrinsic parameter calibration method, system and medium, wherein the method comprises the following steps: based on a preset position, a depth camera of a robot acquires first coordinate information of a preset three-dimensional obstacle, which is set as cloud; the robot walks around the preset three-dimensional obstacle, acquires second coordinate information of the preset three-dimensional obstacle through a 3D laser radar, constructs a point cloud map Map based on a preset first algorithm; the Map is set as a source point cloud, and the cloud is set as a target point cloud; the target point cloud is registered to the source point cloud through a preset second algorithm to obtain a registration value; the minimum value in the registration value is extracted, and (R, T) corresponding to the minimum registration value is set as the extrinsic parameter of the 3D laser radar and the depth camera on the mobile robot. The present application does not need a calibration board, and the depth point cloud information output by the depth camera and the point cloud map information obtained by the 3D laser radar through a slam algorithm are used for extrinsic parameter calibration, so that the speed of extrinsic parameter calibration is improved.

[0125] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The above described device embodiments are only schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, direct coupling or communication connection between the components can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0126] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units; they can be located in one place, or distributed on multiple network units; and part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0127] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0128] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by relevant hardware of program instructions, and the foregoing program can be stored in a computer readable storage medium, and the program performs the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0129] Alternatively, the integrated unit of the present application can also be stored in a computer readable storage medium if it is realized in the form of a software function module and sold or used as an independent product. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes: a mobile storage device, a ROM, a RAM, a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method for calibrating extrinsic parameters of a 3D laser radar and a depth camera, characterized in that, The method comprises the following steps: Based on a preset position, a depth camera of a robot acquires first coordinate information of a preset three-dimensional obstacle, which is set as cloud, and the preset three-dimensional obstacle is a common object in life; The robot walks around the preset three-dimensional obstacle, the walking speed of the robot is less than or equal to a preset speed threshold, and the turning angle per unit time when walking is less than or equal to a preset angle threshold, second coordinate information of the preset three-dimensional obstacle is acquired through a 3D laser radar, and a point cloud map Map containing the preset three-dimensional obstacle is constructed based on a SLAM algorithm. Set the Map as the source point cloud and the cloud as the target point cloud, register the target point cloud to the source point cloud through the ICP algorithm to obtain a registration value, and set the registration value as The formula is: ， where n is the number of nearest point pairs, is a point in the target point cloud Map, is a point in the source point cloud cloud, is the corresponding nearest point, R represents a rotation matrix, and T represents a displacement vector. The minimum value in the extraction registration value is registered, and the minimum registration value corresponding to Set as the extrinsic parameter of the 3D laser radar and the depth camera on the mobile robot. 2.The method of calibrating extrinsic parameters of a 3D lidar and a depth camera according to claim 1, wherein, The preset speed threshold is 1 m / s.

3. The method of claim 1, wherein, The preset angle threshold is a turning angle in a unit time wherein is a front direction of the robot at time, is a front direction of the robot at time, is a unit time.

4. The method of claim 1, wherein, The method further comprises the following steps: Acquiring 3D laser radar field of view region information; The 3D laser radar field of view region is divided into a plurality of sub-regions from outside to inside, which are a first warning region, a first edge region and a first core region.

5. The method of claim 4, wherein, The method further comprises the following steps: Acquiring 3D laser radar field of view region information of the preset three-dimensional obstacle; Judging whether the preset three-dimensional obstacle is in the first edge region or the first core region, if not, triggering 3D laser radar photography adjustment information, and if yes, displaying normally.

6. The method of claim 1, wherein, The common object in life includes a carton. 7.A system for calibrating extrinsic parameters of a 3D laser radar and a depth camera, characterized in that, The method comprises a memory and a processor, the memory stores a 3D laser radar and depth camera external parameter calibration method program, and the 3D laser radar and depth camera external parameter calibration method program is executed by the processor to realize the following steps: Based on a preset position, a depth camera of a robot acquires first coordinate information of a preset three-dimensional obstacle, which is set as cloud, and the preset three-dimensional obstacle is a common object in life; The robot walks around the preset three-dimensional obstacle, the walking speed of the robot is less than or equal to a preset speed threshold, and the turning angle per unit time when walking is less than or equal to a preset angle threshold, second coordinate information of the preset three-dimensional obstacle is acquired through a 3D laser radar, and a point cloud map Map containing the preset three-dimensional obstacle is constructed based on a SLAM algorithm. Set the Map as the source point cloud and the cloud as the target point cloud, register the target point cloud to the source point cloud through the ICP algorithm to obtain a registration value, and set the registration value as The formula is: ， where n is the number of nearest point pairs, is a point in the target point cloud Map, is the corresponding nearest point in the source point cloud cloud R represents a rotation matrix, and T represents a displacement vector. The minimum value in the extraction registration value is registered, and the minimum registration value corresponding to Set as the extrinsic parameter of the 3D laser radar and the depth camera on the mobile robot.

8. A computer medium, characterized by The medium stores a 3D laser radar and depth camera external parameter calibration method program, and the 3D laser radar and depth camera external parameter calibration method program is executed by the processor to realize the steps of the 3D laser radar and depth camera external parameter calibration method in any one of claims 1 to 6.

Images