A point cloud calibration method, device and medium for a robot depth camera

Abstract

The embodiments of this specification disclose a point cloud calibration method, device and medium for a robot depth camera, the method comprising: placing the depth camera in a ground observation environment, controlling the robot coordinate system and the depth camera coordinate system to perform step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibration rotation increment and a second calibration rotation increment; performing step iterations of a third rotation increment on the depth camera under the first calibration rotation increment and the second calibration rotation increment, to obtain a third calibration rotation increment of the depth camera; obtaining the median value of the vertical coordinates in each ground point cloud data collected by the depth camera, to obtain a first calibration displacement increment of the depth camera; placing the depth camera in a preset target observation environment, to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy, to achieve calibration of the depth camera according to a relative angular coordinate relationship and a relative coordinate relationship.

Description

Technical Field

[0001] This specification relates to the field of intelligent robot technology, and in particular to a point cloud calibration method, device, and medium for a robot depth camera. Background Art

[0002] With rising labor costs and the continuous development of industrial robot control, planning, and recognition methods, the use of industrial robots to replace manual labor in automated production is becoming increasingly common. A crucial aspect of robotic cameras is camera calibration. Only with calibrated camera parameters can the image sensor accurately calculate the image of the external scene projected onto it, ensuring correct robot movement.

[0003] Traditional depth camera point cloud calibration mostly relies on depth images or even color images to perform coordinate relationship calculations and conversions based on visual feature points combined with the camera internal parameters provided by the manufacturer. The calculations are complex, consume more computing resources, and are time-consuming, increasing the computing pressure on the robot embedded system. It relies on the internal parameters provided by the manufacturer. When the manufacturer's technical support cannot be obtained, calibration cannot be performed. It relies on color images and depth images. There is a possibility that the depth camera may not have a color camera. When there is no color image and depth image output, calibration cannot be performed. At the same time, the depth image is not intuitive, the image interference details are more, and there are observation errors. When there are errors in the internal parameters, there are errors in the calibration results. Summary of the Invention

[0004] To solve the above technical problems, the embodiments of this specification provide a point cloud calibration method, device and medium for a robot depth camera.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] One or more embodiments of this specification provide a point cloud calibration method for a robot depth camera, the method comprising:

[0007] Placing a depth camera in the robot in a preset ground observation environment, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and performing step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibrated rotation increment and a second calibrated rotation increment; wherein the first rotation increment and the second rotation increment are relative rotation increments about a vertical coordinate axis;

[0008] Performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera;

[0009] Obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determining a first calibration displacement increment of the depth camera based on the median value;

[0010] Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy;

[0011] The relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system are determined based on the first calibrated rotation increment, the second calibrated rotation increment and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment and the third calibrated displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship.

[0012] Optionally, in one or more embodiments of this specification,

[0013] The controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera to perform step iteration of the first rotation increment and the second rotation increment respectively to determine the first calibration rotation increment and the second calibration rotation increment specifically includes:

[0014] Increasing the first rotation increment based on a preset increment value; wherein when the first rotation increment is stepped, the step span is 0.001;

[0015] Acquire a preset number of ground point cloud data collected in real time by the depth camera as first sampling point cloud data; wherein the horizontal coordinate values ​​of the first sampling point cloud data are the same;

[0016] Obtaining a current vertical coordinate value in each of the first sampling point cloud data to determine whether the current vertical coordinate values ​​in the preset number of ground point cloud data are the same;

[0017] If they are the same, the first rotation increment corresponding to the current vertical coordinate value is used as the first rotation calibration increment;

[0018] Performing a stepwise increase of the second rotation increment of the depth camera based on a preset increment value under the first calibrated rotation increment;

[0019] Acquire a preset number of ground point cloud data collected in real time by the depth camera as second sampling point cloud data, determine current horizontal coordinate values ​​of the second sampling point cloud data, and determine whether the current horizontal coordinate values ​​of each second sampling point cloud data are the same; wherein the vertical coordinate values ​​of the second sampling point cloud data are the same;

[0020] If they are the same, the second rotation increment corresponding to the current horizontal coordinate value is used as the second rotation calibration increment.

[0021] Optionally, in one or more embodiments of the present specification, performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera specifically includes:

[0022] The depth camera is controlled to increase the third rotation increment by a step value based on a preset increment value under the first calibrated rotation increment and the second calibrated rotation increment;

[0023] Obtain a preset number of ground point cloud data collected by the depth camera in real time as third sampling point cloud data, determine the vertical coordinate values ​​corresponding to the third sampling point cloud data, and determine whether the current vertical coordinate values ​​of each of the third sampling point cloud data are the same. If they are the same, use the current third rotation increment as the third calibration rotation increment of the depth camera; wherein the vertical coordinate values ​​of the third sampling point cloud data are the same.

[0024] Optionally, in one or more embodiments of the present specification, obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera to determine a first calibration displacement increment of the depth camera based on the median value specifically includes:

[0025] Sorting the vertical coordinate values ​​corresponding to each of the ground point cloud data to generate a vertical coordinate value sequence, so as to obtain the median value of the vertical coordinate in each of the ground point cloud data collected by the depth camera based on the vertical coordinate value sequence;

[0026] The relative coordinates between the current robot coordinate system and the depth camera coordinate system are determined based on the median value of the vertical coordinates, so as to use the relative coordinates as the first calibration displacement increment of the depth camera.

[0027] Optionally, in one or more embodiments of the present specification, the depth camera is placed in a preset target observation environment to determine the second calibration displacement increment and the third calibration displacement increment of the depth camera based on a preset adjustment strategy, specifically including:

[0028] The depth camera is placed in a preset target observation environment, and the two-dimensional coordinate position of the depth camera and the target point cloud data collected by the depth camera are obtained; wherein the preset target observation environment is preset with a vertical rectangular object with a height higher than the uppermost field of view of the depth camera in the center of the field of view of the depth camera as an observation target, and one side of the observation target is facing the depth camera;

[0029] Detecting and obtaining an average value of the distance between the depth camera and the observation target by an ultrasonic sensor set at the two-dimensional coordinate position;

[0030] Determining a second calibration displacement increment of the depth camera based on the target point cloud data and the distance average;

[0031] An average value of the horizontal coordinates of each target point cloud data is obtained to determine a third calibration displacement increment of the depth camera based on the distance average value, the average value of the horizontal coordinates of the target point cloud data, and the angle data corresponding to the second calibration rotation increment.

[0032] Optionally, in one or more embodiments of the present specification, determining a second calibration displacement increment of the depth camera based on the target point cloud data and the average distance value specifically includes:

[0033] Traversing the ground point cloud data to obtain vertical coordinate values ​​in each ground point cloud data, and dividing the vertical coordinate values ​​into sets based on a magnitude relationship between the vertical coordinate values ​​and a zero value;

[0034] Obtaining an average value of the horizontal coordinates corresponding to each of the ground point cloud data in a set whose vertical coordinate values ​​are greater than zero;

[0035] Adjusting the initial second displacement increment of the relative coordinates of the current robot coordinate system and the depth camera coordinate system to obtain an adjusted average value of the horizontal coordinates corresponding to each of the ground point cloud data;

[0036] If the difference between the adjusted horizontal coordinate average value and the average value of the distance to the observed target is less than a preset horizontal coordinate difference threshold, the adjusted initial second displacement increment is used as the second calibration displacement increment of the depth camera.

[0037] Optionally, in one or more embodiments of the present specification, obtaining an average value of the horizontal coordinates of each target point cloud data to determine a third calibration displacement increment of the depth camera based on the average distance value, the average value of the horizontal coordinates of the target point cloud data, and the angle data corresponding to the second calibration rotation increment specifically includes:

[0038] Performing format conversion on the target point cloud data to obtain two-dimensional data of the target point cloud data, and determining points in the two-dimensional data corresponding to edge inner points at both ends of the observation target; wherein the edge inner points at both ends of the observation target are edge inner points on the left and right sides of the observation target;

[0039] Based on the inner edge point on the left side of the observation target, obtaining the left angle range corresponding to the depth camera;

[0040] Obtaining angle values ​​of inner points on the left and right edges of the observed target relative to the forward direction based on the coordinate transformation relationship corresponding to the second calibrated rotation increment, the left angle range, and the angular resolution of the depth camera;

[0041] Increasing the initial third calibration displacement increment of the depth camera step by step based on the preset incremental value, and acquiring the current edge inner points on the left and right sides of the observed target in real time based on the preset single-line laser radar;

[0042] Obtaining current angle values ​​of inner points of the current edges on the left and right sides of the observed target relative to the forward direction based on the coordinate transformation relationship corresponding to the second calibrated rotation increment, the current left angular range, and the angular resolution of the depth camera;

[0043] An angle difference between the angle value and the current angle value is obtained, and if the angle difference is less than a preset angle difference, a third reference position shift increment corresponding to the current angle value is determined as a third reference position shift increment of the depth camera.

[0044] Optionally, in one or more embodiments of the present specification, determining the relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system based on the first calibrated rotation increment, the second calibrated rotation increment, and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment, and the third calibrated displacement increment, so as to implement calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship, specifically includes:

[0045] Obtaining initial values ​​of relative coordinates and relative angular coordinates between the robot coordinate system and the depth camera coordinate system;

[0046] updating the initial relative coordinate value based on the first calibration displacement increment, the second calibration displacement increment, and the third calibration displacement increment to obtain a relative coordinate relationship between the robot coordinate system and the depth camera coordinate system;

[0047] updating the initial value of the relative angular coordinate based on the first calibrated rotation amount, the second calibrated rotation amount, and the third calibrated rotation amount to obtain a relative angular coordinate relationship between the robot coordinate system and the depth camera coordinate system;

[0048] The depth camera is calibrated according to the relative coordinate relationship and the relative angular coordinate relationship.

[0049] One or more embodiments of this specification provide a point cloud calibration device for a robot depth camera, including:

[0050] at least one processor; and,

[0051] a memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0053] Placing a depth camera in the robot in a preset ground observation environment, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and performing step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibrated rotation increment and a second calibrated rotation increment; wherein the first rotation increment and the second rotation increment are relative rotation increments about a vertical coordinate axis;

[0054] Performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera;

[0055] Obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determining a first calibration displacement increment of the depth camera based on the median value;

[0056] Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy;

[0057] The relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system are determined based on the first calibrated rotation increment, the second calibrated rotation increment and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment and the third calibrated displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship.

[0058] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to:

[0059] Placing a depth camera in the robot in a preset ground observation environment, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and performing step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibrated rotation increment and a second calibrated rotation increment; wherein the first rotation increment and the second rotation increment are relative rotation increments about a vertical coordinate axis;

[0060] Performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera;

[0061] Obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determining a first calibration displacement increment of the depth camera based on the median value;

[0062] Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy;

[0063] The relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system are determined based on the first calibrated rotation increment, the second calibrated rotation increment and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment and the third calibrated displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship.

[0064] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0065] By placing the robot's depth camera in a preset ground observation environment and a preset target observation environment, the relevant rotation increments and displacement increments are iteratively increased, and the first, second, and third calibration rotation increments, as well as the first, second, and third calibration displacement increments corresponding to the depth camera are determined, thereby determining the relative angular coordinate relationship and relative coordinate relationship between the robot coordinate system and the depth camera coordinate system. By directly calibrating the point cloud, no internal parameter calculation is required, which reduces the errors caused by the internal parameter conversion calculation and improves the calibration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some of the embodiments described in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without inventive work. In the drawings:

[0067] Figure 1 A schematic diagram of a flow chart of a point cloud calibration method for a robot depth camera provided in an embodiment of this specification;

[0068] Figure 2A schematic diagram of the internal structure of a point cloud calibration device for a robot depth camera provided in an embodiment of this specification;

[0069] Figure 3 A schematic diagram of the internal structure of a non-volatile storage medium provided in an embodiment of this specification. DETAILED DESCRIPTION

[0070] The embodiments of this specification provide a method, device, and medium for point cloud calibration of a robot depth camera.

[0071] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative work should fall within the scope of protection of this specification.

[0072] like Figure 1 As shown, one or more embodiments of this specification provide a flow chart of a point cloud calibration method for a robot depth camera. Figure 1 It can be seen that a point cloud calibration method for a robot depth camera includes the following steps:

[0073] S101: Place the depth camera in the robot in a preset ground observation environment, control the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and perform step iterations of the first rotation increment and the second rotation increment respectively to determine the first calibrated rotation increment and the second calibrated rotation increment; wherein, the first rotation increment and the second rotation increment are relative rotation increments of the vertical coordinate axis.

[0074] In order to calibrate the robot's depth camera without any intrinsic coordinate conversion and without the need for manufacturer technical support, the depth camera must first be placed in an observation environment with only the ground. After obtaining the point cloud data collected by the depth camera, the point cloud data is randomly sampled to obtain ground point cloud data. The robot coordinate system and the depth camera coordinate system corresponding to the depth camera are then controlled to perform step iterations of the first rotation increment and the second rotation increment, respectively, to determine the first calibration rotation increment and the second calibration rotation increment; it should be noted that the first rotation increment and the second rotation increment are relative rotation increments of the vertical coordinate axis. In other words, the relative rotation coordinate increment rx0 of the robot coordinate system and the depth camera coordinate system relative to the z-axis is controlled to be the first rotation increment, and rz0 is the second rotation increment.

[0075] Specifically, in one or more embodiments of the present specification, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera to perform step iterations of the first rotation increment and the second rotation increment, respectively, to determine the first calibration rotation increment and the second calibration rotation increment, specifically includes the following process:

[0076] The first rotation increment is incremented in steps according to a preset increment value. It should be noted that the stepping span of the first rotation increment is 0.001. A preset number of ground point cloud data with the same horizontal coordinate value, collected in real time by the depth camera, are then used as first sampling point cloud data. The current vertical coordinate value of each of the first sampling point cloud data is then obtained to determine whether the current vertical coordinate values ​​of each of the first sampling point cloud data are the same. If they are the same, the first rotation increment corresponding to the current vertical coordinate value is used as the first rotation calibration increment. After obtaining the first rotation calibration increment, the depth camera increments a second rotation increment according to the preset increment value under the first calibration rotation increment. A preset number of ground point cloud data with the same vertical coordinate value, collected in real time by the depth camera, are then used as second sampling point cloud data. The current horizontal coordinate value of each of the first sampling point cloud data is then obtained to determine whether the current horizontal coordinate values ​​of each of the second sampling point cloud data are the same. If the two values ​​are determined to be the same, the second rotation increment corresponding to the current horizontal coordinate value is used as the second rotation calibration increment. In addition, it should be noted that each ground point cloud data is obtained by randomly sampling the initial ground point cloud data collected by the depth camera.

[0077] Specifically, in a certain application scenario of the present specification, the depth camera is first placed in a preset ground observation environment with only the ground, and the initial relative coordinate values ​​(xb, yb, zb) of the current robot coordinate system and the depth camera coordinate system, and the initial relative angular coordinate values ​​(rxb, ryb, rzb) are recorded. (x0, y0, z0) (rx0, ry0, rz0) are increments of the calibration process. The relative rotation coordinate increments rx0 and rz0 of the robot coordinate system and the depth camera coordinate system relative to the z-axis, that is, the first calibration rotation increment and the second calibration rotation increment, are controlled to slowly increase rx0 from 0. First, the relative angular coordinate increment rx0 of the robot coordinate system and the depth camera coordinate system relative to the x-axis is controlled to slowly increase from 0 with a step span of 0.001. At each moment when the rx0 value changes, 5 point clouds with equal x coordinates in the ground point cloud are randomly sampled as the first sampled point cloud data. The coordinates of each first sampled point cloud data are 0.05m apart, and it is determined whether the z coordinates of the first sampled point cloud data are equal. Repeat this first rotation increment process until the vertical coordinates of all first sampled point clouds are the same when the first rotation increment value is changed. Record the first rotation increment value at this time as the first rotation calibration increment. Then, keep the first rotation calibration increment unchanged and slowly increase it from the current second rotation increment value. The preset increment value for each increase is rz0. At each rz0 value change, randomly sample 5 point clouds with equal z coordinates in the ground point cloud as the second sampled point cloud data. The coordinates of each first sampled point cloud data are separated by 0.05m. Determine whether the x coordinates of the second sampled point cloud data are equal. Repeat this second rotation increment process until the horizontal coordinate values ​​of all second sampled point cloud data are equal. At this time, the second rotation increment is used as the second rotation calibration increment.

[0078] S102: performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera.

[0079] After determining the first rotation calibration increment and the second rotation calibration increment based on the above steps, in order to quickly and easily calibrate the relative angular coordinate system, in the embodiment of this specification, after adjusting the depth camera to the first calibration rotation increment and the second calibration rotation increment, a step iteration of the third rotation increment is performed, thereby obtaining the third calibration rotation increment of the depth camera during the iterative process.

[0080] Specifically, in one or more embodiments of the present specification, performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera specifically includes the following steps:

[0081] First, the depth camera is subjected to a first calibration rotation increment and a second calibration rotation increment, and a third rotation increment is incrementally increased based on a preset increment value. A preset number of ground point cloud data collected in real time by the depth camera is obtained as third sampled point cloud data, and the vertical coordinate values ​​corresponding to the third sampled point cloud data are determined. It is determined whether the current vertical coordinate values ​​of each third sampled point cloud data are the same. If they are the same, the current third rotation increment is used as the third calibration rotation increment of the depth camera; it should be noted that the vertical coordinate values ​​of the third sampled point cloud data are the same. For example, in a certain application scenario of this specification, rx1 and rz1 obtained in the above steps are kept unchanged, so that the third rotation increment is slowly incremented by ry0 from the current value. At each moment when the ry0 value changes, 5 point clouds with equal y coordinates in the ground point cloud are randomly sampled as the third sampled point cloud data. It is determined whether the z coordinates of the third sampled point cloud data are equal. This ry0 increment process is repeated until the z coordinates of each third sampled point cloud data are equal, and this ry0 value is recorded as ry1, i.e., the third calibration rotation increment.

[0082] S103: Obtain a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determine a first calibration displacement increment of the depth camera based on the median value.

[0083] After determining the relative angular coordinate relationship of the point cloud by determining the rotation increment, in order to calibrate the point cloud and reduce the huge computational pressure caused by the intrinsic parameter conversion operation, the relative coordinate relationship needs to be calibrated. Therefore, in the embodiments of this specification, the median value of the vertical coordinate of each ground point cloud data collected by the depth camera is obtained to achieve the process of obtaining the first calibration displacement increment of the depth camera based on the median value.

[0084] Specifically, in one or more embodiments of the present specification, obtaining the median value of the vertical coordinates in each ground point cloud data collected by the depth camera to determine the first calibration displacement increment of the depth camera based on the median value specifically includes the following process:

[0085] First, the vertical coordinate values ​​corresponding to each ground point cloud data set are sorted to generate a sequence of vertical coordinate values. Based on this sequence of vertical coordinate values, the median value of the vertical coordinates in each ground point cloud data set captured by the depth camera is obtained. Based on the median value of the obtained vertical coordinates, the relative coordinates between the current robot coordinate system and the depth camera coordinate system are determined, and then this relative coordinate is used as the first calibration shift increment of the depth camera. Specifically, in a certain application scenario of this specification, 5 ground point cloud data sets are randomly sampled, the z coordinates of the sampled points are sorted and the median value a is taken, and the relative coordinate z0-a between the current robot coordinate system and the depth camera coordinate system is recorded as z1, which is the first calibration shift increment of the depth camera.

[0086] S104: placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy.

[0087] In order to be able to directly perform simple feature recognition and calibration of planar point clouds, edge laser points, etc. through point clouds, observe the changes in the corresponding point cloud coordinates, and select the required point cloud coordinate transformation relationship, in an embodiment of this specification, the depth camera is placed in a pre-set target observation environment, so that the second and third calibration position shift increments of the depth camera are determined based on a pre-set adjustment strategy. Specifically, in one or more embodiments of this specification, the depth camera is placed in a preset target observation environment to determine the second and third calibration position shift increments of the depth camera based on a preset adjustment strategy, specifically including the following steps:

[0088] First, place the depth camera in a pre-set target observation environment, and obtain the two-dimensional coordinate position of the depth camera and the target point cloud data collected by the depth camera; it should be noted that the pre-set target observation environment has a vertical rectangular object with a height higher than the uppermost field of view of the depth camera pre-set in the middle of the depth camera's field of view as an observation target, and one side of the observation target is facing the depth camera. Then, an ultrasonic sensor set at the two-dimensional coordinate position is used to detect and obtain the average distance between the depth camera and the observation target. It can be understood that the average distance value is the average value of the distances obtained by the ultrasonic sensor through multiple detections. Based on the target point cloud data and the distance average value, the second calibration displacement increment of the depth camera is determined, and the average value of the horizontal coordinates of each target point cloud data is obtained, so that the third calibration displacement increment of the depth camera is determined based on the distance average value, the average value of the horizontal coordinates of the target point cloud data and the angle data corresponding to the second calibration rotation increment.

[0089] Specifically, in a certain application scenario, a vertical rectangular object with a height higher than the depth camera's top field of view is placed directly in the center of the camera's field of view as the observation target, with one side of the rectangular object facing the depth camera and preventing the depth camera from seeing the other sides. An ultrasonic sensor, fixed to the robot and at the same (x, y) position as the depth camera, is aimed at the target object and measures the distance 10 times, taking the average value d as the average distance between the depth camera and the observation target. The second calibration displacement increment of the depth camera is then determined based on the target point cloud data and the average distance value, and the average value of the horizontal coordinates of each target point cloud data is obtained. The third calibration displacement increment of the depth camera is then determined based on the angle data corresponding to the average distance value, the average value of the horizontal coordinates of the target point cloud data, and the second calibration rotation increment.

[0090] Furthermore, in one or more embodiments of the present specification, the second calibration displacement increment of the depth camera is determined based on the target point cloud data and the average value of the distance, which specifically includes the following steps: first, traverse the ground point cloud data to obtain the vertical coordinate values ​​in each ground point cloud data, and then divide the vertical coordinate values ​​into sets according to the size relationship between the vertical coordinate values ​​and the zero value. Obtain the horizontal coordinate average value corresponding to each ground point cloud data in the set whose vertical coordinate value is greater than the zero value. Adjust the initial second displacement increment of the relative coordinates of the current robot coordinate system and the depth camera coordinate system to obtain the adjusted horizontal coordinate adjusted average value corresponding to each ground point cloud data. If the difference between the horizontal coordinate adjusted average value and the distance average value of the observed target is less than the preset horizontal coordinate difference threshold, then the adjusted initial second displacement increment is used as the second calibration displacement increment of the depth camera. Specifically, in a certain application scenario of this specification, the point cloud is traversed and divided into two categories: points with vertical coordinate values ​​greater than 0 and points approximately equal to 0. Then, 10 points are sampled in the point cloud with vertical coordinate values ​​greater than 0, and the average value e of their x-coordinates is calculated. The relative coordinate x0 of the current robot coordinate system and the depth camera coordinate system is increased or decreased until the difference between e and d is less than the preset horizontal coordinate difference threshold f=0.1. The adjusted initial second displacement increment is recorded at this time as the second calibration displacement increment x1 of the depth camera.

[0091] Furthermore, in one or more embodiments of the present specification, obtaining the average value of the horizontal coordinates of each target point cloud data to determine the third calibration displacement increment of the depth camera based on the average distance value, the average value of the horizontal coordinates of the target point cloud data, and the angle data corresponding to the second calibration rotation increment specifically includes the following process:

[0092] First, the target point cloud data is formatted to obtain two-dimensional data. Points in the two-dimensional data corresponding to the inner edge points at either end of the observed target are identified. The inner edge points at either end of the observed target are the inner edge points on the left and right sides of the observed target. Then, based on the inner edge points on the left side of the observed target, the left angular range corresponding to the depth camera is obtained. The angular values ​​of the inner edge points on the left and right sides of the observed target relative to the forward direction are obtained based on the coordinate transformation relationship corresponding to the second calibration rotation increment, the left angular range, and the angular resolution of the depth camera. The initial third calibration displacement increment of the depth camera is incremented based on a preset increment value, and the current inner edge points on the left and right sides of the observed target are acquired in real time using a pre-set single-line laser radar. The current angular values ​​of the current inner edge points on the left and right sides of the observed target relative to the forward direction are obtained based on the coordinate transformation relationship corresponding to the second calibration rotation increment, the current left angular range, and the angular resolution of the depth camera. Then, the angle difference between the above angle value and the current angle value is obtained. If it is determined that the angle difference between the two is less than the preset angle difference, the third calibration displacement increment corresponding to the current angle value can be determined as the third calibration displacement increment of the depth camera.

[0093] Specifically, in an embodiment of a certain application scenario of the present specification, the third calibration displacement increment of the depth camera is determined based on the average distance, the average value of the horizontal coordinate of the target point cloud data and the angle data corresponding to the second calibration rotation increment. The specific process is: converting the camera point cloud into two-dimensional lidar format data, referred to as two-dimensional data, and then traversing the two-dimensional data from right to left to calculate the difference between two adjacent points. When the difference is less than the threshold value -m=-0.9 for the first time, the minuend point number is recorded as h1; when the difference is greater than the threshold value m=0.9 for the first time, the minuend point number is recorded as h2, then h1 and h2 correspond to the right and left edge laser points of the target object respectively; according to the minimum angle angle_min, maximum angle angle_max, and angular resolution angle_increment of the two-dimensional data at this time, the angle value g1 of the right edge point is , the angle value g2 of the left edge point is Then, a single-line laser radar fixed to the robot and installed at the same (x, y) position and (rx, ry, rz) angular coordinate orientation as the depth camera is used to observe the target. The angle values ​​g3 and g4 of the right and left edge points are calculated using the same method as the two-dimensional data. Starting from 0, the size of y0 is slowly stepped, and the comparison between g1 and g3, and g2 and g4 is performed. When the difference is less than the threshold k = 0.03, the third-position displacement increment corresponding to the current angle value is recorded as the third-position displacement increment of the depth camera.

[0094] S105: Determine the relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system based on the first calibrated rotation increment, the second calibrated rotation increment and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment and the third calibrated displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship.

[0095] After obtaining the corresponding calibration rotation increment and calibration displacement increment based on the above steps, in order to realize fast calibration of the depth camera without internal parameters. In the embodiment of this specification, the relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system are determined according to the first calibration rotation increment, the second calibration rotation increment and the third calibration rotation increment, and the first calibration displacement increment, the second calibration displacement increment and the third calibration displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship. It should be noted that after obtaining the first calibration rotation increment, the second calibration rotation increment and the third calibration rotation increment, it is necessary to determine whether the first calibration rotation increment, the second calibration rotation increment and the third calibration rotation increment are greater than If it is greater than, subtract the value At the same time, determine whether the first calibration rotation increment, the second calibration rotation increment and the third calibration rotation increment are less than - If it is less than, increase the value .

[0096] The embodiments of this specification do not rely on any internal parameter coordinate conversion and do not require manufacturer technical support. With the cooperation of the robot ultrasonic sensor and single-line lidar data, simple feature recognition and calibration such as plane point cloud and edge laser point are directly performed through the point cloud. The coordinate transformation is performed based on the conversion relationship, the corresponding point cloud coordinate changes are observed, the required point cloud coordinate transformation relationship is selected, and the transformation relationship is fixed to the robot coordinate system to realize the calibration of the depth camera point cloud. By directly calibrating the point cloud, no internal parameter operation is required, which reduces the error in the internal parameter conversion operation and improves the efficiency of calibration.

[0097] Specifically, in one or more embodiments of the present specification, determining the relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system according to the first calibrated rotation increment, the second calibrated rotation increment, and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment, and the third calibrated displacement increment, so as to implement calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship, specifically includes the following steps:

[0098] First, the initial relative coordinate values ​​and the initial relative angular coordinate values ​​(xb, yb, zb) and (rxb, ryb, rzb) of the robot coordinate system and the depth camera coordinate system are obtained. The initial relative coordinate values ​​are updated according to the obtained first calibration displacement increment, second calibration displacement increment, and third calibration displacement increment, thereby obtaining the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system. Then, the initial relative angular coordinate values ​​are updated according to the first calibration rotation, second calibration rotation, and third calibration rotation, thereby obtaining the relative angular coordinate relationship between the robot coordinate system and the depth camera coordinate system. The depth camera is calibrated using the relative coordinate relationship and the relative angular coordinate relationship. Specifically, in an embodiment of a certain application scenario of this specification, it is determined whether the first calibration rotation, the second calibration rotation, and the third calibration rotation, as well as the first calibration displacement increment, the second calibration displacement increment, and the third calibration displacement increment are greater than 3.14. If so, 3.14 is subtracted from the value. Then add the corresponding increments to the initial values ​​of the relative coordinates and the initial values ​​of the relative angular coordinates to obtain (xb+x1, yb+y1, zb+z2), (rxb+rx1, ryb+ry1, rzb+rz1), and use them as the relative coordinate relationship and relative angular coordinate relationship between the current robot coordinate system and the depth camera coordinate system respectively. Fix the conversion relationship to the robot coordinate system to complete the calibration of the depth camera point cloud.

[0099] like Figure 2 As shown, one or more embodiments of this specification provide a schematic diagram of the internal structure of a point cloud calibration device for a robot depth camera, which is composed of Figure 2 It can be seen that the equipment includes:

[0100] at least one processor; and,

[0101] a memory communicatively connected to the at least one processor; wherein,

[0102] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0103] Placing a depth camera in the robot in a preset ground observation environment, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and performing step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibrated rotation increment and a second calibrated rotation increment; wherein the first rotation increment and the second rotation increment are relative rotation increments about a vertical coordinate axis;

[0104] Performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera;

[0105] Obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determining a first calibration displacement increment of the depth camera based on the median value;

[0106] Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy;

[0107] The relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system are determined based on the first calibrated rotation increment, the second calibrated rotation increment and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment and the third calibrated displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship.

[0108] like Figure 3 As shown, one or more embodiments of this specification provide a schematic diagram of the internal structure of a non-volatile storage medium. Figure 3 It is known that a non-volatile storage medium stores computer-executable instructions, wherein the computer-executable instructions can:

[0109] Placing a depth camera in the robot in a preset ground observation environment, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and performing step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibrated rotation increment and a second calibrated rotation increment; wherein the first rotation increment and the second rotation increment are relative rotation increments about a vertical coordinate axis;

[0110] Performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera;

[0111] Obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determining a first calibration displacement increment of the depth camera based on the median value;

[0112] Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy;

[0113] The relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system are determined based on the first calibrated rotation increment, the second calibrated rotation increment and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment and the third calibrated displacement increment, so as to realize the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship.

[0114] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0115] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0116] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A point cloud calibration method for a robot depth camera, characterized in that: The method comprises: Placing a depth camera in the robot in a preset ground observation environment, controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera, and performing step iterations of a first rotation increment and a second rotation increment, respectively, to determine a first calibrated rotation increment and a second calibrated rotation increment; wherein the first rotation increment and the second rotation increment are relative rotation increments about a vertical coordinate axis; Performing a step iteration of a third rotation increment on the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain a third calibrated rotation increment of the depth camera; Obtaining a median value of a vertical coordinate in each ground point cloud data collected by the depth camera, and determining a first calibration displacement increment of the depth camera based on the median value; Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy; Determining a relative angular coordinate relationship and a relative coordinate relationship between the robot coordinate system and the depth camera coordinate system based on the first calibrated rotation increment, the second calibrated rotation increment, and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment, and the third calibrated displacement increment, respectively, so as to calibrate the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship; The controlling the robot coordinate system and the depth camera coordinate system corresponding to the depth camera to perform step iteration of the first rotation increment and the second rotation increment respectively to determine the first calibration rotation increment and the second calibration rotation increment specifically includes: Increasing the first rotation increment based on a preset increment value; wherein when the first rotation increment is stepped, the step span is 0.001; Acquire a preset number of ground point cloud data collected in real time by the depth camera as first sampling point cloud data; wherein the horizontal coordinate values ​​of the first sampling point cloud data are the same; Obtaining a current vertical coordinate value in each of the first sampling point cloud data to determine whether the current vertical coordinate values ​​in the preset number of ground point cloud data are the same; If they are the same, the first rotation increment corresponding to the current vertical coordinate value is used as the first rotation calibration increment; Performing a stepwise increase of the second rotation increment of the depth camera based on a preset increment value under the first calibrated rotation increment; Acquire a preset number of ground point cloud data collected in real time by the depth camera as second sampling point cloud data, determine current horizontal coordinate values ​​of the second sampling point cloud data, and determine whether the current horizontal coordinate values ​​of each second sampling point cloud data are the same; wherein the vertical coordinate values ​​of the second sampling point cloud data are the same; If they are the same, the second rotation increment corresponding to the current horizontal coordinate value is used as the second rotation calibration increment.

2. The point cloud calibration method for a robot depth camera according to claim 1, characterized in that: The stepping and iterating of the third rotation increment of the depth camera under the first calibrated rotation increment and the second calibrated rotation increment to obtain the third calibrated rotation increment of the depth camera specifically includes: The depth camera is controlled to increase the third rotation increment by a step value based on a preset increment value under the first calibrated rotation increment and the second calibrated rotation increment; Obtain a preset number of ground point cloud data collected by the depth camera in real time as third sampling point cloud data, determine the vertical coordinate values ​​corresponding to the third sampling point cloud data, and determine whether the current vertical coordinate values ​​of each of the third sampling point cloud data are the same. If they are the same, use the current third rotation increment as the third calibration rotation increment of the depth camera; wherein the vertical coordinate values ​​of the third sampling point cloud data are the same.

3. The point cloud calibration method for a robot depth camera according to claim 1, characterized in that: The obtaining of a median value of a vertical coordinate in each ground point cloud data collected by the depth camera to determine a first calibration displacement increment of the depth camera based on the median value specifically includes: Sorting the vertical coordinate values ​​corresponding to each of the ground point cloud data to generate a vertical coordinate value sequence, so as to obtain the median value of the vertical coordinate in each of the ground point cloud data collected by the depth camera based on the vertical coordinate value sequence; The relative coordinates between the current robot coordinate system and the depth camera coordinate system are determined based on the median value of the vertical coordinates, so as to use the relative coordinates as the first calibration displacement increment of the depth camera.

4. The point cloud calibration method for a robot depth camera according to claim 1, characterized in that: Placing the depth camera in a preset target observation environment to determine a second calibration displacement increment and a third calibration displacement increment of the depth camera based on a preset adjustment strategy specifically includes: The depth camera is placed in a preset target observation environment, and the two-dimensional coordinate position of the depth camera and the target point cloud data collected by the depth camera are obtained; wherein the preset target observation environment is preset with a vertical rectangular object with a height higher than the uppermost field of view of the depth camera in the center of the field of view of the depth camera as an observation target, and one side of the observation target is facing the depth camera; Detecting and obtaining an average value of the distance between the depth camera and the observation target by an ultrasonic sensor set at the two-dimensional coordinate position; Determining a second calibration displacement increment of the depth camera based on the target point cloud data and the distance average; An average value of the horizontal coordinates of each target point cloud data is obtained to determine a third calibration displacement increment of the depth camera based on the distance average value, the average value of the horizontal coordinates of the target point cloud data, and the angle data corresponding to the second calibration rotation increment.

5. The point cloud calibration method for a robot depth camera according to claim 4, characterized in that: Determining a second calibration displacement increment of the depth camera based on the target point cloud data and the average distance value specifically includes: Traversing the ground point cloud data to obtain vertical coordinate values ​​in each ground point cloud data, and dividing the vertical coordinate values ​​into sets based on a magnitude relationship between the vertical coordinate values ​​and a zero value; Obtaining an average value of the horizontal coordinates corresponding to each of the ground point cloud data in a set whose vertical coordinate values ​​are greater than zero; Adjusting the initial second displacement increment of the relative coordinates of the current robot coordinate system and the depth camera coordinate system to obtain an adjusted average value of the horizontal coordinates corresponding to each of the ground point cloud data; If the difference between the adjusted horizontal coordinate average value and the average value of the distance to the observed target is less than a preset horizontal coordinate difference threshold, the adjusted initial second displacement increment is used as the second calibration displacement increment of the depth camera.

6. The point cloud calibration method for a robot depth camera according to claim 4, characterized in that: Obtaining an average value of the horizontal coordinates of each target point cloud data to determine a third calibration displacement increment of the depth camera based on the average distance value, the average value of the horizontal coordinates of the target point cloud data, and the angle data corresponding to the second calibration rotation increment specifically includes: Performing format conversion on the target point cloud data to obtain two-dimensional data of the target point cloud data, and determining points in the two-dimensional data corresponding to edge inner points at both ends of the observation target; wherein the edge inner points at both ends of the observation target are edge inner points on the left and right sides of the observation target; Based on the inner edge point on the left side of the observation target, obtaining the left angle range corresponding to the depth camera; Obtaining angle values ​​of inner points on the left and right edges of the observed target relative to the forward direction based on the coordinate transformation relationship corresponding to the second calibrated rotation increment, the left angle range, and the angular resolution of the depth camera; Increasing the initial third calibration displacement increment of the depth camera step by step based on the preset incremental value, and acquiring the current edge inner points on the left and right sides of the observed target in real time based on the preset single-line laser radar; Obtaining current angle values ​​of inner points of the current edges on the left and right sides of the observed target relative to the forward direction based on the coordinate transformation relationship corresponding to the second calibrated rotation increment, the current left angular range, and the angular resolution of the depth camera; An angle difference between the angle value and the current angle value is obtained, and if the angle difference is less than a preset angle difference, a third reference position shift increment corresponding to the current angle value is determined as a third reference position shift increment of the depth camera.

7. The point cloud calibration method for a robot depth camera according to claim 1, characterized in that: The determining the relative angular coordinate relationship and the relative coordinate relationship between the robot coordinate system and the depth camera coordinate system based on the first calibrated rotation increment, the second calibrated rotation increment, and the third calibrated rotation increment, and the first calibrated displacement increment, the second calibrated displacement increment, and the third calibrated displacement increment, respectively, so as to implement the calibration of the depth camera according to the relative angular coordinate relationship and the relative coordinate relationship, specifically including: Obtaining initial values ​​of relative coordinates and relative angular coordinates between the robot coordinate system and the depth camera coordinate system; updating the initial relative coordinate value based on the first calibration displacement increment, the second calibration displacement increment, and the third calibration displacement increment to obtain a relative coordinate relationship between the robot coordinate system and the depth camera coordinate system; updating the initial value of the relative angular coordinate based on the first calibrated rotation amount, the second calibrated rotation amount, and the third calibrated rotation amount to obtain a relative angular coordinate relationship between the robot coordinate system and the depth camera coordinate system; The depth camera is calibrated according to the relative coordinate relationship and the relative angular coordinate relationship.

8. A point cloud calibration device for a robot depth camera, characterized in that: The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the methods described in claims 1-7 above.

9. A non-volatile storage medium storing computer-executable instructions, characterized in that: The computer-executable instructions can execute the method according to any one of claims 1 to 7.

Images