A laser radar-based multi-box precise detection positioning system and method

Abstract

The present application relates to a kind of based on laser radar multi-box accurate detection positioning system and method, system includes being arranged in the detection area of pre-set just above at least two laser radars, the laser radar is distributed in different azimuth;The area surrounded by the laser radar is greater than the planar area occupied by all boxs.The method includes the following steps: step one: multiple laser radars point cloud data as input quantity;Step two: multiple radar point cloud data is spliced together;Step three: multiple frames point cloud is integrated and superimposed;Step four: point cloud is rasterized on horizontal plane, the variance of point cloud in z axis in each grid is calculated;Step five: the point cloud set in grid is clustered;Step six: calculate bounding box, and the box is positioned by the eight corner points obtained by calculation.This patent is jointly calibrated by multiple laser radars, increases the detection area range, increases point cloud density, to improve the accuracy of positioning, realize the differentiation of multiple targets.

Description

Technical Field

[0001] This invention relates to the field of positioning technology, and more specifically, to a multi-box precision detection and positioning system and method based on lidar. Background Technology

[0002] With the development of logistics technology, the detection and positioning of containers plays a crucial role in the automated sorting and transportation of containers. Currently, methods for container detection and positioning can be categorized based on the sensors used into image-based methods and LiDAR-based methods, and based on the application method into feature-based methods and deep learning-based methods.

[0003] Image-based methods typically use box features or deep learning to detect boxes, then estimate the box's position using bounding boxes. However, because two-dimensional images lack three-dimensional position information, estimating the box's 3D position from the image results in significant errors. Furthermore, images are easily affected by lighting conditions, impacting box detection. LiDAR-based methods provide accurate 3D box position information and are unaffected by lighting. However, existing methods all employ single-radar detection, such as real-time 3D target detection methods based on sparse point cloud data. These methods have small detection areas and insufficient accuracy, especially when multiple targets need to be detected simultaneously.

[0004] Deep learning-based methods achieve good detection accuracy through training on datasets, but to achieve good results, large datasets are required, which are costly to create and difficult to generalize to different scenarios. Furthermore, achieving high detection speeds requires more expensive computing equipment, increasing deployment costs. Feature-based methods, on the other hand, detect using box features, requiring neither datasets nor expensive computing equipment, but parameter selection is more challenging. Summary of the Invention

[0005] To overcome the problems of small detection area and low accuracy of single radar detection in the prior art, the present invention provides a multi-box precision detection and positioning system and method based on lidar, which accurately detects and positions multiple targets by combining multiple radars.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a multi-box precise detection and positioning system based on lidar, including at least two lidars arranged directly above a preset detection area, wherein the lidars are distributed in different directions; the area enclosed by the lidars is larger than the plane area occupied by all the boxes.

[0007] Preferably, the centerline of the detection surface of the lidar forms an acute angle with the vertical direction, and the detection surface of the lidar faces towards the center of the detection area. By changing the angle between the lidar and the vertical direction, the lidar can cover a detection range as large as possible.

[0008] Preferably, the lidar is DJI's Livox Mid-70 lidar. This model of lidar is relatively inexpensive, and multiple combinations can cover most target areas, with a high point cloud density in the target area.

[0009] A method for precise detection and positioning of multiple boxes based on lidar includes the following steps: Step 1: Use point cloud data from multiple LiDAR sensors as input. Step 2: Combine the point cloud data from multiple radars; Step 3: Integrate and superimpose the point clouds from multiple frames; Step 4: Rasterize the point cloud on a horizontal plane, calculate the variance of the point cloud on the z-axis in each grid cell, and obtain the distribution of the point cloud on the z-axis in the grid cell; Step 5: Cluster the point cloud collection within the raster to separate the point clouds of different boxes; Step 6: Use principal component analysis to calculate the eigenvalues ​​and eigenvectors of the covariance matrix for point cloud clustering. Select the three eigenvectors with the largest eigenvalues ​​as the new basis, transform the original clustered point cloud to the coordinate system of the new basis, and then calculate the bounding box. Use the eight corner points obtained from the calculation to locate the box.

[0010] Preferably, in step two, all radars are calibrated, the transformation relationship between radars is calculated, and the coordinates are calibrated to the global coordinate system. The origin is located on the ground at the center point of the two radars, the xy plane is flush with the ground, and the z axis is perpendicular to the ground and upwards, thus obtaining the final calibration matrix. The point cloud data of multiple radars are stitched together based on the calibration matrix between multiple radars.

[0011] Preferably, in step four, the point cloud is rasterized on a horizontal plane to form individual point columns, resulting in a rasterized point cloud set, as shown in the following formula: , Where P represents the original point cloud set, and Let x and y represent the minimum values ​​of the point cloud on the x and y axes, respectively. and This represents the x and y coordinates of point p. and This represents the raster resolution on the x and y axes, and is generally determined by specific requirements. This represents the point information in the i-th row and j-th column of the raster point cluster; i represents the row number of point p after rasterization; j represents the column number of point p after rasterization.

[0012] Preferably, in step five, Euclidean clustering is performed only on the horizontal plane point cloud set.

[0013] Preferably, in step five, by determining whether the variance within each grid cell is greater than a preset variance threshold, the grid cells are divided into horizontal plane grid cells and vertical plane grid cells. The grid cells with variance less than the preset variance threshold are classified as horizontal plane grid cells, and the point cloud set of the horizontal plane grid cells is the horizontal plane point cloud set.

[0014] Preferably, in step six, the specific calculation method for the eight corner points is as follows:

[0015]

[0016]

[0017]

[0018]

[0019]

[0020] Where PCA() means using PCA to calculate the eigenvalues ​​and eigenvectors of the covariance matrix. This indicates the point cloud clustering transformation under the new basis. This represents the center point of the point cloud cluster in the new coordinate system. This indicates the side length of the bounding box. This represents the eight corner points of the bounding box calculated using the center point and side lengths. This represents the eight corner points transformed back to the original coordinate system; This is represented as the feature vector obtained by PCA for the i-th cluster; This represents the feature value of the i-th cluster calculated by PCA; Let x, y, x coordinates be the sets of x, y, x coordinates of the i-th cluster transformed to the new base, respectively. It is represented as the coordinate matrix of the i-th cluster in the original point cloud P; , , Let represent the maximum values ​​of points in the i-th cluster along the x, y, and z coordinate axes, respectively; , Let x, y, and z represent the minimum values ​​of points in the i-th cluster along the x, y, and z coordinate axes, respectively. The term "corner point calculation" refers to the calculation obtained by adding or subtracting half the side length from the center point. It is represented as the first three eigenvectors of the i-th cluster obtained by PCA.

[0021] Compared with existing technologies, the beneficial effects of this invention are: This patent uses multiple lidars for joint calibration, increasing the detection area and point cloud density, thereby improving positioning accuracy and enabling the differentiation of multiple targets. By classifying the box into horizontal and vertical point clouds and performing bounding box calculations after clustering to complete the positioning task, the purpose of separating and positioning the box can be achieved without complex calculations, making it simpler and more efficient. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of a multi-box precision detection and positioning system based on lidar according to the present invention; Figure 2 This is a flowchart of a multi-box precise detection and positioning method based on lidar according to the present invention; Figure 3 This is a comparison diagram of the point cloud integral superposition effect of a multi-box precise detection and positioning method based on lidar according to the present invention. Figure 4 This is a detection effect diagram of a vehicle being located in the center of a multi-box precise detection and positioning method based on lidar according to the present invention. Figure 5 This is a diagram illustrating the detection effect of a vehicle placed at an angle, based on a multi-box precise detection and positioning method using lidar according to the present invention. Figure 6 This is a detection effect diagram of the detection area at a vehicle location according to a multi-box precise detection and positioning method based on lidar according to the present invention. Detailed Implementation

[0023] The accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. To better illustrate this embodiment, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0024] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "long," and "short" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0025] In the accompanying drawings of this embodiment of the invention, for ease of reading and understanding, the front plate, rear plate, and top plate of the casing structure are not shown.

[0026] The technical solution of the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings: Example 1 like Figure 1 The illustration shows an embodiment of a multi-box precision detection and positioning system based on lidar. It includes at least two lidars positioned directly above a preset detection area. The lidars are distributed in different orientations: if there are only two lidars, they are located on opposite sides of the detection area; if there are three lidars, they are equidistantly distributed in a circle around the center of the detection area; if there are four lidars, they are located in the front, back, left, and right directions. The area enclosed by the lidars is larger than the total planar area occupied by all the boxes. The centerline of the lidar's detection surface forms an acute angle with the vertical direction, and the detection surface faces towards the center of the detection area. By changing the angle between the lidar and the vertical direction, the lidar can cover a large detection range. The lidar is a DJI Livox Mid-70. This model is relatively inexpensive, and multiple lidars combined can cover most target areas with high target area point cloud density.

[0027] In this embodiment, two lidar sensors are used to detect targets including a container measuring 3.74m x 1.91m x 1.66m and an AGV vehicle measuring 9.65m x 2.69m x 1.94m. The centerline of the lidar detection surface forms a 30-degree angle with the vertical direction, the distance between the two lidar sensors is 4.9m, and the height above the ground is 7.16m.

[0028] The working principle or workflow of this embodiment is as follows: After the AGV vehicle is loaded with the box, it travels to the detection area. The lidar above the detection area performs detection, and the lidar collects the point cloud data as input to the calculation system. The system stitches together the point cloud data from multiple lidars and integrates and superimposes the point cloud data from multiple frames. Then, the point cloud is rasterized on the horizontal plane, and the variance of the point cloud in each grid on the z-axis is calculated to obtain the distribution of the point cloud in the grid on the z-axis. Then, the point cloud set in the grid is clustered to separate the point clouds of different boxes. Finally, principal component analysis is used to calculate the eigenvalues ​​and eigenvectors of the covariance matrix of the point cloud clusters. The three eigenvectors with the largest eigenvalues ​​are selected as the new basis, and the original clustered point cloud is transformed into the coordinate system of the new basis. Then, the bounding box is calculated, and the box is located by the eight corner points obtained from the calculation.

[0029] The beneficial effects of this embodiment are: by performing joint calibration with multiple lidars, the detection area is increased, the point cloud density is increased, thereby improving the positioning accuracy and enabling the differentiation of targets between containers and between containers and vehicles.

[0030] Example 2 An embodiment of a multi-box precise detection and positioning method based on lidar, based on the system of embodiment 1, includes the following steps: Step 1: Use point cloud data from multiple LiDAR sensors as input. Step 2: Combine the point cloud data from multiple radars; Step 3: Integrate and superimpose the point clouds from multiple frames; Step 4: Rasterize the point cloud on a horizontal plane, calculate the variance of the point cloud on the z-axis in each grid cell, and obtain the distribution of the point cloud on the z-axis in the grid cell; Step 5: Cluster the point cloud collection within the raster to separate the point clouds of different boxes; Step 6: Use principal component analysis to calculate the eigenvalues ​​and eigenvectors of the covariance matrix for point cloud clustering. Select the three eigenvectors with the largest eigenvalues ​​as the new basis, transform the original clustered point cloud to the coordinate system of the new basis, and then calculate the bounding box. Use the eight corner points obtained from the calculation to locate the box.

[0031] The beneficial effects of this embodiment are: by performing joint calibration with multiple lidars, the detection area is increased, the point cloud density is increased, thereby improving the positioning accuracy and enabling the differentiation of multiple targets.

[0032] Example 3 Another embodiment of a multi-box precise detection and positioning method based on lidar, based on the system of embodiment 1, includes the following steps: Step 1: Use point cloud data from multiple LiDAR sensors as input. Step 2: Calibrate all radars, calculate the transformation relationship between radars, and calibrate them to the global coordinate system. The origin is located on the ground at the center point of the two radars, the xy plane is flush with the ground, and the z axis is perpendicular to the ground and upwards. The final calibration matrix is ​​obtained. The point cloud data of multiple radars are stitched together according to the calibration matrix between multiple radars. Step 3: Integrate and superimpose multiple frame point clouds to increase point cloud density and cover more detection areas. In this example, the integration time is 1 second. The results before and after integration are shown below. Figure 3 As shown, Figure 3 The left image shows the effect before integration, and the right image shows the effect after integration.

[0033] Step 4: Rasterize the point cloud on a horizontal plane to form individual point columns, resulting in a rasterized point cloud set, as shown in the following formula: , Where P represents the original point cloud set, and Let x and y represent the minimum values ​​of the point cloud on the x and y axes, respectively. and This represents the x and y coordinates of point p. and This represents the raster resolution on the x and y axes, and is generally determined by specific requirements. This represents the point information in the i-th row and j-th column of the raster point cluster; i represents the row number of point p after rasterization; j represents the column number of point p after rasterization.

[0034] Calculate the variance of the point cloud on the z-axis within each grid cell to obtain the distribution of the point cloud on the z-axis within the grid cell; Step 5: By determining whether the variance within each grid cell is greater than a preset variance threshold, the grid cells are divided into horizontal plane grid cells and vertical plane grid cells. Because the edges of the boxes are relatively clear, the difference between the horizontal plane and the vertical plane is relatively large, and the threshold can be easily selected. Among them, the cells with variance less than the preset variance threshold are classified as horizontal plane grid cells, and the point cloud set of the horizontal plane grid cells is the horizontal plane point cloud set. Euclidean clustering is only performed on the horizontal plane point cloud set to separate the point clouds of different boxes. Step Six: Use principal component analysis to calculate the eigenvalues ​​and eigenvectors of the covariance matrix for point cloud clustering. Select the three eigenvectors with the largest eigenvalues ​​as the new basis, transform the original clustered point cloud to the coordinate system of the new basis, and then calculate the bounding box. Use the eight corner points obtained from the calculation to locate the box. Different poses are shown below. Figures 4-6 As shown. The specific calculation method for the eight corner points is as follows:

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Where PCA() means using PCA to calculate the eigenvalues ​​and eigenvectors of the covariance matrix. This indicates the point cloud clustering transformation under the new basis. This represents the center point of the point cloud cluster in the new coordinate system. This indicates the side length of the bounding box. This represents the eight corner points of the bounding box calculated using the center point and side lengths. This represents the eight corner points transformed back to the original coordinate system; This is represented as the feature vector obtained by PCA for the i-th cluster; This represents the feature value of the i-th cluster calculated by PCA; Let x, y, x coordinates be the sets of x, y, x coordinates of the i-th cluster transformed to the new base, respectively. It is represented as the coordinate matrix of the i-th cluster in the original point cloud P; , , Let represent the maximum values ​​of points in the i-th cluster along the x, y, and z coordinate axes, respectively; , Let x, y, and z represent the minimum values ​​of points in the i-th cluster along the x, y, and z coordinate axes, respectively. The term "corner point calculation" refers to the calculation obtained by adding or subtracting half the side length from the center point. It is represented as the first three eigenvectors of the i-th cluster obtained by PCA.

[0041] The beneficial effects of this embodiment are as follows: By jointly calibrating multiple lidars, the detection area is increased, and the point cloud density is increased, thereby improving the positioning accuracy and enabling the differentiation of multiple targets. By classifying the box into horizontal and vertical point clouds and performing bounding box calculations after clustering to complete the positioning task, the purpose of separating and locating the box can be achieved without complex calculations, making it simpler and more efficient.

[0042] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for precise detection and positioning of multiple boxes based on lidar, characterized in that, Specifically, the steps include the following: Step 1: Use point cloud data from multiple LiDAR sensors as input. Step 2: Combine the point cloud data from multiple radars; Step 3: Integrate and superimpose the point clouds from multiple frames; Step 4: Rasterize the point cloud on a horizontal plane, calculate the variance of the point cloud on the z-axis in each grid cell, and obtain the distribution of the point cloud on the z-axis in the grid cell; Step 5: Cluster the point cloud collection within the raster to separate the point clouds of different boxes; Step 6: Use principal component analysis to cluster the point cloud to calculate the eigenvalues ​​and eigenvectors of the covariance matrix. Select the three eigenvectors with the largest eigenvalues ​​as the new basis, transform the original clustered point cloud into the coordinate system of the new basis, and then calculate the bounding box. Use the eight corner points obtained from the calculation to locate the box. The specific calculation method for the eight corner points is as follows: Where PCA() means using PCA to calculate the eigenvalues ​​and eigenvectors of the covariance matrix. This indicates the point cloud clustering transformation under the new basis. This represents the center point of the point cloud cluster in the new coordinate system. This indicates the side length of the bounding box. This represents the eight corner points of the bounding box calculated using the center point and side lengths. This represents the eight corner points transformed back to the original coordinate system; This is represented as the feature vector obtained by PCA for the i-th cluster; This represents the feature value of the i-th cluster calculated by PCA; Let x, y, x coordinates be the sets of x, y, x coordinates of the i-th cluster transformed to the new base, respectively. It is represented as the coordinate matrix of the i-th cluster in the original point cloud P; , , Let represent the maximum values ​​of points in the i-th cluster along the x, y, and z coordinate axes, respectively; , Let x, y, and z represent the minimum values ​​of points in the i-th cluster along the x, y, and z coordinate axes, respectively. The term "corner point calculation" refers to the calculation obtained by adding or subtracting half the side length from the center point. It is represented as the first three eigenvectors of the i-th cluster obtained by PCA.

2. The method for precise detection and positioning of multiple boxes based on lidar according to claim 1, characterized in that, In step two, all radars are calibrated, the transformation relationship between radars is calculated, and the coordinates are calibrated to the global coordinate system. The origin is located on the ground at the center point of the two radars, the xy plane is flush with the ground, and the z axis is perpendicular to the ground and upwards, thus obtaining the final calibration matrix. Based on the calibration matrix between multiple radars, the point cloud data of multiple radars are stitched together.

3. The method for precise detection and positioning of multiple boxes based on lidar according to claim 1, characterized in that, In step four, the point cloud is rasterized on a horizontal plane to form individual point columns, resulting in a rasterized point cloud set, as shown in the following formula: , Where P represents the original point cloud set, and Let x and y represent the minimum values ​​of the point cloud on the x and y axes, respectively. and This represents the x and y coordinates of point p. and This represents the raster resolution on the x and y axes, and is generally determined by specific requirements. This represents the point information in the i-th row and j-th column of the raster point cluster; i represents the row number of point p after rasterization; j represents the column number of point p after rasterization.

4. The method for precise detection and positioning of multiple boxes based on lidar according to claim 3, characterized in that, In step five, clustering is performed only on the horizontal plane point cloud set.

5. The method for precise detection and positioning of multiple boxes based on lidar according to claim 4, characterized in that, In step five, by determining whether the variance within each grid cell is greater than a preset variance threshold, the grid cells are divided into horizontal plane grid cells and vertical plane grid cells. The grid cells with variance less than the preset variance threshold are classified as horizontal plane grid cells, and the point cloud set of the horizontal plane grid cells is the horizontal plane point cloud set.

6. The method for precise detection and positioning of multiple boxes based on lidar according to claim 1, characterized in that, In step five, Euclidean clustering is used.

7. A multi-box precision detection and positioning system based on lidar, characterized in that, To implement the multi-box precision detection and positioning system based on lidar as described in any one of claims 1-6, at least two lidars are arranged directly above a preset detection area, the lidars being distributed in different orientations; the area enclosed by the lidars is larger than the planar area occupied by all the boxes.

8. A multi-box precision detection and positioning system based on lidar according to claim 7, characterized in that, The centerline of the detection surface of the lidar forms an acute angle with the vertical direction, and the detection surface of the lidar faces the direction close to the center of the detection area.

9. A multi-box precision detection and positioning system based on lidar according to claim 7, characterized in that, The lidar mentioned is DJI's Livox Mid-70 lidar.

Images