Uniform Bird's-Eye View Generation Method for Target Detection in Sparse Point Clouds

Abstract

The present invention relates to a method for generating a uniform bird's-eye view applied to sparse point cloud target detection, and belongs to the field of point cloud processing technology. The method of the present invention includes performing dimensionality reduction sampling on the point cloud in a cylindrical coordinate system, using a two-dimensional image to represent a three-dimensional point cloud, and obtaining a uniform bird's-eye view; mapping the sampling information back to the original point cloud, so that the points in the original point cloud are all recorded with the sampling information of the sampling unit; performing dimensionality reduction sampling on the point cloud in a rectangular coordinate system, using the recorded uniform bird's-eye view information, using a two-dimensional image to represent the three-dimensional point cloud, and obtaining a bird's-eye view suitable for the existing point cloud target detection method based on dimensionality reduction. Compared with the existing method, the present invention can improve the problem of uneven distribution of sparse point cloud sampling.

Description

Technical Field

[0001] The present invention relates to the field of point cloud processing technology, and in particular to a method for generating a uniform bird's-eye view image for sparse point cloud target detection. Background Art

[0002] Point cloud is a data expression form that describes three-dimensional environments and objects. It is easy to obtain and has obvious advantages in stability and expression ability compared with other data forms such as images in some applications. Therefore, it has gained attention in academic research and also gained supporters in engineering applications. Point cloud processing methods can be roughly divided into three categories, namely, point cloud processing methods based on dimensionality reduction, point cloud processing methods based on voxels, and point-based point cloud processing methods.

[0003] The point cloud processing method based on dimensionality reduction removes a certain dimension of the 3D point cloud, and reduces the 3D point cloud into a 2D image, such as a front view, a bird's-eye view, etc., by projecting the 3D data onto a 2D plane and combining statistics, so that it can be further processed using a relatively mature image processing method with extensive research and practical results. This type of method has the fastest processing speed among the three types of methods, and some of the technologies are also the most mature. If the application requires not to be affected by the information loss caused by dimensionality reduction, this type of method is a suitable technical choice.

[0004] The voxel-based point cloud processing method organizes the 3D point cloud into discrete and ordered 3D voxels, and then processes the voxel data using methods such as 3D convolution. Point clouds are continuous in space and are therefore not conducive to representation and calculation in matrix form. This type of method samples and counts the point cloud, and uses discrete and ordered voxels instead of point clouds to express the 3D environment. It retains more 3D information than dimensionality reduction methods, but due to its high computational overhead, it is difficult to apply to tasks that have certain real-time requirements.

[0005] The point-based point cloud processing method retains the point cloud as the basic data representation unit, and uses a multi-layer perceptron to process the point cloud. Its advantage is that it retains the three-dimensional information to the greatest extent. However, due to the large number of points in the point cloud, which can reach the order of 100,000, processing each point will consume a lot of computing resources, so it is common to sample some points in the point cloud. The commonly used sampling method is the farthest point sampling, and the number of samples is manually specified based on experience.

[0006] The above three methods all sample point clouds to varying degrees, but due to the collection method and sparsity of point clouds, the data sampled by these sampling methods have the problem of uneven distribution. LiDAR, as a sensor for collecting point clouds, can be regarded as a device that emits lasers from a single point and collects light information. Similar to a point light source, all lasers emitted by a LiDAR are emitted radially in all directions and reflected at obstacles. The angle between two adjacent lasers is fixed, and the farther the emission distance, the greater the spatial distance between the two reflection points. Such characteristics of LiDAR make the point cloud appear scattered and have distance-related sparsity. If the sampling methods in the above three methods are used, without considering the sparsity of the point cloud, and the same sampling standard is used for points at different distances, the sampled data will be unevenly distributed. Summary of the invention

[0007] In order to improve the problem of uneven distribution of sparse point cloud sampling and apply point cloud processing technology to the field of mobile robots with high real-time requirements, the present invention provides a uniform bird's-eye view generation method for sparse point cloud target detection.

[0008] The present invention adopts the following technical solution: a method for generating a uniform bird's-eye view image for sparse point cloud target detection, comprising:

[0009] The point cloud is sampled in a cylindrical coordinate system with reduced dimension, and a two-dimensional image is used to represent the three-dimensional point cloud to obtain a uniform bird's-eye view.

[0010] Map the sampling information back to the original point cloud so that all points in the original point cloud record the sampling information of the sampling unit;

[0011] The point cloud is sampled in a rectangular coordinate system with reduced dimensionality. The information of the recorded uniform bird's-eye view is used to represent the three-dimensional point cloud with a two-dimensional image, and a bird's-eye view suitable for the existing point cloud target detection method based on dimensionality reduction is obtained.

[0012] Furthermore, the point cloud is sampled in a cylindrical coordinate system with reduced dimensionality, and a two-dimensional image is used to represent the three-dimensional point cloud to obtain a uniform bird's-eye view, including:

[0013] In the cylindrical coordinate system, the point cloud is sampled using a fan-shaped cylinder as a sampling unit, and each sampling unit is statistically obtained to obtain the height statistics, intensity statistics, and density statistics;

[0014] Find the point with the maximum height coordinate value among all sampling points. The height coordinate value of this point is the height statistic of the unit, the reflection intensity of this point is the intensity statistic of the unit, and the related calculation of the number of sampling points in the unit is the density statistic of the unit.

[0015] Furthermore, the point cloud is uniformly sampled and features are extracted in the cylindrical coordinate system as follows:

[0016] The point cloud after clipping and other preprocessing is recorded as N is the point cloud The number of midpoints, 4 is the characteristic number of the point; in the rectangular coordinate system, a point in the point cloud In vector form it is:

[0017]

[0018] Among them, the coordinate origin is the origin of the point cloud, [x i ,y i ,z i ] is a point Three-dimensional coordinates in a rectangular coordinate system with the Z axis perpendicular to the ground, intensity i is the laser reflection intensity at this point; correspondingly, in the cylindrical coordinate system, point In vector form it is:

[0019]

[0020] Among them, [ρ i ,θ i ,z i ] is a point Three-dimensional coordinates in the cylindrical coordinate system, intensity i It does not change due to the transformation of the coordinate system;

[0021] The sampling unit in the cylindrical coordinate system is a fan-shaped cylinder G with a size of g. ρ ×g θ ×∞, the units are tightly arranged in an orderly manner without overlap to sample the three-dimensional point cloud space. j is the sampling unit at the jth sampling time, all coordinates [ρ i ,θ i ,z i ] falls in unit G j point For unit G j The sampling point is denoted as Unit G j All sampling points are

[0022] For sampling unit G j Perform statistics to get the height of the unit j , intensity j and density j ;

[0023] Find the height coordinate values ​​z of all sampling points j , take the point with the maximum height coordinate value z The height coordinate value z of the point k The height channel value height for this sampling statistics j ; The intensity value of this point k The intensity channel value intensity for this sampling statistics j ; The density channel value density obtained by this sampling j is the number of sampling points N j The statistical value of; the corresponding formula can be expressed as:

[0024] k = argmax(z j )

[0025] height j =z k

[0026] intensity j =intensity k

[0027]

[0028] Where α is a hyperparameter required for density calculation.

[0029] Furthermore, in the rectangular coordinate system and the cylindrical coordinate system, the height coordinate value z i same.

[0030] Furthermore, the sampling information is mapped back to the original point cloud. In addition to recording the original information, each point in the point cloud also records the sector sampling unit z j Three statistics of [height j ,intensity j ,density j ].

[0031] Furthermore, the method for enhancing the point cloud features is as follows:

[0032] Expanding point cloud for In the rectangular coordinate system, a point in the point cloud In vector form it is:

[0033]

[0034] Among them, except [x i ,y i ,z i ,intensity i ] In addition to the original information, is the height statistic, intensity statistic and density statistic of the point, that is, the sampling unit G j Statistics of j ,intensity j ,density j ].

[0035] Furthermore, a cylindrical sampling unit is used to sample the point cloud in a rectangular coordinate system, and each sampling unit obtains a height statistic, an intensity statistic, and a density statistic;

[0036] Find the point with the maximum height coordinate value among all sampling points. The height statistic value recorded at this point is the height statistic value of the unit, the intensity statistic value recorded at this point is the intensity statistic value of the unit, and the density statistic value recorded at this point is the density statistic value of the unit.

[0037] The bird's-eye view map generated in the rectangular coordinate system retains the information of the uniform bird's-eye view map and can directly replace the bird's-eye view map in the existing point cloud target detection method.

[0038] Furthermore, the method for generating a uniform bird's-eye view image suitable for the existing point cloud target detection method based on dimensionality reduction by using the enhanced information in a rectangular coordinate system is as follows:

[0039] The sampling unit in the rectangular coordinate system is a cylinder G, with a size of g. x ×g y ×∞, the units are tightly and orderly arranged without overlap to sample the 3D point cloud space, and all coordinates [x i ,y i ,z i ] falls in unit G q point For unit G q The sampling point is denoted as Unit G q All sampling points are

[0040] Different from the fan-shaped sampling unit, the cylindrical sampling unit does not change due to different spatial positions, and the point cloud range it covers remains fixed.

[0041] In one sampling, find all sampling points The height coordinate value z q , take the point with the maximum height coordinate z The height statistics of the point The height value obtained for this sampling statistics is height q ; The strength statistics of this point The intensity value obtained for this sampling statistics q; Density statistics of the point The density value density obtained for this sampling statistics q ; The corresponding formula can be expressed as:

[0042] k = argmax(z q )

[0043]

[0044]

[0045]

[0046] The unit statistical information can form three channels of a two-dimensional image according to the spatial coordinates and arrangement of the unit during sampling, namely the height channel, intensity channel and density channel; the information of each channel retains the result of uniform sampling, and the obtained two-dimensional image is a uniform bird's-eye view.

[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0048] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0049] Compared with the prior art, the significant advantages of the present invention are: the bird's-eye views used in the existing point cloud target detection methods based on dimensionality reduction are all generated in a rectangular coordinate system. Except for the different statistical information of the sampling units, the other steps are the same as the third step. Therefore, the bird's-eye view generated in the third step not only retains the result of uniform sampling, but can also be directly applied to the existing point cloud target detection methods based on dimensionality reduction without making other changes to the existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flow chart of a method for generating a uniform bird's-eye view for sparse point cloud target detection. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following embodiments and appended Figure 1 The present invention is described in further detail.

[0052] In the code implementation, the point cloud It is stored in the form of an N×4 two-dimensional matrix, which has N rows representing N points in the point cloud and 4 columns representing 4 features for each point, i.e.

[0053]

[0054] After clipping, at least all points in the point cloud (x, y) should satisfy the lower bound x min and min , upper bound x max and max .

[0055] Calculate the coordinates of each point in the cylindrical coordinate system And the corresponding lower bound ρ min and θ min With upper bound ρ max and θ max .

[0056] The sampling unit in the cylindrical coordinate system is a fan-shaped cylinder with a size of g. ρ ×g θ ×∞, then for every point The following calculation can complete the sampling of the point cloud and obtain the spatial position of the uniform sampling unit during sampling (ρ j ,θ j ):

[0057]

[0058]

[0059] The size of the fan-shaped sampling unit is fixed, but as the distance between the space where it is located and the coordinate origin increases, the point cloud range covered by the unit also increases. Therefore, the sampling using the fan-shaped sampling unit ensures that the number of laser lines passing through the unit at different spatial positions is the same under ideal conditions, which is uniform sampling.

[0060] The above formula calculates (ρ j ,θ j ) The same points belong to the same sampling point These sampling points are sorted according to the height coordinate value z and the point with the maximum height coordinate value z is found. The height coordinate value z of the point k The height channel value height obtained for this sampling j ; The reflection intensity value of this point k The intensity channel value obtained for this sampling statistics is intensity j ; The density channel value density obtained by this sampling j is the number of sampling points N j The corresponding formula can be expressed as:

[0061]

[0062]

[0063]

[0064]

[0065] Where α is a hyperparameter required for density calculation, which is generally the number of laser lines or lidar lines passing through the cell under ideal conditions. Here, the hyperparameter α = 64, which means that the lidar used is a 64-line lidar.

[0066] Expanding point cloud for That is, use an N×7 two-dimensional matrix to store a point in the point cloud. In vector form it is:

[0067]

[0068] Among them, except [x i ,y i ,z i ,intensity i ] In addition to the original information, is the statistical value of the height, intensity and density of the point, that is, the sampling unit G j Statistics of [height j ,intensity j ,density j ].

[0069] The sampling unit in the rectangular coordinate system is a cylinder with a size of g. x ×g y ×∞, then for every point The following calculation can complete the sampling of the point cloud and obtain the spatial position of the uniform sampling unit during sampling (x j ,y j ):

[0070]

[0071]

[0072] The above formula calculates (x j ,y j ) The same points belong to the same sampling point These sampling points are sorted according to the height coordinate value z and the point with the maximum height coordinate value z is found. The height statistics of the point The height value obtained for this sampling statistics is height q ; The strength statistics of this point The intensity value obtained for this sampling statistics q ; Density statistics of the point The density value density obtained for this sampling statistics q The corresponding formula can be expressed as:

[0073]

[0074]

[0075]

[0076]

[0077] The size of the generated uniform bird's-eye view image is H×W×3, where 3 means that the bird's-eye view image has 3 channels. The initial values ​​of the bird's-eye view image are all 0, and then the sampling unit is divided into two channels according to its spatial position (x j ,y j ) Fill the three features of the unit into the corresponding positions of the three channels of the bird's-eye view. H and W can be calculated as follows:

[0078]

[0079]

[0080] The uniform bird's-eye view obtained in this way can directly replace the bird's-eye view in the point cloud target detection method based on dimensionality reduction.

[0081] Furthermore, the present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0082] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0083] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0084] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for generating a uniform bird's-eye view image for sparse point cloud target detection, characterized in that: include: The point cloud is sampled in a cylindrical coordinate system with reduced dimension, and a two-dimensional image is used to represent the three-dimensional point cloud to obtain a uniform bird's-eye view. Map the sampling information back to the original point cloud so that all points in the original point cloud record the sampling information of the sampling unit; The point cloud is sampled in a rectangular coordinate system with reduced dimension. The information of the recorded uniform bird's-eye view is used to represent the three-dimensional point cloud with a two-dimensional image, and a uniform bird's-eye view applied to sparse point cloud target detection is obtained. The details are as follows: Let the sampling unit in the rectangular coordinate system be a cylinder G, with a size of g. x ×g y ×∞, the units are tightly and orderly arranged without overlap to sample the 3D point cloud space, and all coordinates [x i ,y i ,z i ] falls in unit G q point For unit G q The sampling point is denoted as Unit G q All sampling points are In one sampling, find all sampling points The height coordinate value z q , take the point with the maximum height coordinate z The height statistics of the point The height value obtained for this sampling statistics is height q ; The strength statistics of this point The intensity value obtained for this sampling statistics q ; Density statistics of the point The density value density obtained for this sampling statistics q ; The corresponding formula can be expressed as: k=argmax(z q ) The unit statistical information can form three channels of a two-dimensional image according to the spatial coordinates and arrangement of the unit during sampling, namely the height channel, intensity channel and density channel; the information of each channel retains the result of uniform sampling, and the obtained two-dimensional image is a uniform bird's-eye view.

2. The uniform bird's-eye view generation method for sparse point cloud target detection according to claim 1, characterized in that: The point cloud is sampled in a cylindrical coordinate system with reduced dimension, and a two-dimensional image is used to represent the three-dimensional point cloud to obtain a uniform bird's-eye view, including: In the cylindrical coordinate system, the point cloud is sampled using a fan-shaped cylinder as a sampling unit, and each sampling unit is statistically obtained to obtain the height statistics, intensity statistics, and density statistics; Find the point with the maximum height coordinate value among all sampling points. The height coordinate value of this point is the height statistic of the unit, the reflection intensity of this point is the intensity statistic of the unit, and the related calculation of the number of sampling points in the unit is the density statistic of the unit.

3. The uniform bird's-eye view generation method for sparse point cloud target detection according to claim 2, characterized in that: Perform dimensionality reduction sampling on the point cloud in the cylindrical coordinate system, use a two-dimensional image to represent the three-dimensional point cloud, and obtain a uniform bird's-eye view. The specific method is as follows: The point cloud after clipping and other preprocessing is recorded as N is the point cloud The number of midpoints, 4 is the characteristic number of the point; in the rectangular coordinate system, a point in the point cloud In vector form it is: Among them, the coordinate origin is the origin of the point cloud, [x i ,y i ,z i ] is a point Three-dimensional coordinates in a rectangular coordinate system with the Z axis perpendicular to the ground, intensity i is the laser reflection intensity at this point; correspondingly, in the cylindrical coordinate system, point In vector form it is: Among them, [ρ i ,θ i ,z i ] is a point Three-dimensional coordinates in the cylindrical coordinate system, intensity i It does not change due to the transformation of the coordinate system; The sampling unit in the cylindrical coordinate system is a fan-shaped cylinder G with a size of g. ρ ×g θ ×∞, the units are tightly and orderly arranged without overlap to sample the three-dimensional point cloud space; denoted by G j is the sampling unit at the jth sampling time, all coordinates [ρ i ,θ i ,z i ] falls in unit G j point For unit G j The sampling point is denoted as Unit G j All sampling points are For sampling unit G j Perform statistics to get the height of the unit j , intensity j and density j ; Find the height coordinate values ​​z of all sampling points j , take the point with the maximum height coordinate value z The height coordinate value z of the point k The height channel value height for this sampling statistics j ; The intensity value of this point k The intensity channel value intensity for this sampling statistics j ; The density channel value density obtained by this sampling j is the number of sampling points N j The statistical value of; the corresponding formula can be expressed as: k=argmax(z j ) height j =z k intensity j =intensity k Where α is a hyperparameter required for density calculation.

4. The uniform bird's-eye view generation method for sparse point cloud target detection according to claim 3 is characterized in that: In the rectangular coordinate system and cylindrical coordinate system, the height coordinate value z i same.

5. The uniform bird's-eye view generation method for sparse point cloud target detection according to claim 1, characterized in that: The sampling information is mapped back to the original point cloud. In addition to recording the original information, each point in the point cloud also records the sector sampling unit G. j Three statistics of [height j ,intensity j ,demsity j ].

6. The uniform bird's-eye view generation method for sparse point cloud target detection according to claim 1 or 5, characterized in that: Map the sampling information back to the original point cloud as follows: Expanding point cloud for In the rectangular coordinate system, a point in the point cloud In vector form it is: Among them, except [x i ,y i ,z i ,intensity i ] In addition to the original information, is the height statistic, intensity statistic and density statistic of the point, that is, the sampling unit G j Statistics of [height j ,intensity j ,density j ].

7. The uniform bird's-eye view generation method for sparse point cloud target detection according to claim 1, characterized in that: The point cloud is sampled using cylindrical sampling units in a rectangular coordinate system, and each sampling unit obtains statistical values ​​of height, intensity and density; Find the point with the maximum height coordinate value among all sampling points. The height statistic value recorded at this point is the height statistic value of the unit, the intensity statistic value recorded at this point is the intensity statistic value of the unit, and the density statistic value recorded at this point is the density statistic value of the unit.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images