Point cloud data processing method, device and equipment, storage medium and vehicle

By using a dynamic voxelization grouping method, the problem of missing information in point cloud voxelization is solved, enabling more accurate feature data acquisition and more efficient computation.

CN119006758APending Publication Date: 2024-11-22BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310552627.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In existing technologies, the voxelization method for point clouds results in the loss of point cloud information within voxels, leading to inaccurate feature data and an inability to effectively utilize redundant point cloud information.

Method used

A dynamic voxelization grouping method is adopted. By determining the index relationship between each point and voxel in the 3D point cloud data, the voxel data of the points contained in each voxel are calculated, and the feature data of the voxels are obtained based on these data.

Benefits of technology

It reduces the loss of point cloud information within voxels, improves the accuracy of feature data, reduces computational costs, and improves computational efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006758A_ABST
    Figure CN119006758A_ABST
Patent Text Reader

Abstract

The invention relates to a point cloud data processing method and device, equipment, a storage medium and a vehicle, and the method comprises the steps: obtaining three-dimensional point cloud data, carrying out the dynamic voxelization grouping of the three-dimensional point cloud data, determining the index relation between each point and voxel in the three-dimensional point cloud data, and carrying out the processing of the three-dimensional point cloud data according to the index relation between each point and voxel in the three-dimensional point cloud data. Calculating voxel data of each point in all the points contained in each voxel, and obtaining feature data of each voxel based on the voxel data of each point in all the points contained in each voxel. Compared with fixed voxelization in the prior art, dynamic voxelization grouping is carried out on the three-dimensional point cloud data, missing of point cloud information in voxels can be reduced, meanwhile, the point clouds in the voxels contain more information, and then the obtained feature data are more accurate. Moreover, the dynamic voxelization does not need voxel filling, so that the calculation cost is reduced, and the operation efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a point cloud data processing method, device, equipment, storage medium and vehicle. Background Art

[0002] In the existing process of obtaining the feature data of voxels, the point cloud needs to be voxelized first, and then the feature data of the voxels is determined in combination with the point cloud data. Among them, the existing point cloud voxelization method is fixed voxelization, that is, the space is evenly divided into a fixed number of voxels K in combination with the preset memory quantity, and each voxel also stores a fixed number of points T and their corresponding data, so that the memory usage can be maximized; however, in the subsequent grouping process of the point cloud, the set number of voxels cannot be greater than K, and the number of points stored in the voxel cannot be greater than T. For the case where the corresponding number is less than K and T, the voxels and points can be filled, but because the spatial position of each voxel is fixed, the number of points in the spatial position of the voxel is likely to be greater than T. At this time, it is necessary to immediately discard the excess points to ensure that the storage number of points in the voxel is not exceeded. The points that are discarded immediately will cause the point cloud information in the voxel to be missing, and the obtained feature data is inaccurate. Therefore, how to obtain more accurate voxel feature data has become an urgent problem to be solved. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides a point cloud data processing method, device, equipment, storage medium and vehicle.

[0004] In a first aspect, an embodiment of the present disclosure provides a point cloud data processing method, comprising:

[0005] Obtain 3D point cloud data;

[0006] Dynamically voxelizing and grouping the three-dimensional point cloud data, and determining an index relationship between each point and a voxel in the three-dimensional point cloud data;

[0007] Calculating voxel data of each point among all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data;

[0008] Based on the voxel data of each point among all the points included in the voxels, the feature data of the voxels are obtained.

[0009] In some embodiments, the obtaining of three-dimensional point cloud data includes:

[0010] Obtain point cloud data of surrounding scenes collected by radar;

[0011] Establishing a Cartesian coordinate system with the position of the radar as the coordinate origin;

[0012] The point cloud data is converted into the Cartesian coordinate system to obtain the three-dimensional point cloud data.

[0013] In some embodiments, the dynamically voxelizing and grouping the three-dimensional point cloud data includes:

[0014] Determine the number of voxels;

[0015] The three-dimensional point cloud data is divided into the number of voxels.

[0016] In some embodiments, determining the number of voxels comprises:

[0017] Determining range data of the three-dimensional point cloud data;

[0018] The number of voxels is obtained by dividing the range data of the three-dimensional point cloud data by a preset voxel resolution.

[0019] In some embodiments, determining the index relationship between each point in the three-dimensional point cloud data and a voxel includes:

[0020] Determine the coordinate set corresponding to each voxel and the index of each voxel;

[0021] For each point in the three-dimensional point cloud data, determine the target coordinate set to which the coordinates of each point belong, determine the index of the voxel corresponding to the target coordinate set as the index of the voxel where each point is located, and obtain the index relationship between each point and the voxel in the three-dimensional point cloud data.

[0022] In some embodiments, the step of calculating the voxel data of each point among all points contained in each voxel according to the index relationship between each point in the three-dimensional point cloud data and the voxel includes:

[0023] Determine all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data;

[0024] Calculate the average coordinates of all points within the voxel;

[0025] For each point in all the points in the voxel, subtract the coordinate of each point from the average coordinate to obtain the first offset position coordinate of each point;

[0026] Subtract the coordinates of each point from the voxel body center coordinates to obtain the second offset position coordinates of each point;

[0027] Voxel data of each point among all points contained in each voxel is obtained according to the coordinates of each point among all points in the voxel, the first offset position coordinates and the second offset position coordinates.

[0028] In some embodiments, obtaining feature data of each voxel based on voxel data of each point in all points contained in each voxel includes:

[0029] Performing preliminary feature extraction processing on the voxel data of each point among all the points contained in each voxel to obtain local feature data of each voxel;

[0030] Performing maximum pooling processing on the local feature data of each voxel to obtain the global feature data of each voxel;

[0031] The local feature data of each voxel and the global feature data of each voxel are aggregated to obtain the feature data of each voxel.

[0032] In a second aspect, an embodiment of the present disclosure provides a point cloud data processing device, comprising:

[0033] An acquisition module, used to acquire three-dimensional point cloud data;

[0034] A partitioning module, used for dynamically voxelizing and grouping the three-dimensional point cloud data, and determining an index relationship between each point and voxel in the three-dimensional point cloud data;

[0035] A calculation module, used for calculating voxel data of each point among all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data;

[0036] The obtaining module is used to obtain the feature data of each voxel based on the voxel data of each point among all the points contained in each voxel.

[0037] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0038] Memory;

[0039] Processor; and

[0040] Computer programs;

[0041] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.

[0042] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in the first aspect.

[0043] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the point cloud data processing method as described above is implemented.

[0044] In a sixth aspect, an embodiment of the present disclosure further provides a vehicle, including:

[0045] Memory;

[0046] Processor; and

[0047] Computer programs;

[0048] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the point cloud data processing method as described above.

[0049] The point cloud data processing method, device, equipment, storage medium and vehicle provided in the embodiments of the present disclosure acquire three-dimensional point cloud data, dynamically voxelize and group the three-dimensional point cloud data, and determine the index relationship between each point and the voxel in the three-dimensional point cloud data. According to the index relationship between each point and the voxel in the three-dimensional point cloud data, the voxel data of each point in all the points contained in each voxel is calculated, and the feature data of each voxel is obtained based on the voxel data of each point in all the points contained in each voxel. Compared with the fixed voxelization in the prior art, the present disclosure dynamically voxelizes and groups the three-dimensional point cloud data. The dynamic voxelization can keep the voxel grouping unchanged, that is, keep the voxel size and the number of voxels unchanged. When selecting points, the index relationship between the points and the voxels is retained, which can reduce the loss of point cloud information in the voxel, and at the same time make the point cloud in the voxel contain more information, so that the obtained feature data is more accurate. Moreover, the dynamic voxelization does not require voxel filling, which reduces the calculation cost and improves the operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A flow chart of a point cloud data processing method provided in an embodiment of the present disclosure;

[0053] Figure 2A flow chart of a point cloud data processing method provided by another embodiment of the present disclosure;

[0054] Figure 3 A flow chart of a point cloud data processing method provided by another embodiment of the present disclosure;

[0055] Figure 4 A schematic diagram of the structure of a point cloud data processing device provided in an embodiment of the present disclosure;

[0056] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0059] The present disclosure provides a point cloud data processing method, which is described below in conjunction with a specific embodiment.

[0060] Figure 1 This is a flow chart of the point cloud data processing method provided in the embodiment of the present disclosure. The method can be applied to a vehicle-mounted terminal, and the method can be applied to a scene where point cloud data is processed, gridding the point cloud voxels, reducing the missing point cloud within the voxel, and making the point cloud within the voxel contain more information, thereby making the obtained feature data more accurate. It is understandable that the point cloud data processing method provided in the embodiment of the present disclosure can also be applied in other scenes.

[0061] Below Figure 1 The point cloud data processing method shown in FIG. 1 is introduced, and the specific steps of the method are as follows:

[0062] S101. Acquire three-dimensional point cloud data.

[0063] The method of obtaining the three-dimensional point cloud data can use the existing method. For example, the vehicle terminal can obtain the three-dimensional point cloud data of the scene around the vehicle or the acquisition device. In some embodiments, the vehicle is equipped with acquisition devices such as laser radar and millimeter wave radar. The vehicle terminal can obtain the three-dimensional point cloud data collected by the laser radar, millimeter wave radar, etc.

[0064] S102: Dynamically voxelize and group the three-dimensional point cloud data, and determine an index relationship between each point and voxel in the three-dimensional point cloud data.

[0065] After acquiring the three-dimensional point cloud data, the processing device (such as a vehicle-mounted terminal) can dynamically voxelize and group the three-dimensional point cloud data, and determine the index relationship between each point and the voxel in the three-dimensional point cloud data. For example, the spatial cube composed of the three-dimensional point cloud data is divided into multiple small cubes (small cubes can be regarded as voxels), wherein each small cube contains a part of the point cloud data in the three-dimensional point cloud data, and then it can be determined in which voxel each point in the three-dimensional point cloud data is, that is, the index relationship between each point in the three-dimensional point cloud data and the voxel is determined. The use of dynamic voxelization grouping can keep the grouping of voxels unchanged, that is, keep the voxel size and the number of voxels unchanged. When selecting points, the index relationship between points and voxels is retained, which can reduce the loss of point cloud information in the voxel, and at the same time make the point cloud in the voxel contain more information, thereby making the obtained feature data more accurate.

[0066] S103 . Calculate voxel data of each point among all points contained in each voxel according to the index relationship between each point in the three-dimensional point cloud data and the voxel.

[0067] After determining the index relationship between each point and the voxel in the three-dimensional point cloud data, the vehicle terminal can calculate the voxel data of each point in all the points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data. For example, each point in the three-dimensional point cloud data is a 4-dimensional point, represented by (x, y, z, r), where x is the horizontal coordinate, y is the vertical coordinate, z is the vertical coordinate, and r is the reflection intensity. Through calculation, each point can be expanded to a 10-dimensional point. This embodiment is just an example and does not specifically limit the dimension of the data.

[0068] S104 . Obtain feature data of each voxel based on the voxel data of each point among all the points contained in each voxel.

[0069] After calculating the voxel data of each point among all the points contained in each voxel, the feature data of each voxel can be obtained according to the voxel data of each point among all the points contained in each voxel. Specifically, the feature data of each voxel can be obtained through a pre-trained model, or the feature data of each voxel can be extracted through a convolutional neural network, without limitation.

[0070] The embodiment of the present disclosure obtains three-dimensional point cloud data, performs dynamic voxelization grouping on the three-dimensional point cloud data, and determines the index relationship between each point and the voxel in the three-dimensional point cloud data. According to the index relationship between each point and the voxel in the three-dimensional point cloud data, the voxel data of each point in all the points contained in each voxel is calculated, and the feature data of each voxel is obtained based on the voxel data of each point in all the points contained in each voxel. Compared with the fixed voxelization in the prior art, the present disclosure performs dynamic voxelization grouping on the three-dimensional point cloud data. The dynamic voxelization can keep the voxel grouping unchanged, that is, keep the voxel size and the number of voxels unchanged. When selecting points, the index relationship between the points and the voxels is retained, which can reduce the loss of point cloud information in the voxel, and at the same time make the point cloud in the voxel contain more information, thereby making the obtained feature data more accurate. Moreover, dynamic voxelization does not require voxel filling, which reduces the calculation cost and improves the operation efficiency.

[0071] Figure 2 A flow chart of a point cloud data processing method provided by another embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the method includes the following steps:

[0072] S201, acquiring three-dimensional point cloud data collected by a collection device.

[0073] For example, a vehicle is equipped with a laser radar, a millimeter wave radar, etc. The vehicle terminal can obtain the point cloud data of the surrounding scene collected by the radar. Specifically, the radar transmits the collected point cloud data to the vehicle terminal, and the vehicle terminal receives the point cloud data transmitted by the radar.

[0074] S202: Establish a Cartesian coordinate system with the position of the acquisition device as the coordinate origin.

[0075] The vehicle-mounted terminal establishes a Cartesian coordinate system based on the position of the radar and takes the position of the radar as the coordinate origin. Three non-coplanar axes intersecting at the origin constitute a three-dimensional radial coordinate system. A radial coordinate system with equal measurement units on the three axes is called a three-dimensional Cartesian coordinate system or a spatial Cartesian coordinate system. A Cartesian coordinate system with three axes perpendicular to each other is called a spatial Cartesian rectangular coordinate system, otherwise it is called a spatial Cartesian oblique coordinate system. The Cartesian coordinate system of this embodiment is a spatial Cartesian coordinate system or a three-dimensional Cartesian coordinate system.

[0076] S203: Convert the point cloud data into the Cartesian coordinate system to obtain the three-dimensional point cloud data.

[0077] Since the Cartesian coordinate system is a coordinate system established with the position of the radar as the coordinate origin, and the point cloud data is in the image coordinate system, it is necessary to convert the point cloud data in the image coordinate system to the Cartesian coordinate system to obtain the three-dimensional point cloud data. For example, the conversion can be performed through a conversion matrix, and the vehicle-mounted terminal calculates the conversion matrix from the coordinate system where the point cloud data is located to the Cartesian coordinate system, and then converts the point cloud data to the Cartesian coordinate system according to the conversion matrix to obtain the three-dimensional point cloud data. For example, the vehicle-mounted terminal can obtain the position of each point in the point cloud data in the image coordinate system, and can obtain the position relationship between each point and the radar, and then based on the position of each point in the image coordinate system and the position relationship between each point and the radar, calculate the conversion matrix from the coordinate system where the point cloud data is located to the Cartesian coordinate system, and then convert the position of each point in the image coordinate system to the Cartesian coordinate system through the conversion matrix to obtain the three-dimensional point cloud data.

[0078] S204: Determine the number of voxels.

[0079] To dynamically voxelize and group the three-dimensional point cloud data, the number of divisions, that is, the number of voxels, must first be determined. The number of divisions can be preset. In some embodiments, the number of voxels is represented by (voxel x ,voxel y ,voxel z ), voxel x Indicates the number of voxels in the horizontal direction, voxel y Indicates the number of voxels vertically, voxel z Indicates the number of voxels in the longitudinal direction. For example, if the 3D point cloud data is divided into 18 voxels, the number of voxels is expressed as (3, 3, 2), the number of voxels in the transverse direction is 3, the number of voxels in the vertical direction is 3, and the number of voxels in the longitudinal direction is 2.

[0080] Specifically, S204 includes but is not limited to S2041 and S2042:

[0081] S2041. Determine range data of the three-dimensional point cloud data.

[0082] In practice, the data collection space of the collection device is fixed, so the range of the data collection space is also fixed (such as the detection range of the lidar). The range of the collection space can be determined as the range of the three-dimensional point cloud data, and then the range data of the three-dimensional point cloud data can be obtained. In addition, other existing methods can also be used to determine the range data of the three-dimensional point cloud data, which is not limited here.

[0083] In some embodiments, the range data of the point cloud data is determined as (range x , range y , rangez ), range x Indicates the range of the vehicle, for example -60m to 60m, range y Indicates the range in front and behind the vehicle, for example, 0m to 120m, range z Indicates the range of the vehicle up and down, for example, -10m to 10m.

[0084] S2042: Obtain the number of voxels based on the range data of the three-dimensional point cloud data and a preset voxel resolution.

[0085] The voxel resolution can be preset in advance to determine the size of a single voxel. After determining the voxel resolution, the number of voxels can be determined in combination with the range data of the 3D point cloud data. It can also be understood that after obtaining a portion of the acquisition space occupied by the 3D point cloud, the number of voxels in this portion of space is determined according to the voxel resolution.

[0086] For example, the user preset voxel resolution is (grid x , grid y , grid z ), grid x Indicates the horizontal voxel resolution, such as 0.2m, grid y Indicates the vertical voxel resolution, such as 0.2m, grid z Indicates the longitudinal voxel resolution, for example, 0.2m. The voxel resolution is used to characterize the size of a single voxel, for example, 0.2m×0.2m×0.2m. The vehicle-mounted terminal can obtain the number of voxels based on the range data of the three-dimensional point cloud data and the preset voxel resolution. Specifically, the vehicle-mounted terminal obtains the size data of the three-dimensional point cloud data based on the range data of the three-dimensional point cloud data, and further obtains the number of voxels based on the size data of the three-dimensional point cloud data divided by the preset voxel resolution.

[0087] S205: Determine a spatial coordinate set corresponding to each voxel and an index of each voxel.

[0088] In this step, the spatial coordinate set corresponding to each voxel is determined according to the range data of the three-dimensional point cloud data and the number of voxels, and the index of each voxel is determined, and each voxel corresponds to an index.

[0089] Furthermore, since the voxel resolution is determined and the space to be voxelized is also determined, the position of each voxel in space and its occupied range can be described by coordinates. For example, the occupied range of a voxel in space can be represented by a set of spatial coordinates, thus obtaining the set of spatial coordinates corresponding to each voxel. In addition, the relative position relationship of each voxel in space can be used as the index of each voxel. For example, the index can be the third voxel in the horizontal direction, the second voxel in the vertical direction, and the first voxel in the longitudinal direction in space. In this way, the corresponding spatial coordinate set and index can be determined for each voxel.

[0090] S206. For each point in the three-dimensional point cloud data, determine the target space coordinate set to which the coordinates of each point belong, determine the index of the voxel corresponding to the target space coordinate set as the index of the voxel where each point is located, and obtain the index relationship between each point in the three-dimensional point cloud data and the voxel.

[0091] Further, for each point in the three-dimensional point cloud data, the target space coordinate set to which the coordinates of each point belong is determined, and the index of the voxel corresponding to the target space coordinate set is determined as the index of the voxel where each point is located, so as to obtain the index relationship between each point in the three-dimensional point cloud data and the voxel. pi (1, 2, 1) indicates that the voxel where the point is located is the first voxel in the horizontal direction, the second voxel in the vertical direction, and the first voxel in the longitudinal direction. It can be understood that each voxel has an index, and all points in the voxel correspond to the index of the voxel.

[0092] In some embodiments, each point p i The voxel index is And at the same time to meet Thus, an index relationship between each point and voxel in the three-dimensional point cloud data is obtained.

[0093] Specifically, the spatial coordinate set corresponding to each voxel can be determined to determine in which spatial coordinate set the coordinates of each point are located, and then the index relationship between the coordinates of each point and the voxel can be obtained.

[0094] Among them, pi (x, y, z) represents any point in the three-dimensional point cloud data, (rize x , rize y , rize z ) represents the size data of the point cloud data, (voxel x ,voxel y ,voxel z ) represents the number of voxels. For example, the range data of point cloud data is (range x , range y , rangez ), range x Indicates the range of the vehicle, for example -60m to 60m, range y Indicates the range in front and behind the vehicle, for example, 0m to 120m, range z Indicates the range of the vehicle up and down, for example -10m to 10m, then calculate the absolute value of the difference between the start and end points of each dimension of the range data (left and right range, front and back range, up and down range) to obtain the size data, then (rize x , rize y , rize z ) is (120m, 120m, 20m).

[0095] S207 . Calculate voxel data of each point among all points contained in each voxel according to the index relationship between each point in the three-dimensional point cloud data and the voxel.

[0096] Specifically, the implementation process and principle of S208 and S103 are the same, and will not be repeated here.

[0097] S208 . Obtain feature data of each voxel based on the voxel data of each point among all the points contained in each voxel.

[0098] Specifically, the implementation process and principle of S208 and S104 are the same, and will not be repeated here.

[0099] The disclosed embodiment obtains point cloud data of the surrounding scene collected by the radar, establishes a Cartesian coordinate system with the position of the radar as the coordinate origin, and converts the point cloud data into the Cartesian coordinate system to obtain the three-dimensional point cloud data. Further, the number of voxels is determined, the three-dimensional point cloud data is divided into the number of voxels, the coordinate set corresponding to each voxel and the index of each voxel are determined, and for each point in the three-dimensional point cloud data, the target coordinate set to which the coordinates of each point belong is determined, and the index of the voxel corresponding to the target coordinate set is determined as the index of the voxel where each point is located, and the index relationship between each point and the voxel in the three-dimensional point cloud data is obtained. Then, according to the index relationship between each point and the voxel in the three-dimensional point cloud data, the voxel data of each point in all the points contained in each voxel is calculated, and based on the voxel data of each point in all the points contained in each voxel, the feature data of each voxel is obtained. Compared with the fixed voxelization in the prior art, the present invention performs dynamic voxelization grouping on three-dimensional point cloud data. Dynamic voxelization can keep the voxel grouping unchanged, that is, keep the voxel size and the number of voxels unchanged. When selecting points, the index relationship between points and voxels is retained, which can reduce the loss of point cloud information in the voxel and make the point cloud in the voxel contain more information, thereby making the obtained feature data more accurate. Moreover, dynamic voxelization does not require voxel filling, which reduces the calculation cost and improves the operation efficiency.

[0100] Figure 3 A flow chart of a point cloud data processing method provided by another embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the method includes the following steps:

[0101] S301, obtaining three-dimensional point cloud data.

[0102] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0103] S302: Dynamically voxelize and group the three-dimensional point cloud data, and determine an index relationship between each point and voxel in the three-dimensional point cloud data.

[0104] Specifically, the implementation process and principle of S302 and S102 are the same, and will not be repeated here.

[0105] S303: Determine all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data.

[0106] The vehicle terminal obtains the index of the voxel corresponding to each point from the index relationship between each point and the voxel in the three-dimensional point cloud data, and determines all the points contained in each voxel, that is, determines all the points with the same index. pi(1, 2, 1) contains 30 points.

[0107] S304, calculating the average coordinates of all points in the voxel.

[0108] After determining all the points contained in each voxel, the vehicle terminal calculates the average coordinates of all the points in the voxel. The average coordinates are calculated by dividing the vector sum of the coordinates of all the points in the voxel by the number of points in the voxel to obtain the average coordinates of all the points in the voxel (x m ,y m , z m ), the average coordinates of all points within the voxel are also called the point cloud center of the voxel.

[0109] S305 . For each point in all the points in the voxel, subtract the coordinate of each point from the average coordinate to obtain the first offset position coordinate of each point.

[0110] Subtract the coordinates of each point from the average coordinates to obtain the first offset position coordinates of each point relative to the average coordinates. The first offset position coordinates are expressed as (x c ,y c , z c )express.

[0111] S306: Subtract the coordinates of each point from the voxel body center coordinates to obtain the second offset position coordinates of each point.

[0112] Subtract the coordinates of each point from the voxel center coordinates to obtain the second offset position coordinates of each point relative to the voxel center coordinates. The second offset position coordinates are expressed as (x p ,y p , z p ). For example, a voxel can be regarded as a small cube, and the center (or center) of the small cube can be the voxel center.

[0113] S307 , obtaining voxel data of each point among all points contained in each voxel according to the coordinates of each point among all points in the voxel, the first offset position coordinates, and the second offset position coordinates.

[0114] The vehicle terminal can obtain the voxel data of each point in all the points contained in each voxel according to the coordinates of each point in all the points in the voxel, the first offset position coordinates and the second offset position coordinates. In some embodiments, the vehicle terminal arranges the coordinates of the point, the first offset position coordinates and the second offset position coordinates in a certain order to obtain the voxel data of the point in the three-dimensional point cloud data, and the voxel data is multi-dimensional data. For example, the coordinates of the point are expanded from four-dimensional data (x, y, z, r) to ten-dimensional data (x, y, z, r, x c ,y c , z c , x p ,y p , z p ).

[0115] S308 , performing preliminary feature extraction processing on the voxel data of each point among all the points contained in each voxel to obtain local feature data of each voxel.

[0116] In some embodiments, all points in the same voxel are input into a point cloud feature extraction module, and the data is processed using a fully connected layer, a batch normalization layer, and an activation function to obtain local features.

[0117] S309 , performing maximum pooling processing on the local feature data of each voxel to obtain global feature data of each voxel.

[0118] After obtaining the local feature data of each voxel, the vehicle-mounted terminal performs maximum pooling processing on the local feature data of each voxel to obtain the global feature data of each voxel. Maximum pooling (max-pooling) is to take the point with the largest value in the local acceptance field.

[0119] S310 , aggregate the local feature data of each voxel and the global feature data of each voxel to obtain feature data of each voxel.

[0120] Furthermore, the vehicle-mounted terminal aggregates the local feature data of each voxel and the global feature data of each voxel to obtain feature data of each voxel.

[0121] The disclosed embodiment acquires three-dimensional point cloud data, performs dynamic voxel grouping on the three-dimensional point cloud data, and determines the index relationship between each point and the voxel in the three-dimensional point cloud data. Further, according to the index relationship between each point and the voxel in the three-dimensional point cloud data, all points contained in each voxel are determined, and the average coordinates of all points in the voxel are calculated. For each point in all points in the voxel, the coordinates of each point are subtracted from the average coordinates to obtain the first offset position coordinates of each point, and the coordinates of each point are subtracted from the voxel body center coordinates to obtain the second offset position coordinates of each point. According to the coordinates of each point in all points in the voxel, the first offset position coordinates and the second offset position coordinates, the voxel data of each point in all points contained in each voxel are obtained. Then, the voxel data of each point in all points contained in each voxel is subjected to preliminary feature extraction processing to obtain local feature data of each voxel, and the local feature data of each voxel is subjected to maximum pooling processing to obtain global feature data of each voxel. Then, the local feature data of each voxel and the global feature data of each voxel are aggregated to obtain the feature data of each voxel. Compared with the fixed voxelization in the prior art, the present invention performs dynamic voxelization grouping on the three-dimensional point cloud data. Dynamic voxelization can keep the voxel grouping unchanged, that is, keep the voxel size and the number of voxels unchanged. When selecting points, the index relationship between points and voxels is retained, which can reduce the loss of point cloud information within the voxel, and at the same time make the point cloud within the voxel contain more information, thereby making the obtained feature data more accurate. Moreover, dynamic voxelization does not require voxel filling, which can reduce computing costs and improve computing efficiency.

[0122] Figure 4 The structure diagram of the point cloud data processing device provided in the embodiment of the present disclosure is shown in FIG. The point cloud data processing device may be the vehicle-mounted terminal as described in the above embodiment, or the point cloud data processing device may be a component or assembly in the vehicle-mounted terminal. The point cloud data processing device provided in the embodiment of the present disclosure may execute the processing flow provided in the point cloud data processing method embodiment, such as Figure 4 As shown, the point cloud data processing device 40 includes: an acquisition module 41, a division module 42, a calculation module 43, and an acquisition module 44; wherein the acquisition module 41 is used to acquire three-dimensional point cloud data; the division module 42 is used to dynamically voxelize and group the three-dimensional point cloud data, and determine the index relationship between each point and the voxel in the three-dimensional point cloud data; the calculation module 43 is used to calculate the voxel data of each point among all the points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data; the acquisition module 44 is used to obtain the feature data of each voxel based on the voxel data of each point among all the points contained in each voxel.

[0123] Optionally, when the acquisition module 41 acquires three-dimensional point cloud data, it is specifically used to: acquire point cloud data acquired by an acquisition device; establish a Cartesian coordinate system with the position of the acquisition device as the coordinate origin; and convert the point cloud data into the Cartesian coordinate system to obtain the three-dimensional point cloud data.

[0124] Optionally, when the division module 42 performs dynamic voxel grouping on the three-dimensional point cloud data, it is specifically used to: determine the number of voxels; and divide the three-dimensional point cloud data into the number of voxels.

[0125] Optionally, when the division module 42 determines the number of voxels, it is specifically used to: determine range data of the three-dimensional point cloud data; and obtain the number of voxels based on dividing the range data of the three-dimensional point cloud data by a preset voxel resolution.

[0126] Optionally, when the division module 42 determines the index relationship between each point and the voxel in the three-dimensional point cloud data, it is specifically used to: determine the coordinate set corresponding to each voxel and the index of each voxel; for each point in the three-dimensional point cloud data, determine the target coordinate set to which the coordinates of each point belong, determine the index of the voxel corresponding to the target coordinate set as the index of the voxel where each point is located, and obtain the index relationship between each point and the voxel in the three-dimensional point cloud data.

[0127] Optionally, when the calculation module 43 calculates the voxel data of each point among all the points contained in each voxel according to the index relationship between each point in the three-dimensional point cloud data and the voxel, it is specifically used to: determine all the points contained in each voxel according to the index relationship between each point in the three-dimensional point cloud data and the voxel; calculate the average coordinates of all the points in the voxel; for each point among all the points in the voxel, subtract the coordinates of each point from the average coordinates to obtain the first offset position coordinates of each point; subtract the coordinates of each point from the voxel body center coordinates to obtain the second offset position coordinates of each point; and obtain the voxel data of each point among all the points contained in each voxel according to the coordinates of each point among all the points in the voxel, the first offset position coordinates and the second offset position coordinates.

[0128] Optionally, when the acquisition module 44 obtains the feature data of each voxel based on the voxel data of each point among all the points contained in each voxel, it is specifically used to: perform preliminary feature extraction processing on the voxel data of each point among all the points contained in each voxel to obtain local feature data of each voxel; perform maximum pooling processing on the local feature data of each voxel to obtain global feature data of each voxel; aggregate the local feature data of each voxel and the global feature data of each voxel to obtain the feature data of each voxel.

[0129] Figure 4 The point cloud data processing device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0130] Figure 5 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 5 , which shows a structural schematic diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0131] like Figure 5 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the point cloud data processing method of the embodiment described in the present disclosure. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0132] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0133] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart, thereby implementing the point cloud data processing method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0134] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0135] In addition, an embodiment of the present disclosure also provides a vehicle, comprising: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the point cloud data processing method as described above.

[0136] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0137] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0138] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0139] Obtain 3D point cloud data;

[0140] Dynamically voxelizing and grouping the three-dimensional point cloud data, and determining an index relationship between each point and a voxel in the three-dimensional point cloud data;

[0141] Calculating voxel data of each point among all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data;

[0142] Based on the voxel data of each point among all the points included in the voxels, the feature data of the voxels are obtained.

[0143] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0144] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0145] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0146] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0147] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0148] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0149] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0150] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0151] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A point cloud data processing method, characterized in that: The method comprises: Obtain 3D point cloud data; Dynamically voxelizing and grouping the three-dimensional point cloud data, and determining an index relationship between each point and a voxel in the three-dimensional point cloud data; Calculating voxel data of each point among all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data; Based on the voxel data of each point among all the points included in the voxels, the feature data of the voxels are obtained.

2. The method according to claim 1, characterized in that: The obtaining of three-dimensional point cloud data comprises: Obtain point cloud data collected by the acquisition device; Establishing a Cartesian coordinate system with the position of the acquisition device as the coordinate origin; The point cloud data is converted into the Cartesian coordinate system to obtain the three-dimensional point cloud data.

3. The method according to claim 1, characterized in that The dynamically voxelizing and grouping the three-dimensional point cloud data comprises: Determine the number of voxels; The three-dimensional point cloud data is divided into the number of voxels.

4. The method according to claim 3, characterized in that The determining the number of voxels comprises: Determining range data of the three-dimensional point cloud data; The number of voxels is obtained by dividing the range data of the three-dimensional point cloud data by a preset voxel resolution.

5. The method according to claim 4, characterized in that Determining the index relationship between each point and voxel in the three-dimensional point cloud data includes: Determine the coordinate set corresponding to each voxel and the index of each voxel; For each point in the three-dimensional point cloud data, determine the target coordinate set to which the coordinates of each point belong, determine the index of the voxel corresponding to the target coordinate set as the index of the voxel where each point is located, and obtain the index relationship between each point and the voxel in the three-dimensional point cloud data.

6. The method according to claim 1, characterized in that The step of calculating the voxel data of each point among all the points contained in each voxel according to the index relationship between each point in the three-dimensional point cloud data and the voxel comprises: Determine all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data; Calculate the average coordinates of all points within the voxel; For each point in all the points in the voxel, subtract the coordinate of each point from the average coordinate to obtain the first offset position coordinate of each point; Subtract the coordinates of each point from the voxel body center coordinates to obtain the second offset position coordinates of each point; Voxel data of each point among all points contained in each voxel is obtained according to the coordinates of each point among all points in the voxel, the first offset position coordinates and the second offset position coordinates.

7. The method according to claim 1, characterized in that The step of obtaining feature data of each voxel based on the voxel data of each point among all points contained in each voxel comprises: Performing preliminary feature extraction processing on the voxel data of each point among all the points contained in each voxel to obtain local feature data of each voxel; Performing maximum pooling processing on the local feature data of each voxel to obtain the global feature data of each voxel; The local feature data of each voxel and the global feature data of each voxel are aggregated to obtain the feature data of each voxel.

8. A point cloud data processing device, characterized in that: The device comprises: An acquisition module, used to acquire three-dimensional point cloud data; A partitioning module, used for dynamically voxelizing and grouping the three-dimensional point cloud data, and determining an index relationship between each point and voxel in the three-dimensional point cloud data; A calculation module, used for calculating voxel data of each point among all points contained in each voxel according to the index relationship between each point and the voxel in the three-dimensional point cloud data; The obtaining module is used to obtain the feature data of each voxel based on the voxel data of each point among all the points contained in each voxel.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: Comprising the point cloud data processing device as described in claim 8.