Space occupation voxelization method and device for point cloud data processing

Abstract

The invention discloses a space occupation voxelization method and device for point cloud data processing, and the method comprises the steps: data collection, tracking detection of a dynamic object, point cloud segmentation processing, static point cloud frame stacking, dynamic point cloud frame stacking, dynamic and static point cloud fusion, point cloud voxelization and voxel visibility calculation. According to the method, information such as the category, the offset value, the radar visibility, the camera visibility and the speed of each voxel in a three-dimensional space can be obtained, and high-precision voxelization data provides a theoretical basis for automatic driving and robot systems.

Description

Technical Field

[0001] The present invention belongs to the field of point cloud data processing, and in particular relates to a method and device for spatial occupancy voxelization for point cloud data processing. Background Art

[0002] As an advanced sensing technology, LiDAR technology is widely used in fields that require high-precision positioning and environmental perception, such as autonomous driving, robot navigation, and drone flight control. LiDAR can obtain point cloud data of the surrounding environment in real time by emitting laser pulses and receiving their reflected signals. These point cloud data contain the precise position and shape information of objects in three-dimensional space, and are the basis for building refined environmental models and achieving high-precision perception.

[0003] In lidar data processing, voxelization refers to the process of converting discrete point cloud data into continuous space occupancy data. Voxel is the smallest unit in three-dimensional space, similar to pixels in two-dimensional images. The voxelization process divides the space into units of fixed size and records the point cloud data in the area in each voxel. In this way, the original point cloud is converted into a regular three-dimensional network representation, making the storage and processing of data more efficient and intuitive, and more suitable for real-time environment perception and path planning tasks, especially in the fields of autonomous driving and robot navigation.

[0004] However, there is currently no method that can realize spatial occupancy voxelization in point cloud data processing. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method and device for spatial occupancy voxelization for point cloud data processing.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention discloses a method for spatial occupancy voxelization for point cloud data processing, comprising:

[0008] Step S1: Collect point cloud data, images and vehicle posture data;

[0009] Step S2: Tracking and detecting dynamic objects based on the data collected in step S1, and obtaining the position, size, orientation, speed and rotational angular velocity of the tracking detection frame of each dynamic object in each frame;

[0010] Step S3: segment the point cloud data to determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud or a noise point;

[0011] Step S4: according to the posture data of the ego vehicle in each frame in the entire time sequence, convert the static point cloud in the ego vehicle coordinate system of the frame into the world coordinate system;

[0012] The point cloud of the entire time series in the world coordinate system is superimposed to obtain a static point cloud scene of the time series superimposed frames in the world coordinate system;

[0013] Step S5: Match the tracking detection frame of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling in the tracking detection frame as the object point cloud, and record its position relative to the tracking detection frame, and overlap the point cloud falling in the tracking detection frame in the entire time sequence at the relative position to obtain a dynamic point cloud instance of each dynamic object in the time sequence overlapped frame;

[0014] Step S6: fusing the static point cloud scene of the time-series overlapping frames in the world coordinate system obtained in step S4 with the dynamic point cloud instance of the time-series overlapping frames of each dynamic object obtained in step S5 to obtain a complete overlapping point cloud scene in each frame;

[0015] Step S7: Based on the complete overlapping point cloud scene in each frame, the space around the vehicle is divided into a number of voxels, and the category and position of the point cloud appearing in each voxel are counted to obtain the category and bias value of each voxel;

[0016] Step S8: performing voxel radar visibility calculation and voxel camera visibility calculation on each voxelized scene.

[0017] Based on the above technical solution, the following improvements can be made:

[0018] As a preferred solution, step S6 includes:

[0019] Step S6.1: for the posture data of the ego vehicle in any frame in the entire time sequence, convert the static point cloud scene of the time sequence stacked frames in the world coordinate system obtained in step S4 to the ego vehicle coordinate system to obtain the static point cloud scene in the ego vehicle coordinate system in the frame;

[0020] Step S6.2: placing the dynamic point cloud instance of the time-series stacked frames of each dynamic object obtained in step S5 in the static point cloud scene obtained in step S6.1 according to the position of the tracking detection frame in the frame, so as to merge and fuse the dynamic and static point clouds and obtain a complete stacked point cloud scene in the frame;

[0021] Step S6.3: Repeat steps S6.1 to S6.2 until a complete overlapping point cloud scene is obtained for each frame.

[0022] As a preferred solution, step S7 includes:

[0023] Step S7.1: Based on the complete overlapping point cloud scene in each frame, the space around the vehicle is divided into a number of voxels according to a preset size;

[0024] Step S7.2: if the number of point clouds of a single category in each voxel exceeds a set threshold, the voxel is considered occupied, and the category of the occupied voxel is set to the category to which the point cloud with the largest number of point clouds in the voxel belongs;

[0025] Step S7.3: Count the positions of each point cloud of the same category in the occupied voxel relative to the center point of the voxel, take the average, and use the average as the bias value of the voxel.

[0026] As a preferred solution, step S7 further includes:

[0027] Step S7.4: for all low-position blank voxels, count the number of ground voxels in their neighborhoods. If the number of neighboring ground voxels exceeds a set threshold, fill the blank voxel with a ground voxel.

[0028] At the same time, the bias mean of the neighboring ground voxels is calculated as the bias value of the padded ground voxel.

[0029] As a preferred solution, step S8 includes:

[0030] Step S8.1: Calculate the radar visibility of each voxel of the voxelized scene, specifically:

[0031] Take the position of the ego vehicle radar in each frame as the origin and the position of the offset value of each occupied voxel in the outermost layer of the preset size as the end point, shoot a ray, mark all voxels passed before the first occupied voxel passed by the ray as empty, mark the first occupied voxel passed by the ray as visible, and mark all voxels passed after the ray passes through the first occupied voxel as invisible;

[0032] When the same voxel is traversed by multiple rays, the priorities are as follows from high to low: radar visible, empty, radar invisible;

[0033] Step S8.2: Calculate the camera visibility of each voxel in the voxelized scene, specifically:

[0034] Taking the position of the ego vehicle camera in each frame as the origin, a ray is shot to each pixel of each picture in each frame, and the ray is back-projected to the ego vehicle coordinate system through the ego vehicle's posture data. All voxels in the preset space are pre-marked as invisible to the camera, all voxels passed before the first occupied voxel passed by the ray are marked as empty, and the first occupied voxel passed by the ray is marked as visible to the camera;

[0035] When the same voxel is traversed by multiple rays, their priorities are from high to low: visible to the camera, empty, invisible to the camera.

[0036] As a preferred solution, the space occupancy voxelization method further includes:

[0037] Step S9: Calculate the velocity of each voxel;

[0038] Step S9 includes:

[0039] Step S9.1: Preset the velocity of all voxels to zero;

[0040] Step S9.2: for each tracking detection frame of a dynamic object, find all voxels whose center points are located in the tracking detection frame and whose voxel categories are the same as those of the tracking detection frame, and mark the voxels as frame voxels corresponding to the tracking detection frame;

[0041] Step S9.3: setting the reference speed of each frame voxel to the speed of the corresponding tracking detection frame;

[0042] Step S9.4: Calculate the displacement of the offset value of each box voxel relative to the central axis of the corresponding tracking detection box;

[0043] Step S9.5: Calculate the rotational angular velocity of each frame voxel relative to the central axis of the corresponding tracking detection frame based on the displacement of each frame voxel and the rotational angular velocity of the corresponding tracking detection frame;

[0044] Step S9.6: Combine the reference velocity and rotational angular velocity of each frame voxel to obtain the velocity of the voxel.

[0045] In a second aspect, the present invention further discloses a spatial occupancy voxelization device for point cloud data processing, comprising:

[0046] The acquisition module is used to collect point cloud data, images and the position and posture data of the vehicle;

[0047] The tracking detection module is used to track and detect dynamic objects based on the data collected by the acquisition module, and obtain the position, size, direction, speed and rotation angular velocity of the tracking detection frame of each dynamic object in each frame;

[0048] Point cloud segmentation module, used to segment point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud or a noise point;

[0049] The static point cloud frame stacking module is used to convert the static point cloud in the ego-vehicle coordinate system of each frame into the world coordinate system according to the ego-vehicle posture data in each frame in the entire time sequence;

[0050] The point cloud of the entire time series in the world coordinate system is superimposed to obtain a static point cloud scene of the time series superimposed frames in the world coordinate system;

[0051] The dynamic point cloud overlay module is used to match the tracking detection frame of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling in the tracking detection frame as the point cloud of the object, and record its position relative to the tracking detection frame, and overlay the point cloud falling in the tracking detection frame in the entire time sequence at the relative position to obtain a dynamic point cloud instance of the time sequence overlay frame of each dynamic object;

[0052] The dynamic and static point cloud fusion module is used to fuse the static point cloud scene of the time-series stacked frames in the world coordinate system obtained by the static point cloud stacking module with the dynamic point cloud instance of the time-series stacked frames of each dynamic object obtained by the dynamic point cloud stacking module to obtain a complete stacked point cloud scene in each frame;

[0053] The point cloud voxelization module is used to divide the space around the vehicle into several voxels based on the complete overlapping point cloud scene in each frame, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel;

[0054] The voxel visibility calculation module is used to calculate the voxel radar visibility and the voxel camera visibility for each frame of the voxelized scene.

[0055] As a preferred solution, the dynamic and static point cloud fusion module includes:

[0056] A conversion unit is used to convert the static point cloud scene of the time sequence stacked frames in the world coordinate system obtained by the static point cloud stacking module into the self-vehicle coordinate system for the posture data of the self-vehicle in any frame in the entire time sequence, so as to obtain the static point cloud scene in the self-vehicle coordinate system in the frame;

[0057] A fusion unit is used to place the dynamic point cloud instance of the time-series superimposed frames of each dynamic object obtained by the dynamic point cloud superimposed frame module in the static point cloud scene obtained by the conversion unit according to the position of the tracking detection frame in the frame, so as to merge and fuse the dynamic and static point clouds and obtain a complete superimposed point cloud scene under the frame;

[0058] The repeated execution unit is used to repeatedly execute the methods in the conversion unit and the fusion unit until a complete overlapping point cloud scene is obtained in each frame.

[0059] As a preferred solution, the point cloud voxelization module includes:

[0060] A voxel division unit is used to divide the space around the vehicle into a number of voxels according to a preset size based on the point cloud scene of the complete overlapping frames in each frame;

[0061] a voxel category determination unit, configured to regard each voxel as occupied if the number of point clouds of a single category in each voxel exceeds a set threshold, and set the category of the occupied voxel to the category to which the point cloud with the largest number of point clouds in the voxel belongs;

[0062] The voxel offset value determination unit is used to count the position of each point cloud of the same category in the voxel relative to the center point of the voxel, take the average, and use the average as the offset value of the voxel.

[0063] As a preferred solution, the point cloud voxelization module also includes:

[0064] The ground voxel filling unit is used to count the number of ground voxels in the neighborhood of all low-position blank voxels. If the number of neighboring ground voxels exceeds a set threshold, the blank voxel is filled with a ground voxel.

[0065] At the same time, the bias mean of the neighboring ground voxels is calculated as the bias value of the padded ground voxel.

[0066] As a preferred solution, the voxel visibility calculation module includes:

[0067] The radar visibility calculation unit is used to calculate the radar visibility of each voxel of the voxelized scene, specifically:

[0068] Take the position of the ego vehicle radar in each frame as the origin and the position of the offset value of each occupied voxel in the outermost layer of the preset size as the end point, shoot a ray, mark all voxels passed before the first occupied voxel passed by the ray as empty, mark the first occupied voxel passed by the ray as visible, and mark all voxels passed after the ray passes through the first occupied voxel as invisible;

[0069] When the same voxel is traversed by multiple rays, the priorities are as follows from high to low: radar visible, empty, radar invisible;

[0070] The camera visibility calculation unit is used to calculate the camera visibility of each voxel of the voxelized scene, specifically:

[0071] Taking the position of the ego vehicle camera in each frame as the origin, a ray is shot to each pixel of each picture in each frame, and the ray is back-projected to the ego vehicle coordinate system through the ego vehicle's posture data. All voxels in the preset space are pre-marked as invisible to the camera, all voxels passed before the first occupied voxel passed by the ray are marked as empty, and the first occupied voxel passed by the ray is marked as visible to the camera;

[0072] When the same voxel is traversed by multiple rays, their priorities are from high to low: visible to the camera, empty, invisible to the camera.

[0073] As a preferred solution, the space occupancy voxelization device further includes:

[0074] A voxel velocity calculation module, used to calculate the velocity of each voxel;

[0075] The voxel velocity calculation module includes:

[0076] A voxel velocity preset unit, used to preset the velocity of all voxels to zero;

[0077] A frame voxel search unit is used to search for all voxels whose center points are located in the tracking detection frame and whose voxel categories are the same as those of the tracking detection frame for each dynamic object tracking detection frame, and mark the voxels as frame voxels corresponding to the tracking detection frame;

[0078] a frame voxel reference speed determination unit, used to set the reference speed of each frame voxel to be the same as the speed of the corresponding tracking detection frame;

[0079] A frame voxel displacement calculation unit, used to calculate the displacement of the offset value of each frame voxel relative to the central axis of the corresponding tracking detection frame;

[0080] A frame voxel rotation angular velocity determination unit, used to calculate the rotation angular velocity of each frame voxel relative to the central axis of the corresponding tracking detection frame based on the displacement of each frame voxel and the rotation angular velocity of the corresponding tracking detection frame;

[0081] The frame voxel speed calculation unit is used to combine the reference speed and the rotation angular speed of each frame voxel to obtain the speed value of the voxel.

[0082] In a third aspect, the present invention further discloses a computing device, comprising:

[0083] one or more processors;

[0084] Memory;

[0085] and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-mentioned spatial occupancy voxelization methods for point cloud data processing.

[0086] In a fourth aspect, the present invention further discloses a storage medium storing one or more computer-readable programs, wherein the one or more programs include instructions suitable for being loaded by a memory and executing any one of the above-mentioned spatial occupancy voxelization methods for point cloud data processing.

[0087] The present invention discloses a method and device for spatial occupancy voxelization for point cloud data processing, which has the following beneficial effects:

[0088] First, the present invention can obtain information such as the category, bias value, radar visibility, camera visibility, speed, etc. of each voxel in three-dimensional space. The high-precision voxelized data provides a theoretical basis for autonomous driving and robotic systems.

[0089] Second, the present invention stacks sparse point cloud data into a dense scene by using a frame stacking method, thereby reducing the error during voxelization.

[0090] Third, the present invention calculates the visibility of each voxel through camera and radar visibility algorithms, which facilitates the subsequent application of voxelized data.

[0091] Fourthly, the present invention fills in the occasional gaps in the ground far away from the vehicle by using a ground voxel filling method.

[0092] Fifth, the present invention has important application value and broad application prospects in the fields of high-precision positioning and environmental perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0094] Figure 1 A flow chart of a spatial occupancy voxelization method provided in an embodiment of the present invention.

[0095] Figure 2 A simplified flowchart of a method for spatial occupancy voxelization provided in an embodiment of the present invention.

[0096] Figure 3 A schematic diagram of a space occupancy voxelization device provided in an embodiment of the present invention.

[0097] Figure 4 A schematic diagram of a computing device provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0098] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0099] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0100] The expression of “comprising” an element is an “open” expression, which merely means that a corresponding component or step exists, and should not be interpreted as excluding additional components or steps.

[0101] In order to have a clear understanding of the present invention, main terms are explained.

[0102] Spatial occupancy refers to labeling the state of each voxel in three-dimensional space into specific categories, unoccupied, unknown, etc.

[0103] In order to achieve the purpose of the present invention, some embodiments of the spatial occupancy voxelization method for point cloud data processing are as follows: Figure 1-2 As shown, the spatial occupancy voxelization methods include:

[0104] Step S101: collecting point cloud data, images and vehicle posture data;

[0105] Step S102: performing tracking detection of dynamic objects based on the data collected in step S101, and obtaining the position, size, orientation, speed and rotational angular velocity of the tracking detection frame of each dynamic object in each frame;

[0106] Step S103: segmenting the point cloud data to determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud or a noise point;

[0107] Step S104: according to the posture data of the ego vehicle in each frame in the entire time sequence, convert the static point cloud in the ego vehicle coordinate system of the frame into the world coordinate system;

[0108] The point cloud of the entire time series in the world coordinate system is superimposed to obtain a static point cloud scene of the time series superimposed frames in the world coordinate system;

[0109] Step S105: Match the tracking detection frame of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling in the tracking detection frame as the point cloud of the object, and record its position relative to the tracking detection frame, and overlap the point cloud falling in the tracking detection frame in the entire time sequence at the relative position to obtain a dynamic point cloud instance of the time sequence overlapped frame of each dynamic object;

[0110] Step S106: fusing the static point cloud scene of the time-series overlapping frames in the world coordinate system obtained in step S104 with the dynamic point cloud instance of the time-series overlapping frames of each dynamic object obtained in step S105 to obtain a complete overlapping point cloud scene in each frame;

[0111] Step S107: Based on the complete overlapping point cloud scene in each frame, the space around the vehicle is divided into a number of voxels, and the category and position of the point cloud appearing in each voxel are counted to obtain the category and offset value of each voxel;

[0112] Step S108: performing voxel radar visibility calculation and voxel camera visibility calculation on each voxelized scene frame.

[0113] Each step is described in detail below.

[0114] Step S101 is a collection step, and sensors such as radar and camera may be used to collect data.

[0115] Step S102 is a tracking detection step, which may adopt a BEV detection method or a radar detection method.

[0116] Specifically, for example, an offline BEV tracking detection model can be used to implement 3D detection and tracking of dynamic objects in the data, obtain the detection frame position, size, orientation, speed and rotation angular velocity of each dynamic object in each frame, and achieve matching between the same objects in alternate frames, and give each object a tracking ID. This detection model belongs to the prior art and will not be elaborated.

[0117] Step S103 is a point cloud segmentation processing step. A general point cloud segmentation model can be used to implement semantic segmentation of the point cloud data, obtain the category classification of each point cloud in each frame, and separate static category point clouds, dynamic category point clouds and noise. The point cloud segmentation model belongs to the prior art and will not be described in detail.

[0118] Step S104 is a static point cloud frame stacking step. The static point cloud obtained in step S103 in each frame of the entire time sequence is converted to the world coordinate system through the posture data of the vehicle, and the world coordinate system point cloud of the entire time sequence is superimposed to obtain a static point cloud scene of the time sequence frame stacking.

[0119] Step S105 is a dynamic point cloud instantiation step. The tracking detection frame obtained in step S102 and the dynamic point cloud obtained in step S103 are matched for each frame in the entire time sequence, and the point cloud falling in the frame is marked as the point cloud of the object, and the position relative to the frame is recorded. The point cloud falling in the frame in the entire time sequence is superimposed at the relative position to obtain a dynamic point cloud instance of each object in the time sequence.

[0120] Step S106 is a step of fusing dynamic and static point clouds, including:

[0121] Step S106.1: for the posture data of the ego vehicle in any frame in the entire time sequence, convert the static point cloud scene of the time sequence stacked frames in the world coordinate system obtained in step S104 to the ego vehicle coordinate system, and obtain the static point cloud scene in the ego vehicle coordinate system in the frame;

[0122] Step S106.2: placing the dynamic point cloud instance of the time-series stacked frames of each dynamic object obtained in step S105 in the static point cloud scene obtained in step S106.1 according to the position of the tracking detection frame in the frame, so as to merge and fuse the dynamic and static point clouds and obtain a complete stacked point cloud scene in the frame;

[0123] Step S106.3: Repeat steps S106.1 to S106.2 until a complete overlapping point cloud scene is obtained for each frame.

[0124] Step S107 is a point cloud voxelization step, including:

[0125] Step S107.1: Based on the complete overlapping point cloud scene in each frame, the space around the vehicle is divided into a number of voxels according to a fixed size;

[0126] Step S107.2: If the number of point clouds of a single category in each voxel exceeds a set threshold, the voxel is considered occupied, and the category of the occupied voxel is set to the category to which the point cloud with the largest number of point clouds in the voxel belongs;

[0127] Step S107.3: Count the positions of each point cloud of the same category in the occupied voxel relative to the center point of the voxel, take the average, and use the average as the bias value of the voxel.

[0128] Because the point cloud in the area far away from the vehicle is relatively sparse, ground voxels may disappear.

[0129] Therefore, step S107 also includes a ground voxel filling step, which specifically includes:

[0130] Step S107.4: for all low-position blank voxels, count the number of ground voxels in their neighborhoods. If the number of neighboring ground voxels exceeds a set threshold, fill the blank voxel with a ground voxel.

[0131] At the same time, the bias mean of the neighboring ground voxels is calculated as the bias value of the padded ground voxel.

[0132] Since the above voxels are obtained by voxelizing the time-series multi-frame point cloud, for each frame of the scene, some of the occupied voxels are actually invisible and need to be excluded through visibility calculation.

[0133] Step S108 is a voxel visibility calculation step, including:

[0134] Step S108.1: Calculate the radar visibility of each voxel of the voxelized scene, specifically:

[0135] Take the position of the ego vehicle radar in each frame as the origin and the position of the offset value of each occupied voxel in the outermost layer of the specified size as the end point, shoot a ray, mark all voxels passed before the first occupied voxel passed by the ray as empty, mark the first occupied voxel passed by the ray as visible, and mark all voxels passed after the ray passes through the first occupied voxel as invisible;

[0136] When the same voxel is traversed by multiple rays, the priorities are as follows from high to low: radar visible, empty, radar invisible;

[0137] Step S108.2: Calculate the camera visibility of each voxel of the voxelized scene, specifically:

[0138] Taking the position of the ego-car camera in each frame as the origin, a ray is shot to each pixel of each picture in each frame, and the ray is back-projected to the ego-car coordinate system through the ego-car's posture data. All voxels in the specified space are pre-marked as invisible to the camera, all voxels passed before the first occupied voxel passed by the ray are marked as empty, and the first occupied voxel passed by the ray is marked as visible to the camera;

[0139] When the same voxel is traversed by multiple rays, their priorities are from high to low: visible to the camera, empty, invisible to the camera.

[0140] In some other embodiments, the above-mentioned space occupancy voxelization method further includes:

[0141] Step S109: Calculate the velocity of each voxel;

[0142] Step S109 includes:

[0143] Step S109.1: Preset the velocity of all voxels to zero;

[0144] Step S109.2: for each tracking detection frame of a dynamic object, find all voxels whose center points are located in the tracking detection frame and whose voxel categories are the same as those of the tracking detection frame, and mark the voxels as frame voxels corresponding to the tracking detection frame;

[0145] Step S109.3: setting the reference speed of each frame voxel to the speed of the corresponding tracking detection frame;

[0146] Step S109.4: Calculate the displacement of the offset value of each frame voxel relative to the central axis of the corresponding tracking detection frame;

[0147] Specifically, the displacement refers to physical displacement, which is a vector value formed on a horizontal plane, with the central axis of the final tracking detection frame as the starting point and the point where the offset value of each voxel is located (i.e., the center of gravity of each voxel) as the end point;

[0148] Step S109.5: Calculate the rotational angular velocity of each frame voxel relative to the central axis of the corresponding tracking detection frame based on the displacement of each frame voxel and the rotational angular velocity of the corresponding tracking detection frame;

[0149] Specifically, the above method can use the yaw angle of each tracking detection frame to fit a curve in the time series, and take the derivative at each time point as the rotation angular velocity;

[0150] Step S109.6: Combine the reference speed and rotation angular speed of each frame voxel to obtain the speed value of the voxel.

[0151] In addition, in some other embodiments, Figure 3 As shown, the present invention also discloses a space occupancy voxelization device for point cloud data processing, comprising:

[0152] The acquisition module 201 is used to acquire point cloud data, images and the position and posture data of the vehicle;

[0153] The tracking detection module 202 is used to track and detect dynamic objects based on the data collected by the acquisition module, and obtain the position, size, direction, speed and rotation angular velocity of the tracking detection frame of each dynamic object in each frame;

[0154] The point cloud segmentation module 203 is used to segment the point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud or a noise point;

[0155] The static point cloud frame stacking module 204 is used to convert the static point cloud in the ego vehicle coordinate system of each frame into the world coordinate system according to the ego vehicle posture data in each frame in the entire time sequence;

[0156] The point cloud of the entire time series in the world coordinate system is superimposed to obtain a static point cloud scene of the time series superimposed frames in the world coordinate system;

[0157] The dynamic point cloud frame overlay module 205 is used to match the tracking detection frame of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling in the tracking detection frame as the object point cloud, and record its position relative to the tracking detection frame, and overlay the point cloud falling in the tracking detection frame in the entire time sequence at the relative position to obtain a dynamic point cloud instance of the time sequence overlay frame of each dynamic object;

[0158] The static and dynamic point cloud fusion module 206 is used to fuse the static point cloud scene of the time-series stacked frames in the world coordinate system obtained by the static point cloud stacking module with the dynamic point cloud instance of the time-series stacked frames of each dynamic object obtained by the dynamic point cloud stacking module to obtain a complete stacked point cloud scene in each frame;

[0159] The point cloud voxelization module 207 is used to divide the space around the vehicle into a number of voxels based on the complete overlapping frame point cloud scene in each frame, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel;

[0160] The voxel visibility calculation module 208 is used to perform voxel radar visibility calculation and voxel camera visibility calculation on each frame of the voxelized scene.

[0161] Furthermore, the dynamic and static point cloud fusion module includes:

[0162] A conversion unit is used to convert the static point cloud scene of the time sequence stacked frames in the world coordinate system obtained by the static point cloud stacking module into the self-vehicle coordinate system for the posture data of the self-vehicle in any frame in the entire time sequence, so as to obtain the static point cloud scene in the self-vehicle coordinate system in the frame;

[0163] A fusion unit is used to place the dynamic point cloud instance of the time-series superimposed frames of each dynamic object obtained by the dynamic point cloud superimposed frame module in the static point cloud scene obtained by the conversion unit according to the position of the tracking detection frame in the frame, so as to merge and fuse the dynamic and static point clouds and obtain a complete superimposed point cloud scene under the frame;

[0164] The repeated execution unit is used to repeatedly execute the methods in the conversion unit and the fusion unit until a complete overlapping point cloud scene is obtained in each frame.

[0165] Furthermore, the point cloud voxelization module includes:

[0166] A voxel division unit is used to divide the space around the vehicle into a number of voxels according to a preset size based on the point cloud scene of the complete overlapping frames in each frame;

[0167] a voxel category determination unit, configured to regard each voxel as occupied if the number of point clouds of a single category in each voxel exceeds a set threshold, and set the category of the occupied voxel to the category to which the point cloud with the largest number of point clouds in the voxel belongs;

[0168] The voxel offset value determination unit is used to count the position of each point cloud of the same category in the voxel relative to the center point of the voxel, take the average, and use the average as the offset value of the voxel.

[0169] Furthermore, the point cloud voxelization module also includes:

[0170] The ground voxel filling unit is used to count the number of ground voxels in the neighborhood of all low-position blank voxels. If the number of neighboring ground voxels exceeds a set threshold, the blank voxel is filled with a ground voxel.

[0171] At the same time, the bias mean of the neighboring ground voxels is calculated as the bias value of the padded ground voxel.

[0172] Furthermore, the voxel visibility calculation module includes:

[0173] The radar visibility calculation unit is used to calculate the radar visibility of each voxel of the voxelized scene, specifically:

[0174] Take the position of the ego vehicle radar in each frame as the origin and the position of the offset value of each occupied voxel in the outermost layer of the preset size as the end point, shoot a ray, mark all voxels passed before the first occupied voxel passed by the ray as empty, mark the first occupied voxel passed by the ray as visible, and mark all voxels passed after the ray passes through the first occupied voxel as invisible;

[0175] When the same voxel is traversed by multiple rays, the priorities are as follows from high to low: radar visible, empty, radar invisible;

[0176] The camera visibility calculation unit is used to calculate the camera visibility of each voxel of the voxelized scene, specifically:

[0177] Taking the position of the ego vehicle camera in each frame as the origin, a ray is shot to each pixel of each picture in each frame, and the ray is back-projected to the ego vehicle coordinate system through the ego vehicle's posture data. All voxels in the preset space are pre-marked as invisible to the camera, all voxels passed before the first occupied voxel passed by the ray are marked as empty, and the first occupied voxel passed by the ray is marked as visible to the camera;

[0178] When the same voxel is traversed by multiple rays, their priorities are from high to low: visible to the camera, empty, invisible to the camera.

[0179] Furthermore, the space occupancy voxelization device further comprises:

[0180] A voxel velocity calculation module 209, used to calculate the velocity of each voxel;

[0181] The voxel velocity calculation module includes:

[0182] A voxel velocity preset unit, used to preset the velocity of all voxels to zero;

[0183] A frame voxel search unit is used to search for all voxels whose center points are located in the tracking detection frame and whose voxel categories are the same as those of the tracking detection frame for each dynamic object tracking detection frame, and mark the voxels as frame voxels corresponding to the tracking detection frame;

[0184] a frame voxel reference speed determination unit, used to set the reference speed of each frame voxel to be the same as the speed of the corresponding tracking detection frame;

[0185] A frame voxel displacement calculation unit, used to calculate the displacement of the offset value of each frame voxel relative to the central axis of the corresponding tracking detection frame;

[0186] A frame voxel rotation angular velocity determination unit, used to calculate the rotation angular velocity of each frame voxel relative to the central axis of the corresponding tracking detection frame based on the displacement of each frame voxel and the rotation angular velocity of the corresponding tracking detection frame;

[0187] The frame voxel speed calculation unit is used to combine the reference speed and the rotation angular speed of each frame voxel to obtain the speed value of the voxel.

[0188] Furthermore, it should be noted that: the space occupancy voxelization device for point cloud data processing provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing voxelization processing. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the space occupancy voxelization device for point cloud data processing is divided into different functional modules to complete all or part of the functions described above.

[0189] In addition, the spatial occupancy voxelization device for point cloud data processing provided in the above embodiment and the spatial occupancy voxelization method for point cloud data processing belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0190] In addition, in some other embodiments, Figure 4 As shown, the present invention also discloses a computing device, including:

[0191] One or more processors 301;

[0192] Memory 302;

[0193] And one or more programs, wherein the one or more programs are stored in the memory 302 and configured to be executed by one or more processors 301, and the one or more programs include instructions for the spatial occupancy voxelization method for point cloud data processing disclosed in the above embodiments.

[0194] The processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 301 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0195] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 302 is used to store at least one instruction, which is used to be executed by the processor 301 to implement the spatial occupancy voxelization method for point cloud data processing provided by the method embodiment of the present invention.

[0196] In addition, the computing device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 301, the memory 302 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.

[0197] Of course, the computing device may also include fewer or more components, which is not limited in this embodiment.

[0198] In addition, in some other embodiments, the present invention also discloses a storage medium, which stores one or more computer-readable programs, and the one or more programs include instructions, and the instructions are suitable for being loaded by a memory and executing the spatial occupancy voxelization method for point cloud data processing disclosed in the above embodiments.

[0199] The present invention discloses a method and device for spatial occupancy voxelization for point cloud data processing, which has the following beneficial effects:

[0200] First, the present invention can obtain information such as the category, bias value, radar visibility, camera visibility, speed, etc. of each voxel in three-dimensional space. The high-precision voxelized data provides a theoretical basis for autonomous driving and robotic systems.

[0201] Second, the present invention stacks sparse point cloud data into a dense scene by using a frame stacking method, thereby reducing the error during voxelization.

[0202] Third, the present invention calculates the visibility of each voxel through camera and radar visibility algorithms, which facilitates the subsequent application of voxelized data.

[0203] Fourthly, the present invention fills in the occasional gaps in the ground far away from the vehicle by using a ground voxel filling method.

[0204] Fifth, the present invention has important application value and broad application prospects in the fields of high-precision positioning and environmental perception.

[0205] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A spatial occupancy voxelization method for point cloud data processing, characterized in that: include: Step S1: Collect point cloud data, images and vehicle posture data; Step S2: Based on the data collected in step S1, dynamic object tracking detection is performed to obtain the position, size, orientation, speed and rotation angular velocity of the tracking detection frame of each dynamic object in each frame; Step S3: segment the point cloud data to determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud or a noise point; Step S4: according to the posture data of the ego vehicle in each frame in the entire time sequence, convert the static point cloud in the ego vehicle coordinate system of the frame into the world coordinate system; The point cloud of the entire time series in the world coordinate system is superimposed to obtain a static point cloud scene of the time series superimposed frames in the world coordinate system; Step S5: Match the tracking detection frame of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling in the tracking detection frame as the object point cloud, and record its position relative to the tracking detection frame, and overlap the point cloud falling in the tracking detection frame in the entire time sequence at the relative position to obtain a dynamic point cloud instance of each dynamic object in the time sequence overlapped frame; Step S6: fusing the static point cloud scene of the time-series overlapping frames in the world coordinate system obtained in step S4 with the dynamic point cloud instance of the time-series overlapping frames of each dynamic object obtained in step S5 to obtain a complete overlapping point cloud scene in each frame; Step S7: Based on the complete overlapping point cloud scene in each frame, the space around the vehicle is divided into a number of voxels, and the category and position of the point cloud appearing in each voxel are counted to obtain the category and bias value of each voxel; Step S8: performing voxel radar visibility calculation and voxel camera visibility calculation on each voxelized scene.

2. The spatial occupancy voxelization method according to claim 1, characterized in that: The step S6 comprises: Step S6.1: for the posture data of the ego vehicle in any frame in the entire time sequence, convert the static point cloud scene of the time sequence stacked frames in the world coordinate system obtained in step S4 to the ego vehicle coordinate system to obtain the static point cloud scene in the ego vehicle coordinate system in the frame; Step S6.2: placing the dynamic point cloud instance of the time-series stacked frames of each dynamic object obtained in step S5 in the static point cloud scene obtained in step S6.1 according to the position of the tracking detection frame in the frame, so as to merge and fuse the dynamic and static point clouds and obtain a complete stacked point cloud scene in the frame; Step S6.3: Repeat steps S6.1 to S6.2 until a complete overlapping point cloud scene is obtained for each frame.

3. The spatial occupancy voxelization method according to claim 1, characterized in that: The step S7 comprises: Step S7.1: Based on the complete overlapping point cloud scene in each frame, the space around the vehicle is divided into a number of voxels according to a preset size; Step S7.2: if the number of point clouds of a single category in each voxel exceeds a set threshold, the voxel is considered occupied, and the category of the occupied voxel is set to the category to which the point cloud with the largest number of point clouds in the voxel belongs; Step S7.3: Count the positions of each point cloud of the same category in the occupied voxel relative to the center point of the voxel, take the average, and use the average as the bias value of the voxel.

4. The spatial occupancy voxelization method according to claim 3, characterized in that: The step S7 further comprises: Step S7.4: for all low-position blank voxels, count the number of ground voxels in their neighborhoods. If the number of neighboring ground voxels exceeds a set threshold, fill the blank voxel with a ground voxel. At the same time, the bias mean of the neighboring ground voxels is calculated as the bias value of the padded ground voxel.

5. The spatial occupancy voxelization method according to claim 1, wherein step S8 comprises: Step S8.1: Calculate the radar visibility of each voxel of the voxelized scene, specifically: Take the position of the ego vehicle radar in each frame as the origin and the position of the offset value of each occupied voxel in the outermost layer of the preset size as the end point, shoot a ray, mark all voxels passed before the first occupied voxel passed by the ray as empty, mark the first occupied voxel passed by the ray as visible, and mark all voxels passed after the ray passes through the first occupied voxel as invisible; When the same voxel is traversed by multiple rays, the priorities are as follows from high to low: radar visible, empty, radar invisible; Step S8.2: Calculate the camera visibility of each voxel in the voxelized scene, specifically: Taking the position of the ego vehicle camera in each frame as the origin, a ray is shot to each pixel of each picture in each frame, and the ray is back-projected to the ego vehicle coordinate system through the ego vehicle's posture data. All voxels in the preset space are pre-marked as invisible to the camera, all voxels passed before the first occupied voxel passed by the ray are marked as empty, and the first occupied voxel passed by the ray is marked as visible to the camera; When the same voxel is traversed by multiple rays, their priorities are from high to low: visible to the camera, empty, invisible to the camera.

6. The space occupancy voxelization method according to claim 1, further comprising: Step S9: Calculate the velocity of each voxel; The step S9 comprises: Step S9.1: Preset the velocity of all voxels to zero; Step S9.2: for each tracking detection frame of a dynamic object, find all voxels whose center points are located in the tracking detection frame and whose voxel categories are the same as those of the tracking detection frame, and mark the voxels as frame voxels corresponding to the tracking detection frame; Step S9.3: setting the reference speed of each frame voxel to be the same as the speed of the corresponding tracking detection frame; Step S9.4: Calculate the displacement of the offset value of each box voxel relative to the central axis of the corresponding tracking detection box; Step S9.5: Calculate the rotational angular velocity of each frame voxel relative to the central axis of the corresponding tracking detection frame based on the displacement of each frame voxel and the rotational angular velocity of the corresponding tracking detection frame; Step S9.6: Combine the reference velocity and rotational angular velocity of each frame voxel to obtain the velocity of the voxel.

7. A spatial occupancy voxelization device for point cloud data processing, characterized in that: include: The acquisition module is used to collect point cloud data, images and the position and posture data of the vehicle; The tracking detection module is used to track and detect dynamic objects based on the data collected by the acquisition module, and obtain the position, size, direction, speed and rotation angular velocity of the tracking detection frame of each dynamic object in each frame; Point cloud segmentation module, used to segment point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud or a noise point; The static point cloud frame stacking module is used to convert the static point cloud in the ego-vehicle coordinate system of each frame into the world coordinate system according to the ego-vehicle posture data in each frame in the entire time sequence; The point cloud of the entire time series in the world coordinate system is superimposed to obtain a static point cloud scene of the time series superimposed frames in the world coordinate system; The dynamic point cloud overlay module is used to match the tracking detection frame of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling in the tracking detection frame as the point cloud of the object, and record its position relative to the tracking detection frame, and overlay the point cloud falling in the tracking detection frame in the entire time sequence at the relative position to obtain a dynamic point cloud instance of the time sequence overlay frame of each dynamic object; The dynamic and static point cloud fusion module is used to fuse the static point cloud scene of the time-series stacked frames in the world coordinate system obtained by the static point cloud stacking module with the dynamic point cloud instance of the time-series stacked frames of each dynamic object obtained by the dynamic point cloud stacking module to obtain a complete stacked point cloud scene in each frame; The point cloud voxelization module is used to divide the space around the vehicle into several voxels based on the complete overlapping point cloud scene in each frame, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel; The voxel visibility calculation module is used to calculate the voxel radar visibility and the voxel camera visibility for each frame of the voxelized scene.

8. The space occupancy voxelization device according to claim 7, characterized in that: The dynamic and static point cloud fusion module includes: A conversion unit is used to convert the static point cloud scene of the time sequence stacked frames in the world coordinate system obtained by the static point cloud stacking module into the self-vehicle coordinate system for the posture data of the self-vehicle in any frame in the entire time sequence, so as to obtain the static point cloud scene in the self-vehicle coordinate system in the frame; A fusion unit is used to place the dynamic point cloud instance of the time-series superimposed frames of each dynamic object obtained by the dynamic point cloud superimposed frame module in the static point cloud scene obtained by the conversion unit according to the position of the tracking detection frame in the frame, so as to merge and fuse the dynamic and static point clouds and obtain a complete superimposed point cloud scene under the frame; The repeated execution unit is used to repeatedly execute the methods in the conversion unit and the fusion unit until a complete overlapping point cloud scene is obtained in each frame.

9. A computing device, characterized in that include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and one or more of the programs include instructions for the spatial occupancy voxelization method for point cloud data processing as described in any of claims 1-6 above.

10. A storage medium, characterized in that The storage medium stores one or more computer-readable programs, and the one or more programs include instructions, which are suitable for being loaded by the memory and executing the spatial occupancy voxelization method for point cloud data processing described in any one of claims 1-6.

Images