Spatial occupancy voxelization method and apparatus for point cloud data processing

Abstract

The application discloses a space occupancy voxelization method and device for point cloud data processing, and comprises data acquisition, tracking detection of dynamic objects, point cloud segmentation processing, static point cloud frame superposition, dynamic point cloud frame superposition, dynamic and static point cloud fusion, point cloud voxelization and voxel visibility calculation. The application can obtain the information of the category, bias value, radar visibility, camera visibility and speed of each voxel in a three-dimensional space, and the high-precision voxelization data provides a theoretical basis for automatic driving and a robot system.

Description

Technical Field

[0001] This invention belongs to the field of point cloud data processing, and specifically relates to a method and apparatus for spatial occupancy voxelization in point cloud data processing. Background Technology

[0002] LiDAR technology, as an advanced sensing technology, is widely used in fields requiring high-precision positioning and environmental perception, such as autonomous driving, robot navigation, and drone flight control. LiDAR obtains point cloud data of the surrounding environment in real time by emitting laser pulses and receiving their reflected signals. This point cloud data contains precise position and shape information of objects in three-dimensional space, forming the foundation for building detailed environmental models and achieving high-precision perception.

[0003] In LiDAR data processing, voxelization refers to the process of converting discrete point cloud data into continuous spatially occupied data. A voxel is the smallest unit in three-dimensional space, similar to a pixel in a two-dimensional image. The voxelization process divides space into fixed-size units and records the point cloud data within each voxel. In this way, the original point cloud is transformed into a regular three-dimensional network representation, making data storage and processing more efficient and intuitive, and more suitable for real-time environmental perception and path planning tasks, especially playing an important role in the fields of autonomous driving and robot navigation.

[0004] However, there is currently no method for spatial occupancy voxelization in point cloud data processing. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a spatial occupancy voxelization method and apparatus for point cloud data processing.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention discloses a spatial occupancy voxelization method for point cloud data processing, comprising:

[0008] Step S1: Collect point cloud data, images, and the vehicle's pose data;

[0009] Step S2: Based on the data collected in step S1, perform dynamic object tracking and detection to obtain the position, size, orientation, velocity, and rotational angular velocity of the tracking and detection box of each dynamic object in each frame;

[0010] Step S3: Segment the point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud, or noise.

[0011] Step S4: Based on the vehicle's pose data in each frame of the entire time series, transform the static point cloud in the vehicle coordinate system of that frame to the world coordinate system.

[0012] By overlaying the point clouds in the world coordinate system for the entire time sequence, a static point cloud scene with time-stacked frames in the world coordinate system is obtained.

[0013] Step S5: Match the tracking detection box of the dynamic object in each frame of the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud that falls within the tracking detection box as the object point cloud, and record its position relative to the tracking detection box. Stack the point clouds that fall within the tracking detection box in the entire time sequence with the relative position to obtain the time-stacked dynamic point cloud instance of each dynamic object.

[0014] Step S6: Perform dynamic and static point cloud fusion between the static point cloud scene of the time-stacked frame in the world coordinate system obtained in step S4 and the dynamic point cloud instance of the time-stacked frame of each dynamic object obtained in step S5 to obtain the complete stacked frame point cloud scene in each frame.

[0015] Step S7: Based on the point cloud scene of the complete stacked frames in each frame, divide the space around the vehicle into several voxels, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel.

[0016] Step S8: Calculate the radar visibility and camera visibility of voxels for each voxelized scene frame.

[0017] Based on the above technical solution, the following improvements can be made:

[0018] As a preferred embodiment, step S6 includes:

[0019] Step S6.1: For the pose data of the vehicle in any frame of the entire time series, transform the static point cloud scene of the time series stacked frames in the world coordinate system obtained in step S4 to the vehicle coordinate system to obtain the static point cloud scene in the vehicle coordinate system of that frame.

[0020] Step S6.2: Place the dynamic point cloud instance of each dynamic object in the time-series overlay frame obtained in step S5 into the static point cloud scene obtained in step S6.1 according to the position of the tracking detection box in that frame, so as to realize the merging and fusion of dynamic and static point clouds and obtain the complete overlay frame point cloud scene under that frame.

[0021] Step S6.3: Repeat steps S6.1-S6.2 until the point cloud scene of the complete stacked frames under each frame is obtained.

[0022] As a preferred embodiment, step S7 includes:

[0023] Step S7.1: Based on the point cloud scene of the complete overlay frame in each frame, divide the space around the vehicle into several voxels according to the preset size;

[0024] Step S7.2: If the number of point clouds of a single category in each voxel exceeds a set threshold, then the voxel is considered occupied, and the category of the occupied voxel is set to the category of the point cloud with the most point clouds in that voxel.

[0025] Step S7.3: Calculate the position of each point cloud of the same category relative to the center point of the voxel, take the average value, and use the average value as the bias value of the voxel.

[0026] As a preferred embodiment, step S7 further includes:

[0027] Step S7.4: For all blank voxels at low positions, count the number of ground voxels in their neighborhood. If the number of ground voxels in the neighborhood exceeds a set threshold, fill the blank voxel with ground voxels.

[0028] At the same time, the mean bias of the neighboring ground voxels is used as the bias value for the completed ground voxel.

[0029] As a preferred embodiment, step S8 includes:

[0030] Step S8.1: Calculate the radar visibility of each voxel in the voxelized scene, specifically as follows:

[0031] Taking the location of the vehicle radar in each frame as the origin and the location of the offset value of each occupied voxel of the preset size as the endpoint, a ray is emitted. All voxels that the ray passes through before the first occupied voxel are marked as empty, the first occupied voxel that the ray passes through is marked as visible, and all voxels that the ray passes through after the first occupied voxel are marked as invisible.

[0032] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: radar visible, air, radar invisible;

[0033] Step S8.2: Perform camera visibility calculation for each voxelized scene frame, specifically:

[0034] Using the location of the vehicle camera in each frame as the origin, a ray is projected to each pixel of each image in each frame. The ray is back-projected into the vehicle coordinate system using the vehicle's pose data. All voxels in the preset space are pre-marked as invisible to the camera. All voxels that the ray passes through before the first occupying voxel are marked as empty. The first occupying voxel that the ray passes through is marked as visible to the camera.

[0035] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: visible to the camera, empty, invisible to the camera.

[0036] As a preferred approach, the space occupancy voxelization method also includes:

[0037] Step S9: Calculate the velocity of each voxel;

[0038] Step S9 includes:

[0039] Step S9.1: Set the velocity of all voxels to zero;

[0040] Step S9.2: For each tracking detection box of a dynamic object, find all voxels whose center point is located in the tracking detection box and whose voxel category is the same as that of the tracking detection box, and mark the voxel as the box voxel of the corresponding tracking detection box;

[0041] Step S9.3: Set the reference velocity of each frame voxel to the velocity of the corresponding tracking detection frame;

[0042] Step S9.4: Calculate the offset value of each frame voxel and the displacement relative to the central axis of the corresponding tracking detection frame;

[0043] Step S9.5: Calculate the rotational angular velocity of each frame element relative to the central axis of the corresponding tracking and detection frame based on the displacement of each frame element and the rotational angular velocity of the corresponding tracking and detection frame.

[0044] Step S9.6: Combine the reference velocity and rotational angular velocity of each frame voxel to obtain the velocity quantity of that voxel.

[0045] Secondly, the present invention also discloses a spatial occupancy voxelization device for point cloud data processing, comprising:

[0046] The acquisition module is used to acquire point cloud data, images, and the vehicle's pose data.

[0047] The tracking and detection module is used to track and detect dynamic objects based on the data collected by the acquisition module, and to obtain the position, size, orientation, velocity and rotational angular velocity of the tracking and detection box of each dynamic object in each frame.

[0048] The point cloud segmentation module is used to segment point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud, or noise.

[0049] The static point cloud overlay module is used to transform the static point cloud in the vehicle coordinate system to the world coordinate system according to the vehicle's pose data in each frame of the entire time series.

[0050] By overlaying the point clouds in the world coordinate system for the entire time sequence, a static point cloud scene with time-stacked frames in the world coordinate system is obtained.

[0051] The dynamic point cloud overlay module is used to match the tracking detection box of the dynamic object in each frame of the entire time series with the dynamic point cloud in the corresponding frame, mark the point cloud that falls within the tracking detection box as the object point cloud, record its position relative to the tracking detection box, and overlay the point clouds that fall within the tracking detection box in the entire time series with the relative position to obtain the dynamic point cloud instance of the time series overlay.

[0052] The static and dynamic point cloud fusion module is used to fuse the static point cloud scene in the world coordinate system obtained by the static point cloud overlay module with the dynamic point cloud instance of each dynamic object in the time-series overlay module to obtain the complete overlay point cloud scene in each frame.

[0053] The point cloud voxelization module is used to divide the space around the vehicle into several voxels based on the point cloud scene of the complete stacked frames in each frame, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel.

[0054] The voxel visibility calculation module is used to calculate the radar visibility of voxels and the camera visibility of voxels for each voxelized scene frame.

[0055] As a preferred solution, the dynamic and static point cloud fusion module includes:

[0056] The conversion unit is used to convert the static point cloud scene of the time-series stacked frames in the world coordinate system obtained by the static point cloud stacking module to the vehicle coordinate system for the pose data of the vehicle in any frame of the entire time series, so as to obtain the static point cloud scene in the vehicle coordinate system of that frame.

[0057] The fusion unit is used to place the dynamic point cloud instance of each dynamic object in the temporal frame of the dynamic point cloud stacking module into the static point cloud scene obtained by the conversion unit according to the position of the tracking detection box in the frame, so as to realize the merging and fusion of dynamic and static point clouds and obtain the complete stacked frame point cloud scene under the frame.

[0058] The repeat execution unit is used to repeatedly execute the methods in the transformation unit and the fusion unit until the complete stacked point cloud scene of each frame is obtained.

[0059] As a preferred solution, the point cloud voxelization module includes:

[0060] The voxel division unit is used to divide the space around the vehicle into several voxels according to a preset size based on the point cloud scene of the complete stacked frames in each frame.

[0061] The voxel category determination unit is used to consider a voxel as occupied if the number of point clouds of a single category in each voxel exceeds a set threshold, and to set the category of the occupied voxel to the category of the point cloud with the most point clouds in that voxel.

[0062] The voxel bias value determination unit is used to count the position of each point cloud of the same category relative to the voxel center point, take the average value, and use the average value as the bias value of the voxel.

[0063] As a preferred solution, the point cloud voxelization module also includes:

[0064] The ground voxel completion unit is used to count the number of ground voxels in the neighborhood of all low-position blank voxels. If the number of ground voxels in the neighborhood exceeds a set threshold, the blank voxel is filled into a ground voxel.

[0065] At the same time, the mean bias of the neighboring ground voxels is used as the bias value for the completed ground voxel.

[0066] As a preferred embodiment, the voxel visibility calculation module includes:

[0067] The radar visibility calculation unit is used to calculate the radar visibility of each voxel in each voxelized scene frame, specifically:

[0068] Taking the location of the vehicle radar in each frame as the origin and the location of the offset value of each occupied voxel of the preset size as the endpoint, a ray is emitted. All voxels that the ray passes through before the first occupied voxel are marked as empty, the first occupied voxel that the ray passes through is marked as visible, and all voxels that the ray passes through after the first occupied voxel are marked as invisible.

[0069] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: radar visible, air, radar invisible;

[0070] The camera visibility calculation unit is used to calculate the camera visibility of voxels for each voxelized scene frame, specifically:

[0071] Using the location of the vehicle camera in each frame as the origin, a ray is projected to each pixel of each image in each frame. The ray is back-projected into the vehicle coordinate system using the vehicle's pose data. All voxels in the preset space are pre-marked as invisible to the camera. All voxels that the ray passes through before the first occupying voxel are marked as empty. The first occupying voxel that the ray passes through is marked as visible to the camera.

[0072] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: visible to the camera, empty, invisible to the camera.

[0073] As a preferred embodiment, the space occupancy voxelization device also includes:

[0074] The voxel velocity calculation module is used to calculate the velocity of each voxel;

[0075] The voxel velocity calculation module includes:

[0076] The voxel velocity preset unit is used to preset the velocity of all voxels to zero.

[0077] The frame voxel finding unit is used to find all voxels whose center point is located in the tracking detection frame for each dynamic object and whose voxel category is the same as that of the tracking detection frame, and to mark the voxel as the frame voxel of the corresponding tracking detection frame.

[0078] The frame element reference velocity determination unit is used to set the reference velocity of each frame element to be the same as the velocity of the corresponding tracking detection frame;

[0079] The frame element displacement calculation unit is used to calculate the offset value of each frame element relative to the displacement of the corresponding tracking detection frame center axis;

[0080] The frame element rotation angular velocity determination unit is used to calculate the rotation angular velocity of each frame element relative to the central axis of the corresponding tracking and detection frame based on the displacement of each frame element and the rotation angular velocity of the corresponding tracking and detection frame.

[0081] The frame element velocity calculation unit is used to combine the reference velocity and rotational angular velocity of each frame element to obtain the velocity quantity of that frame element.

[0082] Thirdly, the present invention also discloses a computing device, comprising:

[0083] One or more processors;

[0084] Memory;

[0085] And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-described spatial occupancy voxelization methods for point cloud data processing.

[0086] Fourthly, the present invention also discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by a memory and executed any of the above-described spatial occupancy voxelization methods for point cloud data processing.

[0087] This invention discloses a method and apparatus for spatial occupancy voxelization in point cloud data processing, which has the following beneficial effects:

[0088] First, this invention can obtain information such as the category, bias value, radar visibility, camera visibility, and velocity of each voxel in three-dimensional space. The high-precision voxelized data provides a theoretical basis for autonomous driving and robotic systems.

[0089] Second, the present invention stacks sparse point cloud data into a dense scene by using a frame stacking method, thereby reducing errors during voxelization.

[0090] Third, the present invention calculates the visibility of each voxel using camera and radar visibility algorithms, which facilitates the subsequent application of voxelized data.

[0091] Fourth, the present invention supplements occasional gaps in the ground that are far from the vehicle by using a ground voxel completion method.

[0092] Fifth, this invention has significant application value and broad application prospects in the fields of high-precision positioning and environmental perception. Attached Figure Description

[0093] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0094] Figure 1 A flowchart of a space occupancy voxelization method provided in an embodiment of the present invention.

[0095] Figure 2 A simplified flowchart of the space occupancy voxelization method provided in an embodiment of the present invention.

[0096] Figure 3 This is a schematic diagram of a space occupancy voxelization device provided in an embodiment of the present invention.

[0097] Figure 4 This is a schematic diagram of a computing device provided in an embodiment of the present invention. Detailed Implementation

[0098] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0099] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0100] The expression “includes” is an “open-ended” expression, which means that there is a corresponding component or step, and should not be interpreted as excluding additional components or steps.

[0101] To provide a clear understanding of this invention, the main terms are explained.

[0102] Spatial occupancy refers to marking the state of each voxel in three-dimensional space, which is divided into specific categories, unoccupied, unknown, etc.

[0103] To achieve the objectives of this invention, some embodiments of the spatial occupancy voxelization method for point cloud data processing, such as Figure 1-2 As shown, the space occupancy voxelization method includes:

[0104] Step S101: Collect point cloud data, images, and the vehicle's pose data;

[0105] Step S102: Based on the data collected in step S101, perform dynamic object tracking and detection to obtain the position, size, orientation, velocity, and rotational angular velocity of the tracking and detection box of each dynamic object in each frame;

[0106] Step S103: Perform segmentation processing on the point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud, or noise.

[0107] Step S104: Based on the vehicle's pose data in each frame of the entire time series, transform the static point cloud in the vehicle coordinate system of that frame to the world coordinate system.

[0108] By overlaying the point clouds in the world coordinate system for the entire time sequence, a static point cloud scene with time-stacked frames in the world coordinate system is obtained.

[0109] Step S105: Match the tracking detection box of the dynamic object in each frame of the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud that falls within the tracking detection box as the object point cloud, and record its position relative to the tracking detection box. Stack the point clouds that fall within the tracking detection box in the entire time sequence with the relative position to obtain the time-stacked dynamic point cloud instance of each dynamic object.

[0110] Step S106: Perform dynamic and static point cloud fusion between the static point cloud scene of the time-stacked frame in the world coordinate system obtained in step S104 and the dynamic point cloud instance of the time-stacked frame of each dynamic object obtained in step S105, to obtain the complete stacked frame point cloud scene in each frame.

[0111] Step S107: Based on the point cloud scene of the complete overlay frame in each frame, divide the space around the vehicle into several voxels, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel.

[0112] Step S108: Perform radar visibility calculation and camera visibility calculation for each voxelized scene frame.

[0113] Each step will be explained in detail below.

[0114] Step S101 is the data acquisition step, which can use sensors such as radar and cameras to acquire data.

[0115] Step S102 is the tracking and detection step, which can be carried out using BEV detection method or radar detection method, etc.

[0116] Specifically, for example, an offline BEV tracking and detection model can be used to perform 3D detection and tracking of dynamic objects in the data, obtaining the detection box position, size, orientation, velocity, and rotational angular velocity of each dynamic object in each frame, while simultaneously matching the same objects across frames and assigning a tracking ID to each object. This detection model is existing technology and will not be elaborated upon.

[0117] Step S103 is the point cloud segmentation process. A general point cloud segmentation model can be used to perform semantic segmentation on the point cloud data, obtain the category classification of each point cloud in each frame, and separate static category point clouds, dynamic category point clouds, and noise. This point cloud segmentation model is existing technology and will not be described in detail.

[0118] Step S104 is the static point cloud overlay step. The static point cloud obtained in step S103 for each frame in the entire time series is transformed into the world coordinate system through the vehicle's pose data, and then the world coordinate system point cloud of the entire time series is overlaid to obtain the static point cloud scene of the time series overlay.

[0119] Step S105 is the dynamic point cloud instantiation step. The tracking detection box obtained in step S102 and the dynamic point cloud obtained in step S103 are matched in each frame of the entire time series. The point cloud falling in the box is marked as the object point cloud, and the position relative to the box is recorded. The point clouds falling in the box in the entire time series are overlaid with the relative position to obtain the time-stacked dynamic point cloud instance of each object.

[0120] Step S106 is the dynamic and static point cloud fusion step, which includes:

[0121] Step S106.1: For the pose data of the vehicle in any frame of the entire time series, transform the static point cloud scene of the time series stacked frames in the world coordinate system obtained in step S104 to the vehicle coordinate system to obtain the static point cloud scene in the vehicle coordinate system of that frame.

[0122] Step S106.2: Place the dynamic point cloud instance of each dynamic object in the time-series overlay frame obtained in step S105 into the static point cloud scene obtained in step S106.1 according to the position of the tracking detection box in that frame, so as to realize the merging and fusion of dynamic and static point clouds and obtain the complete overlay frame point cloud scene under that frame.

[0123] Step S106.3: Repeat steps S106.1-S106.2 until the point cloud scene of the complete stacked frames under each frame is obtained.

[0124] Step S107 is the point cloud voxelization step, which includes:

[0125] Step S107.1: Based on the point cloud scene of the complete stacked frames in each frame, divide the space around the vehicle into several voxels according to a fixed size;

[0126] Step S107.2: If the number of point clouds of a single category in each voxel exceeds a set threshold, then the voxel is considered occupied, and the category of the occupied voxel is set to the category of the point cloud with the most point clouds in that voxel.

[0127] Step S107.3: Calculate the position of each point cloud of the same category relative to the center point of the voxel, take the average value, and use the average value as the bias value of the voxel.

[0128] Because point clouds are sparse in areas far from the vehicle, ground voxels may disappear.

[0129] Therefore, step S107 also includes a ground voxel completion step, specifically including:

[0130] Step S107.4: For all blank voxels at low positions, count the number of ground voxels in their neighborhood. If the number of ground voxels in the neighborhood exceeds a set threshold, fill the blank voxel with ground voxels.

[0131] At the same time, the mean bias of the neighboring ground voxels is used as the bias value for the completed ground voxel.

[0132] Since the above voxels are obtained by stacking point clouds from multiple frames in a temporal sequence, some of the voxels occupying each frame of the scene are actually invisible and need to be excluded through visibility calculation.

[0133] Step S108 is the voxel visibility calculation step, which includes:

[0134] Step S108.1: Perform radar visibility calculation for each voxelized scene frame, specifically as follows:

[0135] Taking the location of the vehicle radar in each frame as the origin and the location of the offset value of each occupied voxel of the outermost layer of the specified size as the endpoint, a ray is emitted. All voxels that the ray passes through before the first occupied voxel are marked as empty, the first occupied voxel that the ray passes through is marked as visible, and all voxels that the ray passes through after the first occupied voxel are marked as invisible.

[0136] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: radar visible, air, radar invisible;

[0137] Step S108.2: Perform camera visibility calculation for each voxelized scene frame, specifically:

[0138] Using the location of the vehicle camera in each frame as the origin, a ray is projected to each pixel of each image in each frame. The ray is back-projected into the vehicle coordinate system using the vehicle's pose data. All voxels in the specified space are pre-marked as invisible to the camera. All voxels that the ray passes through before the first occupying voxel are marked as empty. The first occupying voxel that the ray passes through is marked as visible to the camera.

[0139] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: visible to the camera, empty, invisible to the camera.

[0140] In other embodiments, the above-described space occupancy voxelization method further includes:

[0141] Step S109: Calculate the velocity of each voxel;

[0142] Step S109 includes:

[0143] Step S109.1: Set the velocity of all voxels to zero;

[0144] Step S109.2: For each tracking detection box of a dynamic object, find all voxels whose center point is located in the tracking detection box and whose voxel category is the same as that of the tracking detection box, and mark the voxel as the box voxel of the corresponding tracking detection box;

[0145] Step S109.3: Set the reference velocity of each frame voxel to the velocity of the corresponding tracking detection frame;

[0146] Step S109.4: Calculate the offset value of each frame voxel and the displacement relative to the central axis of the corresponding tracking detection frame;

[0147] Specifically, this displacement refers to physical displacement, which in this case is a vector value on a horizontal plane, starting from the central axis of the final tracking detection frame and connecting the points where the offset values ​​of each voxel are located (i.e., the centroids of each voxel).

[0148] Step S109.5: Calculate the rotational angular velocity of each frame element relative to the central axis of the corresponding tracking and detection frame based on the displacement of each frame element and the rotational angular velocity of the corresponding tracking and detection frame.

[0149] Specifically, the yaw angle of each tracking detection box can be used to fit a curve in time series, and the derivative at each time point can be taken as the rotational angular velocity.

[0150] Step S109.6: Combine the reference velocity and rotational angular velocity of each frame voxel to obtain the velocity quantity of that voxel.

[0151] Furthermore, in some other embodiments, such as Figure 3 As shown, the present invention also discloses a spatial occupancy voxelization device for point cloud data processing, comprising:

[0152] The acquisition module 201 is used to acquire point cloud data, images, and the vehicle's pose data;

[0153] The tracking and detection module 202 is used to track and detect dynamic objects based on the data collected by the acquisition module, and to obtain the position, size, orientation, speed and rotational angular velocity of the tracking and detection box of each dynamic object in each frame.

[0154] The point cloud segmentation module 203 is used to segment point cloud data and determine whether each point cloud in each frame is a static point cloud, a dynamic point cloud, or noise.

[0155] The static point cloud overlay module 204 is used to transform the static point cloud in the vehicle coordinate system to the world coordinate system according to the vehicle's pose data in each frame of the entire time series.

[0156] By overlaying the point clouds in the world coordinate system for the entire time sequence, a static point cloud scene with time-stacked frames in the world coordinate system is obtained.

[0157] The dynamic point cloud overlay module 205 is used to match the tracking detection box of the dynamic object in each frame of the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud that falls within the tracking detection box as the object point cloud, record its position relative to the tracking detection box, and overlay the point clouds that fall within the tracking detection box in the entire time sequence with the relative position to obtain the dynamic point cloud instance of the time sequence overlay.

[0158] The static and dynamic point cloud fusion module 206 is used to fuse the static point cloud scene of time-series stacked frames in the world coordinate system obtained by the static point cloud stacking module with the dynamic point cloud instance of time-series stacked frames of each dynamic object obtained by the dynamic point cloud stacking module to obtain the complete stacked frame point cloud scene in each frame.

[0159] The point cloud voxelization module 207 is used to divide the space around the vehicle into several voxels based on the point cloud scene of the complete stacked frames in each frame, count the category and position of the point cloud appearing in each voxel, and obtain the category and bias value of each voxel.

[0160] The voxel visibility calculation module 208 is used to perform radar visibility calculation and camera visibility calculation of voxels for each voxelized scene frame.

[0161] Furthermore, the dynamic and static point cloud fusion module includes:

[0162] The conversion unit is used to convert the static point cloud scene of the time-series stacked frames in the world coordinate system obtained by the static point cloud stacking module to the vehicle coordinate system for the pose data of the vehicle in any frame of the entire time series, so as to obtain the static point cloud scene in the vehicle coordinate system of that frame.

[0163] The fusion unit is used to place the dynamic point cloud instance of each dynamic object in the temporal frame of the dynamic point cloud stacking module into the static point cloud scene obtained by the conversion unit according to the position of the tracking detection box in the frame, so as to realize the merging and fusion of dynamic and static point clouds and obtain the complete stacked frame point cloud scene under the frame.

[0164] The repeat execution unit is used to repeatedly execute the methods in the transformation unit and the fusion unit until the complete stacked point cloud scene of each frame is obtained.

[0165] Furthermore, the point cloud voxelization module includes:

[0166] The voxel division unit is used to divide the space around the vehicle into several voxels according to a preset size based on the point cloud scene of the complete stacked frames in each frame.

[0167] The voxel category determination unit is used to consider a voxel as occupied if the number of point clouds of a single category in each voxel exceeds a set threshold, and to set the category of the occupied voxel to the category of the point cloud with the most point clouds in that voxel.

[0168] The voxel bias value determination unit is used to count the position of each point cloud of the same category relative to the voxel center point, take the average value, and use the average value as the bias value of the voxel.

[0169] Furthermore, the point cloud voxelization module also includes:

[0170] The ground voxel completion unit is used to count the number of ground voxels in the neighborhood of all low-position blank voxels. If the number of ground voxels in the neighborhood exceeds a set threshold, the blank voxel is filled into a ground voxel.

[0171] At the same time, the mean bias of the neighboring ground voxels is used as the bias value for the completed ground voxel.

[0172] Furthermore, the voxel visibility calculation module includes:

[0173] The radar visibility calculation unit is used to calculate the radar visibility of each voxel in each voxelized scene frame, specifically:

[0174] Taking the location of the vehicle radar in each frame as the origin and the location of the offset value of each occupied voxel of the preset size as the endpoint, a ray is emitted. All voxels that the ray passes through before the first occupied voxel are marked as empty, the first occupied voxel that the ray passes through is marked as visible, and all voxels that the ray passes through after the first occupied voxel are marked as invisible.

[0175] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: radar visible, air, radar invisible;

[0176] The camera visibility calculation unit is used to calculate the camera visibility of voxels for each voxelized scene frame, specifically:

[0177] Using the location of the vehicle camera in each frame as the origin, a ray is projected to each pixel of each image in each frame. The ray is back-projected into the vehicle coordinate system using the vehicle's pose data. All voxels in the preset space are pre-marked as invisible to the camera. All voxels that the ray passes through before the first occupying voxel are marked as empty. The first occupying voxel that the ray passes through is marked as visible to the camera.

[0178] When the same voxel is passed through by multiple rays, their priority from high to low is as follows: visible to the camera, empty, invisible to the camera.

[0179] Furthermore, the space occupancy voxelization device also includes:

[0180] Voxel velocity calculation module 209 is used to calculate the velocity of each voxel;

[0181] The voxel velocity calculation module includes:

[0182] The voxel velocity preset unit is used to preset the velocity of all voxels to zero.

[0183] The frame voxel finding unit is used to find all voxels whose center point is located in the tracking detection frame for each dynamic object and whose voxel category is the same as that of the tracking detection frame, and to mark the voxel as the frame voxel of the corresponding tracking detection frame.

[0184] The frame element reference velocity determination unit is used to set the reference velocity of each frame element to be the same as the velocity of the corresponding tracking detection frame;

[0185] The frame element displacement calculation unit is used to calculate the offset value of each frame element relative to the displacement of the corresponding tracking detection frame center axis;

[0186] The frame element rotation angular velocity determination unit is used to calculate the rotation angular velocity of each frame element relative to the central axis of the corresponding tracking and detection frame based on the displacement of each frame element and the rotation angular velocity of the corresponding tracking and detection frame.

[0187] The frame element velocity calculation unit is used to combine the reference velocity and rotational angular velocity of each frame element to obtain the velocity quantity of that frame element.

[0188] Furthermore, it should be noted that the spatial occupancy voxelization device for point cloud data processing provided in the above embodiments is only illustrated by the division of the above functional modules when performing voxelization processing. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the spatial occupancy voxelization device for point cloud data processing can be divided into different functional modules to complete all or part of the functions described above.

[0189] Furthermore, the spatial occupancy voxelization device for point cloud data processing provided in the above embodiments and the spatial occupancy voxelization method for point cloud data processing belong to the same concept. The specific implementation process can be found in the method embodiments, and will not be repeated here.

[0190] Furthermore, in some other embodiments, such as Figure 4 As shown, the present invention also discloses a computing device, comprising:

[0191] One or more processors 301;

[0192] Memory 302;

[0193] And one or more programs, wherein the one or more programs are stored in memory 302 and configured to be executed by one or more processors 301, the one or more programs including instructions for the spatial occupancy voxelization method for point cloud data processing disclosed in the above embodiments.

[0194] Processor 301 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 301 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 301 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 301 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 301 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0195] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 302 are used to store at least one instruction, which is executed by the processor 301 to implement the spatial occupancy voxelization method for point cloud data processing provided in the method embodiments of the present invention.

[0196] In addition, the computing device may optionally include: a peripheral device interface and at least one peripheral device. The processor 301, memory 302, and peripheral device interface can be connected via a bus or signal lines. Each peripheral device can be connected to the peripheral device interface via a bus, signal lines, or a circuit board. Illustratively, peripheral devices include, but are not limited to: radio frequency circuitry, a touchscreen display, audio circuitry, and a power supply.

[0197] Of course, the computing device may also include fewer or more components, and this embodiment does not limit this.

[0198] Furthermore, in some other embodiments, the present invention also discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by a memory and execute the spatial occupancy voxelization method for point cloud data processing disclosed in the above embodiments.

[0199] This invention discloses a method and apparatus for spatial occupancy voxelization in point cloud data processing, which has the following beneficial effects:

[0200] First, this invention can obtain information such as the category, bias value, radar visibility, camera visibility, and velocity of each voxel in three-dimensional space. The high-precision voxelized data provides a theoretical basis for autonomous driving and robotic systems.

[0201] Second, the present invention stacks sparse point cloud data into a dense scene by using a frame stacking method, thereby reducing errors during voxelization.

[0202] Third, the present invention calculates the visibility of each voxel using camera and radar visibility algorithms, which facilitates the subsequent application of voxelized data.

[0203] Fourth, the present invention supplements occasional gaps in the ground that are far from the vehicle by using a ground voxel completion method.

[0204] Fifth, this invention has significant application value and broad application prospects in the fields of high-precision positioning and environmental perception.

[0205] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A spatial occupancy voxelization method for point cloud data processing, characterized in that, The method comprises the following steps: Step S1: collecting point cloud data, images and the pose data of the ego vehicle; Step S2: performing tracking detection on dynamic objects based on the data collected in step S1, to obtain the position, size, orientation, speed and angular velocity of the tracking detection box of each dynamic object in each frame; Step S3: performing segmentation processing on the point cloud data, to determine whether each point cloud in each frame is static point cloud, dynamic point cloud or noise; Step S4: converting the static point cloud in the ego vehicle coordinate system in each frame to the world coordinate system according to the pose data of the ego vehicle in each frame in the whole time sequence; Superimposing the point clouds in the world coordinate system in the whole time sequence to obtain a static point cloud scene in the world coordinate system in the time sequence; Step S5: matching the tracking detection box of the dynamic object in each frame in the whole time sequence with the dynamic point cloud in the corresponding frame, marking the point cloud falling within the tracking detection box as the object point cloud, recording the position of the object point cloud relative to the tracking detection box, and superimposing the point clouds falling within the tracking detection box in the whole time sequence with the relative position to obtain a dynamic point cloud instance of each dynamic object in the time sequence; Step S6: performing dynamic and static point cloud fusion on the static point cloud scene in the world coordinate system in the time sequence obtained in step S4 and the dynamic point cloud instance of each dynamic object in the time sequence obtained in step S5 to obtain a complete superimposed point cloud scene in each frame; Step S7: dividing the space around the ego vehicle into a plurality of voxels based on the complete superimposed point cloud scene in each frame, and counting the category and position of the point cloud appearing in each voxel to obtain the category and bias value of each voxel; Step S8: performing radar visibility calculation and camera visibility calculation on the voxelized scene in each frame; Step S8.1: performing radar visibility calculation on the voxelized scene in each frame, specifically as follows: Taking the position of the radar of the ego vehicle in each frame as the origin, the position of the bias value of the outermost voxel with a predetermined size as the terminal point, drawing a ray, marking all the voxels in front of the first occupied voxel through by the ray as empty, marking the first occupied voxel through by the ray as visible, and marking all the voxels behind the first occupied voxel through by the ray as invisible; When the same voxel is crossed by multiple rays, the priority from high to low is as follows: radar visible, empty, radar invisible; Step S8.2: performing camera visibility calculation on the voxelized scene in each frame, specifically as follows: Taking the position of the camera of the ego vehicle in each frame as the origin, drawing a ray to each pixel of each picture in each frame, back-projecting the ray into the ego vehicle coordinate system through the pose data of the ego vehicle, and marking all the voxels in the predetermined space as camera invisible in advance, marking all the voxels in front of the first occupied voxel through by the ray as empty, and marking the first occupied voxel through by the ray as camera visible; When the same voxel is crossed by multiple rays, the priority from high to low is as follows: camera visible, empty, camera invisible.

2. The spatial occupancy voxelization method of claim 1, wherein, The step S6 comprises: Step S6.1: For the pose data of the ego vehicle in any frame in the whole time sequence, convert the static point cloud scene of the time sequence stack obtained in step S4 in the world coordinate system into the ego vehicle coordinate system to obtain the static point cloud scene in the ego vehicle coordinate system in the frame; Step S6.2: Place the dynamic point cloud instance of the time sequence stack of each dynamic object obtained in step S5 in the static point cloud scene obtained in step S6.1 according to the position of the tracking detection box in the frame to realize the merging and fusion of the dynamic and static point clouds and obtain the complete stacked point cloud scene in the frame; Step S6.3: Repeat steps S6.1-S6.2 until the complete stacked point cloud scene in each frame is obtained.

3. The spatial occupancy voxelization method of claim 1, wherein, The step S7 comprises: Step S7.1: Based on the complete stacked point cloud scene in each frame, divide the space around the ego vehicle into a plurality of voxels according to a preset size; Step S7.2: If the number of point clouds of a single category in each voxel exceeds a set threshold, the voxel is regarded as an occupied voxel, and the category of the occupied voxel is set as the category to which the point cloud with the largest number of point clouds in the voxel belongs; Step S7.3: Statistics the position of each point cloud of the same category as the occupied voxel relative to the center point of the voxel, take the mean value, and take the mean value as the bias value of the voxel.

4. The spatial occupancy voxelization method of claim 3, wherein, The step S7 further comprises: Step S7.4: For all low-position blank voxels, statistics the number of ground voxels in the neighborhood, if the number of neighborhood ground voxels exceeds a set threshold, the blank voxel is filled as a ground voxel; At the same time, statistics the bias mean value of the neighborhood ground voxel as the bias value of the filled ground voxel.

5. The spatial occupancy voxelization method of claim 1, further comprising: Step S9: Calculate the speed of each voxel; The step S9 comprises: Step S9.1: Pre-set the speed of all voxels to zero; Step S9.2: For the tracking detection box of each dynamic object, find all voxels whose center points are located in the tracking detection box and whose categories are the same as the category of the tracking detection box, and mark the voxels as box voxels corresponding to the tracking detection box; Step S9.3: Set the reference speed of each box voxel as the same as the speed of the corresponding tracking detection box; Step S9.4: Calculate the displacement of the bias value of each box voxel relative to the center axis of the corresponding tracking detection box; Step S9.5: Based on the displacement of each box voxel and the angular velocity of rotation of the corresponding tracking detection box, calculate the angular velocity of rotation of each box voxel relative to the center axis of the corresponding tracking detection box; Step S9.6: Combine the reference speed and the angular velocity of rotation of each box voxel to obtain the speed of the voxel.

6. A spatial occupancy voxelization device for point cloud data processing, characterized in that, Comprise: The acquisition module is used for acquiring point cloud data, images and pose data of the ego vehicle; The tracking detection module is used for tracking detection of dynamic objects based on the data acquired by the acquisition module to obtain the position, size, orientation, speed and angular velocity of rotation of the tracking detection box of each dynamic object in each frame; The point cloud segmentation module is used for segmentation processing of the point cloud data to determine whether each point cloud in each frame is a static point cloud or a dynamic point cloud or a noise point; The static point cloud stacking module is configured to convert the static point cloud in the frame of the ego vehicle in a coordinate system of the ego vehicle to a coordinate system of the world according to the pose data of the ego vehicle in each frame in the entire time sequence; The dynamic point cloud stacking module is configured to match the tracking bounding box of the dynamic object in each frame in the entire time sequence with the dynamic point cloud in the corresponding frame, mark the point cloud falling within the tracking bounding box as the object point cloud, and record the position of the point cloud relative to the tracking bounding box, stack the point cloud falling within the tracking bounding box in the entire time sequence with the relative position to obtain the time sequence stacked dynamic point cloud instance of each dynamic object. The dynamic-static point cloud fusion module is configured to perform dynamic-static point cloud fusion on the time sequence stacked static point cloud scene in the coordinate system of the world obtained by the static point cloud stacking module and the time sequence stacked dynamic point cloud instance of each dynamic object obtained by the dynamic point cloud stacking module to obtain a complete stacked point cloud scene in each frame. The point cloud voxelization module is configured to divide the space around the ego vehicle into a plurality of voxels based on the complete stacked point cloud scene in each frame, and count the category and position of the point cloud appearing in each voxel to obtain the category and bias value of each voxel. The voxel visibility calculation module includes: The radar visibility calculation unit is configured to perform radar visibility calculation on the voxel of the voxelized scene in each frame, specifically as follows: The camera visibility calculation unit is configured to perform camera visibility calculation on the voxel of the voxelized scene in each frame, specifically as follows: The dynamic-static point cloud fusion module includes: The conversion unit is configured to convert the time sequence stacked static point cloud scene in the coordinate system of the world obtained by the static point cloud stacking module to the coordinate system of the ego vehicle according to the pose data of the ego vehicle in any frame in the entire time sequence to obtain the static point cloud scene in the coordinate system of the ego vehicle in the frame. The voxel visibility calculation module includes: The radar visibility calculation unit is configured to perform radar visibility calculation on the voxel of the voxelized scene in each frame, specifically as follows: The camera visibility calculation unit is configured to perform camera visibility calculation on the voxel of the voxelized scene in each frame, specifically as follows: The dynamic-static point cloud fusion module includes:

7. The spatial occupancy voxelization device of claim 6, wherein, The conversion unit is configured to convert the time sequence stacked static point cloud scene in the coordinate system of the world obtained by the static point cloud stacking module to the coordinate system of the ego vehicle according to the pose data of the ego vehicle in any frame in the entire time sequence to obtain the static point cloud scene in the coordinate system of the ego vehicle in the frame. ​ The fusion unit is used for placing the time-series stacked dynamic point cloud instance of each dynamic object obtained by the dynamic point cloud stacking module in the static point cloud scene obtained by the conversion unit according to the position of the tracking detection box of the frame, realizing the merging and fusion of the dynamic and static point clouds, and obtaining the complete stacked point cloud scene under the frame. The repeated execution unit is used for repeatedly executing the methods in the conversion unit and the fusion unit until the complete stacked point cloud scene under each frame is obtained.

8. A computing device, characterized by Comprise: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, one or more programs include instructions of the spatial occupancy voxelization method for point cloud data processing in any one of the above claims 1-5.

9. Storage medium, characterized in that The storage medium stores one or more computer-readable programs, and the one or more programs include instructions suitable for being loaded by the memory and executing the spatial occupancy voxelization method for point cloud data processing in any one of the above claims 1-5.

Images