Point cloud data processing method, point cloud data processing system, and computer storage medium

By combining rasterized processing and sparse computing modes for point cloud data, the problem of low redundant computing and resource utilization in point cloud data processing is solved, memory and calculation amount are saved, computing efficiency and detection accuracy are improved sex.

WO2024230436A1PCT designated stage expired Publication Date: 2024-11-14NIO CO LTD

Patent Information

Application Number
PCT/CN2024/088313
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-06
Filing Date
2024-04-17
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

In point cloud data processing, due to the huge and sparseness of the original point cloud data, a large amount of redundant calculations are generated when directly computed by three-dimensional convolution. High demand for memory and bandwidth, and the computing resource utilization rate is sharply limited by global memory bandwidth. decline.

Method used

By rastering the point cloud data, multiple three-dimensional point cloud rasters are formed and entered the sparse operation mode. Three-dimensional convolution operations are performed based on the raster mapping relationship, and the sparse vector operation is converted into dense matrix operation, saving memory and calculation amount .

Benefits of technology

It realizes the reduction of computing volume, shortening computing time, improving hardware utilization, and supporting fast and accurate three-dimensional object detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024088313_14112024_PF_FP_ABST
    Figure CN2024088313_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of data processing, and in particular, to a point cloud data processing method, a point cloud data processing system, and a computer storage medium. The point cloud data processing method according to one aspect of the present application comprises: gridding a point cloud space of point cloud data to form a plurality of three-dimensional point cloud blocks; selectively entering a sparse operation mode according to the point cloud distribution of the point cloud space of the point cloud data and the plurality of three-dimensional point cloud blocks; and performing the following operations in the sparse operation mode: using point cloud blocks having point cloud data among the plurality of three-dimensional point cloud blocks as target point cloud blocks, generating a block mapping relationship according to a corresponding storage address of each target point cloud block, and performing a three-dimensional convolution operation on each target point cloud block according to the block mapping relationship to generate an output point cloud block.
Need to check novelty before this filing date? Find Prior Art

Description

Point cloud data processing method, point cloud data processing system and computer storage medium

[0001] This application claims priority to Chinese patent application 202310508679.4, filed on May 6, 2023, and entitled “Point cloud data processing method, point cloud data processing system and computer storage medium”. The entire disclosure of this Chinese patent application is incorporated herein by reference. Technical Field

[0002] The present application relates to the field of data processing technology, and more specifically to a point cloud data processing method, a point cloud data processing system, and a computer storage medium. Background Art

[0003] With the advancement of artificial intelligence and computer vision technologies, and the increasing emphasis on driving safety in assisted and autonomous driving, technologies for autonomous driving have become a hot topic. From a sensor perspective, lidar (LiDAR) provides high-precision sensing data while being less susceptible to environmental influences, making it an essential sensor in the autonomous driving field. Because laser light only generates an echo when it strikes an object, the raw point cloud data acquired by lidar is typically distributed in a discrete and uneven manner, resulting in a large yet sparse point cloud space. Typically, feature extraction is required for object detection tasks.

[0004] Currently, object detection methods based on point cloud data typically divide the unstructured point cloud space into voxel spaces and directly compute 3D convolutions to extract features from these voxel spaces. These features are then used for point cloud perception tasks. However, since raw point cloud data typically appears as a large and sparse point cloud space, directly computing 3D convolutions results in a large amount of redundant computation and places high demands on memory and bandwidth. Once global memory bandwidth is limited, computational resource utilization plummets.

[0005] Summary of the Invention

[0006] In order to solve or at least alleviate one or more of the above problems, the following technical solutions are provided.

[0007] According to a first aspect of the present application, a point cloud data processing method is provided, the method comprising the following steps: rasterizing the point cloud space of point cloud data to form a plurality of three-dimensional point cloud grids; selectively entering a sparse operation mode based on the point cloud distribution of the point cloud space of the point cloud data and the plurality of three-dimensional point cloud grids; and performing the following operations in the sparse operation mode: taking a point cloud grid having point cloud data in the plurality of three-dimensional point cloud grids as a target point cloud grid and generating a grid mapping relationship based on the corresponding storage address of each target point cloud grid, and performing a three-dimensional convolution operation on each target point cloud grid based on the grid mapping relationship to generate an output point cloud grid.

[0008] According to the point cloud data processing method described in one embodiment of the present application, rasterizing the point cloud space of point cloud data to form multiple three-dimensional point cloud grids includes: using a correction data set to determine the size information of the three-dimensional point cloud grid, wherein the correction data set and the point cloud data have the same probability distribution; and rasterizing the point cloud space of the point cloud data based on the size information to form the multiple three-dimensional point cloud grids.

[0009] According to the point cloud data processing method described in one embodiment of the present application or any one of the above embodiments, generating a grid mapping relationship based on the corresponding storage address of each target point cloud grid includes: determining the size information of the mapping matrix based on the size information of the point cloud space and the size information of the three-dimensional point cloud grid; creating a mapping matrix according to the size information of the mapping matrix; and using the mapping matrix to establish a correspondence between each target point cloud grid and the corresponding storage address to generate the grid mapping relationship.

[0010] According to the point cloud data processing method described in one embodiment of the present application or any one of the above embodiments, the method further includes: determining whether the size of the output point cloud grid is consistent with that of the target point cloud grid; and in response to determining that the size of the output point cloud grid is inconsistent with that of the target point cloud grid, rearranging the output point cloud grid so that the size of the output point cloud grid is consistent with that of the target point cloud grid.

[0011] According to the point cloud data processing method described in one embodiment or any one of the above embodiments of the present application, the method further includes: filtering the output point cloud raster using a mask, and the elements in the mask are determined based on the dimension of the feature vector of the point cloud data contained in the target point cloud raster.

[0012] According to the point cloud data processing method described in one embodiment or any one of the above embodiments of the present application, the elements in the mask are determined in the following manner: when the dimension of the feature vector of the point cloud data contained in the target point cloud grid is zero, the elements in the mask are set to zero; and when the dimension of the feature vector of the point cloud data contained in the target point cloud grid is not zero, the elements in the mask are set to one.

[0013] According to the point cloud data processing method described in one embodiment of the present application or any one of the above embodiments, selectively entering the sparse operation mode based on the point cloud distribution of the point cloud space of the point cloud data and the multiple three-dimensional point cloud grids includes: inputting the point cloud distribution of the point cloud space of the point cloud data and the multiple three-dimensional point cloud grids into a sparse operation evaluation model to determine whether to enter the sparse operation mode based on the output result of the sparse operation evaluation model.

[0014] According to the point cloud data processing method described in one embodiment or any one of the above embodiments of the present application, the method also includes performing the following operations in response to choosing not to enter the sparse operation mode: dividing the point cloud space of the point cloud data into voxel space; and generating a voxel feature map by sampling the features of the point cloud data in each voxel space determined; and performing a three-dimensional convolution operation or a sparse convolution operation on the voxel feature map.

[0015] According to the second aspect of the present application, a point cloud data processing system is provided, comprising: a memory; a processor coupled to the memory; and a computer program stored on the memory and running on the processor, wherein the execution of the computer program results in the execution of the steps of the point cloud data processing method described in the first aspect of the present application.

[0016] According to a third aspect of the present application, a computer storage medium is provided, which includes instructions, and when the instructions are run, the steps of the point cloud data processing method according to the first aspect of the present application are executed.

[0017] The point cloud data processing solution according to one or more embodiments of the present application converts sparse vector operations on the point cloud data into dense matrix operations by using a point cloud grid containing point cloud data as a target point cloud grid and performing a three-dimensional convolution operation on each target point cloud grid, thereby saving memory, reducing the amount of calculation, shortening calculation time, and improving hardware utilization. The point cloud data processing solution according to one or more embodiments of the present application can be applied to various scenarios such as intelligent robot interaction, autonomous driving, and assisted driving, thereby achieving fast and accurate three-dimensional target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or other aspects and advantages of the present application will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar elements are represented by the same reference numerals. The accompanying drawings include:

[0019] FIG1 shows a flowchart of a point cloud data processing method according to one or more embodiments of the present application.

[0020] FIG2 shows a schematic diagram of generating a grid mapping relationship according to one or more embodiments of the present application.

[0021] FIG3 shows a schematic diagram of point cloud data processing according to one or more embodiments of the present application.

[0022] FIG4 shows a block diagram of a point cloud data processing system according to one or more embodiments of the present application. DETAILED DESCRIPTION

[0023] The present application is described more fully below with reference to the accompanying drawings, which illustrate exemplary embodiments of the present application. However, the present application may be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. The foregoing embodiments are provided to make the disclosure herein comprehensive and complete, so as to more fully convey the scope of protection of the present application to those skilled in the art.

[0024] In this specification, terms such as "comprise" and "include" indicate that in addition to the units and steps directly and clearly stated in the specification and claims, the technical solution of this application does not exclude the situation where it has other units and steps that are not directly or clearly stated.

[0025] Unless otherwise specified, terms such as "first" and "second" do not indicate the order of units in terms of time, space, size, etc., but are only used to distinguish between units.

[0026] Hereinafter, various exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings.

[0027] FIG1 shows a flowchart of a point cloud data processing method according to one or more embodiments of the present application.

[0028] As shown in FIG. 1 , in step S101 , a point cloud space of point cloud data is rasterized to form a plurality of three-dimensional point cloud grids.

[0029] Optionally, point cloud data refers to three-dimensional data determined based on the echo signal reflected back to the vehicle after an environmental point in the vehicle's driving environment receives an electromagnetic wave sent by the vehicle to the environmental point, and the three-dimensional data includes the coordinates of the environmental point in the three-dimensional coordinate system. Point cloud space refers to a three-dimensional space formed by three-dimensional point cloud data. In one or more embodiments of the present application, the vehicle can send electromagnetic waves to environmental points in the driving environment through environmental sensing devices such as radar, where the radar can include but is not limited to millimeter wave radar and laser radar. For example, a laser radar can be used as an environmental sensing device to obtain a three-dimensional point cloud in the vehicle's driving environment through the echo signal generated when the laser emitted by the laser radar is irradiated to the environmental point. For example, the angular resolution of the laser radar can be 0.06°×0.06°, the distance resolution can be 2cm, the scanning FOV can be 120°×25°, and the effective detection distance can be 500m.

[0030] Optionally, in step S101, rasterizing the point cloud space of the point cloud data may include dividing the point cloud space according to the three dimensions of the x, y, and z axes in the coordinate system of the point cloud space to form a plurality of three-dimensional point cloud grids. Optionally, the size information of the three-dimensional point cloud grids may be first determined using a correction dataset, wherein the correction dataset and the point cloud data have the same probability distribution, and then rasterizing the point cloud space of the point cloud data based on the determined size information to form the plurality of three-dimensional point cloud grids. Exemplarily, the size information of the three-dimensional point cloud grids may be determined using the correction dataset in an offline phase, and the size information with the best performance may be selected as the determined size information, wherein the best performance may be determined based on one or more of software runtime and hardware utilization.

[0031] In step S103 , a sparse operation mode is selectively entered based on the point cloud distribution in the point cloud space of the point cloud data and a plurality of three-dimensional point cloud grids.

[0032] Optionally, in step S103, the point cloud distribution of the point cloud space of the point cloud data and multiple three-dimensional point cloud grids can be input into a sparse operation evaluation model to determine whether to enter the sparse operation mode based on the output result of the sparse operation evaluation model. Exemplarily, the sparse operation evaluation model can be used to analyze the point cloud distribution of the point cloud space of each frame of point cloud data and the number of point cloud grids generated based on each frame of point cloud data in real time to compare the software running time and hardware utilization under the sparse operation mode and other operation modes (for example, direct three-dimensional convolution operation, sparse convolution operation, etc.) to determine whether to enter the sparse operation mode. When it is determined to enter the sparse operation mode based on the output result of the sparse operation evaluation model, step S105 is entered; when it is determined not to enter the sparse operation mode based on the output result of the sparse operation evaluation model, the point cloud space of the point cloud data can be divided into voxel space, and a voxel feature map is generated by sampling the features of the point cloud data in each voxel space, and a three-dimensional convolution operation or a sparse convolution operation is performed on the voxel feature map. As you can understand, voxel, short for volume pixel, is the smallest unit of digital data in three-dimensional space, conceptually similar to the smallest unit of pixel in two-dimensional space. For example, features of point cloud data may include, but are not limited to, the point cloud's position (e.g., the x-, y-, and z-axis coordinates in a three-dimensional coordinate system) and intensity.

[0033] Optionally, whether to perform a three-dimensional convolution operation or a sparse convolution operation on the voxel feature map can be determined based on the point cloud density of the voxel space, wherein the point cloud density of the voxel space can be determined by the ratio of the number of point clouds contained in each voxel space in the point cloud space to the number of all point clouds in the point cloud data. When the point cloud density of the voxel space is greater than or equal to a preset threshold, a three-dimensional convolution operation is performed on the voxel feature map of the voxel space; when the point cloud density of the voxel space is less than the preset threshold, a sparse convolution operation is performed on the voxel feature map of the voxel space. Since the point cloud density represents the number of point clouds contained in the voxel, the larger the point cloud density, the greater the probability that the voxel space corresponds to an object.

[0034] In step S105, the following operations are performed in the sparse operation mode: a point cloud grid having point cloud data among the multiple three-dimensional point cloud grids is used as a target point cloud grid and a grid mapping relationship is generated based on the corresponding storage address of each target point cloud grid, and a three-dimensional convolution operation is performed on each target point cloud grid based on the grid mapping relationship to generate an output point cloud grid.

[0035] Optionally, in step S105, the size information of the mapping matrix can be determined based on the size information of the point cloud space and the size information of the three-dimensional point cloud grid, the mapping matrix can be created based on the size information of the mapping matrix, and the mapping matrix can be used to establish a correspondence between each target point cloud grid and the corresponding storage address to generate a grid mapping relationship. After the mapping matrix is ​​used to establish the correspondence between each target point cloud grid and the corresponding storage address, the corresponding storage address of each target point cloud grid in the mapping matrix can be traversed to perform a three-dimensional convolution operation on each target point cloud grid.

[0036] Optionally, the point cloud data processing method according to one or more embodiments of the present application may further include determining whether the size of the output point cloud grid is consistent with that of the target point cloud grid, and rearranging the output point cloud grid in response to determining that the size of the output point cloud grid is inconsistent with that of the target point cloud grid, so that the size of the output point cloud grid is consistent with that of the target point cloud grid. Optionally, the point cloud data processing method according to one or more embodiments of the present application may further include filtering the output point cloud grid using a mask, wherein the elements in the mask are determined based on the dimension of the feature vector of the point cloud data contained in the target point cloud grid. Optionally, when the dimension of the feature vector of the point cloud data contained in the target point cloud grid is zero, the elements in the mask are set to zero; and when the dimension of the feature vector of the point cloud data contained in the target point cloud grid is not zero, the elements in the mask are set to one.

[0037] Optionally, the point cloud data processing method according to one or more embodiments of the present application can be applied to autonomous driving vehicles, which perform real-time target detection on the point cloud data collected by the lidar to locate the positions of pedestrians, obstacles, vehicles, etc., and then execute corresponding driving strategies.

[0038] According to one or more embodiments of the present application, the point cloud data processing method can rasterize the point cloud space of the point cloud data according to the size information of the point cloud grid with the best performance to form multiple three-dimensional point cloud grids, perform three-dimensional convolution operations on the point cloud grids with point cloud data, and skip the calculation for the point cloud grids without point cloud data. Therefore, compared with the conventional direct calculation of three-dimensional convolution, the point cloud data processing method according to one or more embodiments of the present application does not store invalid point cloud grids, saves memory, does not calculate invalid point cloud grids, and reduces the amount of calculation. Compared with conventional sparse convolution, the point cloud data processing method according to one or more embodiments of the present application is simple to implement, eliminating the complex hash table building process, a large number of atomic calculations, and a large number of discontinuous read and write operations. For each target point cloud grid, data reading is continuous and calculation is regular, thereby improving the utilization rate of the tensor computing core of the convolution operation on the general image processor and shortening the convolution calculation time.

[0039] The point cloud data processing method according to one or more embodiments of the present application converts sparse vector operations on the point cloud data into dense matrix operations by using a point cloud grid containing point cloud data as a target point cloud grid and performing a three-dimensional convolution operation on each target point cloud grid, thereby saving memory, reducing the amount of calculation, shortening calculation time, and improving hardware utilization. The point cloud data processing solution according to one or more embodiments of the present application can be applied to various scenarios such as intelligent robot interaction, autonomous driving, and assisted driving, thereby achieving fast and accurate three-dimensional target detection.

[0040] The point cloud data processing method according to one or more embodiments of the present application will be further described below with reference to FIG. 2-3 .

[0041] FIG2 shows a schematic diagram of generating a grid mapping relationship according to one or more embodiments of the present application.

[0042] As shown in FIG2 , the point cloud space of the point cloud data can be represented by [D, H, W, C], where D represents the depth of the point cloud space, H represents the height of the point cloud space, W represents the width of the point cloud space, and C represents the dimension of the feature vector of the point cloud data.

[0043] Optionally, after acquiring the point cloud space of the point cloud data using a laser radar, the size information of the three-dimensional point cloud grid can be determined first using a correction data set having the same probability distribution as the point cloud data [D block , H block , W block , C], where D block Indicates the depth of the point cloud grid, H block Indicates the height of the point cloud grid, W block Represents the width of the point cloud grid, and C represents the dimension of the feature vector of the point cloud grid. Then, based on the determined size information of the three-dimensional point cloud grid [D block , H block , W block , C] rasterizes the point cloud space [D, H, W, C] of the point cloud data to form [D map , H map , W map ] three-dimensional point cloud grid, where [D map , H map , W map ] can be calculated by the following formula (1):

[0044] Alternatively, the calibration dataset can be used to determine multiple dimensions of the 3D point cloud grid in an offline phase, and the dimension with the best performance can be selected as the final dimension. The optimal performance can be determined based on one or more of software runtime and hardware utilization. This allows for independent configuration of the dimension of each 3D point cloud grid, improving computational flexibility.

[0045] As shown in FIG2, based on the size information of the determined three-dimensional point cloud grid [D block , H block , W block , C] rasterizes the point cloud space [D, H, W, C] of the point cloud data to form [D map , H map , W map ] three-dimensional point cloud grid, you can use [D map , H map , W map ] three-dimensional point cloud grids with point cloud data (as shown by the gray dots in FIG2 ) as target point cloud grids, and generate a grid mapping relationship based on the corresponding storage address of each target point cloud grid. For example, a grid with a size of [D map , H map , W map ] is used to establish a corresponding relationship between each target point cloud grid and the corresponding storage address to generate a grid mapping relationship. For example, the first element address block[i][0,0,0] of the target point cloud grid can be used to represent the corresponding storage address of each target point cloud grid, wherein the first element address block[i][0,0,0] of the target point cloud grid can be calculated by the following formula (2): block[i][0,0,0]=*(data+i×D block ×H block ×W block ×C) Formula (2)

[0046] Where block[i] represents the i-th target point cloud grid, data represents the starting address of the target point cloud grid, and D block Indicates the depth of the target point cloud grid, H block Indicates the height of the target point cloud grid, W block Denotes the height of the target point cloud grid, and C denotes the dimension of the feature vector of the target point cloud grid. As shown in FIG2 , for example, the size of [D map , H map , W map ] is used to establish a corresponding relationship between the target point cloud grids numbered 0, 1...i and the corresponding storage addresses block[i][0, 0, 0] to generate a grid mapping relationship.

[0047] Through the process of rasterizing the point cloud space [D, H, W, C] of point cloud data shown in Figure 2, the unstructured point cloud data can be converted into a structured three-dimensional point cloud grid, so that the convolution processing process can be accelerated through the matrix calculation unit in the subsequent convolution processing, reducing the software running time and improving the hardware utilization.

[0048] FIG3 shows a schematic diagram of point cloud data processing according to one or more embodiments of the present application.

[0049] As shown in Figure 3, after determining the target point cloud grids Bolck 1, Bolck 2...Bolck n and generating the grid mapping relationship, the corresponding storage addresses addr1, addr2, addr3, addr4... of each target point cloud grid in the grid mapping relationship can be traversed to perform a three-dimensional convolution operation on each target point cloud grid. Optionally, an expansion operation can be performed on the target point cloud grid Bolck 1 first. Assuming that the storage address of the target point cloud grid Bolck 1 is addr2, performing an expansion operation on the target point cloud grid Bolck 1 may include merging the target point cloud grids stored at the storage address adjacent to the storage address addr2 (for example, the storage address addr3) into the target point cloud grid Bolck 1 to generate the point cloud grid Bolck 1'. Next, a three-dimensional convolution operation is performed on the expanded point cloud grid Bolck 1′ to generate an output point cloud grid Bolck 1″. It can be understood that the size of the output point cloud grid Bolck 1″ generated by the three-dimensional convolution operation may become smaller. Therefore, the point cloud data processing method shown in Figure 3 may further include determining whether the size of the output point cloud grid Bolck 1″ is consistent with that of the target point cloud grid Bolck 1, and when it is determined that the size of the output point cloud grid Bolck 1″ is inconsistent with that of the target point cloud grid Bolck 1, the output point cloud grid Bolck 1″ can be rearranged, for example, 8 output point cloud grids Bolck 1″ are combined into one point cloud grid so that the size of the output point cloud grid Bolck 1″ is consistent with that of the target point cloud grid Bolck 1.

[0050] As shown in Figure 3, the output point cloud grid Bolck 1″ can also be filtered using a mask to generate a filtered point cloud grid Bolck 1″′, where the elements in the mask are determined based on the dimension of the feature vector of the point cloud data contained in the target point cloud grid Bolck 1. Optionally, when the dimension of the feature vector of the point cloud data contained in the target point cloud grid Bolck 1 is zero, the elements in the mask are set to zero; and when the dimension of the feature vector of the point cloud data contained in the target point cloud grid Bolck 1 is not zero, the elements in the mask are set to one.

[0051] FIG4 shows a block diagram of a point cloud data processing system according to one or more embodiments of the present application.

[0052] As shown in FIG4 , the point cloud data processing system 40 includes a communication unit 410 , a memory 420 (e.g., a non-volatile memory such as a flash memory, a ROM, a hard drive, a magnetic disk, an optical disk), a processor 430 , and a computer program 440 stored on the memory 420 and executable on the processor 430 .

[0053] The communication unit 410 serves as a communication interface and is configured to establish a communication connection between the point cloud data processing system 40 and an external device or network (eg, a laser radar, etc.).

[0054] The memory 420 stores a computer program 440 that can be executed by the processor 430. In addition, the memory 420 can also store data generated when the processor 430 executes the computer program (e.g., target point cloud grid, grid mapping relationship, etc.) and data or commands received from the outside via the communication unit 410.

[0055] The processor 430 is configured to execute the computer program 440 to implement the point cloud data processing method according to one or more embodiments of the present application.

[0056] According to one or more embodiments of the present application, the point cloud data processing system uses a point cloud grid containing point cloud data as a target point cloud grid and performs a three-dimensional convolution operation on each target point cloud grid. This system can convert sparse vector operations on the point cloud data into dense matrix operations, thereby saving memory, reducing the amount of calculation, shortening calculation time, and improving hardware utilization. According to one or more embodiments of the present application, the point cloud data processing solution can be applied to various scenarios such as intelligent robot interaction, autonomous driving, and assisted driving, thereby achieving fast and accurate three-dimensional target detection.

[0057] In addition, as described above, the present application may also be implemented as a computer-readable storage medium, wherein the computer-readable storage medium includes instructions, and the instructions, when executed, execute the point cloud data processing method according to one aspect of the present application.

[0058] In the applicable situation, the combination of hardware, software or hardware and software can be used to realize the various embodiments provided by the application. Moreover, in the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be combined into a composite component comprising software, hardware and / or both. In the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be divided into a subcomponent comprising software, hardware or both. In addition, in the applicable situation, it is contemplated that the software component can be implemented as a hardware component, and vice versa.

[0059] Software according to the present application (such as program code and / or data) can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special computers and / or computer systems, networked and / or otherwise. Where applicable, the order of the various steps described herein can be changed, combined into composite steps and / or divided into sub-steps to provide the features described herein.

[0060] The embodiments and examples set forth herein are provided to best illustrate embodiments according to the present application and its specific applications, and thereby enable those skilled in the art to make and use the present application. However, those skilled in the art will appreciate that the above description and examples are provided for ease of illustration and example only. The descriptions set forth are not intended to be exhaustive of all aspects of the present application or to limit the present application to the precise forms disclosed.

Claims

1. A point cloud data processing method, characterized in that: The method comprises the following steps: Rasterizing the point cloud space of the point cloud data to form a plurality of three-dimensional point cloud grids; Selectively entering a sparse operation mode based on the point cloud distribution of the point cloud space of the point cloud data and the plurality of three-dimensional point cloud grids; as well as The following operations are performed in the sparse operation mode: a point cloud grid having point cloud data among the multiple three-dimensional point cloud grids is used as a target point cloud grid and a grid mapping relationship is generated based on the corresponding storage address of each target point cloud grid, and a three-dimensional convolution operation is performed on each target point cloud grid based on the grid mapping relationship to generate an output point cloud grid.

2. The method according to claim 1, wherein rasterizing the point cloud space of the point cloud data to form a plurality of three-dimensional point cloud grids comprises: Determining size information of a three-dimensional point cloud grid using a correction data set, wherein the correction data set and the point cloud data have the same probability distribution; as well as The point cloud space of the point cloud data is rasterized based on the size information to form the multiple three-dimensional point cloud grids.

3. The method according to claim 2, wherein generating a grid mapping relationship based on the corresponding storage address of each target point cloud grid comprises: Determining size information of a mapping matrix based on size information of the point cloud space and size information of the three-dimensional point cloud grid; Creating a mapping matrix according to the size information of the mapping matrix; as well as The mapping matrix is ​​used to establish a corresponding relationship between each target point cloud grid and a corresponding storage address to generate the grid mapping relationship.

4. The method according to claim 1, wherein the method further comprises: Determine whether the size of the output point cloud grid is consistent with that of the target point cloud grid; as well as In response to determining that the sizes of the output point cloud grid and the target point cloud grid are inconsistent, the output point cloud grid is rearranged so that the sizes of the output point cloud grid and the target point cloud grid are consistent.

5. The method according to claim 1, wherein the method further comprises: The output point cloud grid is filtered using a mask, wherein elements in the mask are determined based on the dimensions of feature vectors of point cloud data contained in the target point cloud grid.

6. The method according to claim 5, wherein the elements in the mask are determined by: When the dimension of the feature vector of the point cloud data contained in the target point cloud grid is zero, setting the elements in the mask to zero; and When the dimension of the feature vector of the point cloud data contained in the target point cloud grid is not zero, the elements in the mask are set to one.

7. The method according to claim 1, wherein selectively entering the sparse operation mode based on the point cloud distribution of the point cloud space of the point cloud data and the plurality of three-dimensional point cloud grids comprises: The point cloud distribution of the point cloud space of the point cloud data and the multiple three-dimensional point cloud grids are input into a sparse operation evaluation model to determine whether to enter the sparse operation mode based on the output result of the sparse operation evaluation model.

8. The method of claim 1, wherein the method further comprises, in response to selecting not to enter the sparse operation mode, performing the following operations: Dividing the point cloud space of the point cloud data into voxel space; and Generate a voxel feature map by sampling the features of the point cloud data in each voxel space determined; and A three-dimensional convolution operation or a sparse convolution operation is performed on the voxel feature map.

9. A point cloud data processing system, characterized in that: The system comprises: Memory; a processor coupled to the memory; and A computer program stored on the memory and running on the processor, wherein the running of the computer program results in the execution of the point cloud data processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer storage medium comprises instructions, which, when executed, execute the point cloud data processing method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Point cloud data processing method and device, electronic equipment and storage medium

    CN113971712A

  • Point cloud processing method and system, electronic equipment and computer readable storage medium

    CN114863108A

  • Target detection method, computer equipment, computer readable storage medium and vehicle

    CN115082891A

  • Point cloud data processing method, point cloud data processing system and computer storage medium

    CN116778440A

  • Target object detection method and apparatus, electronic device, and storage medium

    WO2022017129A1

Cited By

  • Disordered point cloud search method and device based on sparse matrix

    CN119963849A