Point cloud analysis method, device and readable medium based on data reuse and calculation skipping

Abstract

The present invention discloses a point cloud analysis method, device, and readable medium based on data reuse and computation skipping. The method comprises: inputting point cloud data to be analyzed into a point cloud analysis model, entering the first level, outputting sampling results through the sampling layer of the first level, reusing the sampling results of the current level sampling layer in the sampling process of the next level sampling layer, applying an inter-level data reuse method based on block-level pipeline delayed aggregation to the coordinates and features of the retained points in the sampling results of the current level to obtain aggregated features of the current level, performing dynamic point pruning operations on the aggregated features of the current level through a pruning layer, obtaining point cloud data output by the current level, using the point cloud data output by the current level as the point cloud data input to the next level, and using the point cloud data output by the last level as the analysis result. The present invention can effectively reduce the amount of computation and minimize off-chip memory access.

Description

Technical Field

[0001] The present invention relates to the field of point cloud processing, and in particular to a point cloud analysis method, device and readable medium based on data reuse and calculation skipping. Background Art

[0002] In recent years, point clouds have been increasingly used in various applications, such as automobiles, robots, drones, and smartphone depth cameras. Point cloud data consists of a series of point coordinates and feature data (such as RGB color). Since only discrete points are needed to represent the characteristics of an entire space, point clouds are an efficient data format for spatial representation. Since point cloud acquisition devices such as radar can directly collect the precise coordinates of each point, point cloud data representation is more precise and offers higher resolution. Therefore, point clouds offer significant advantages such as ease of acquisition, high resolution, and efficient data format. Consequently, point clouds have become as important as images and videos for deep learning applications such as autonomous driving, photography, and virtual and augmented reality. Deep neural networks have been used for point cloud analysis. Point cloud neural networks, a type of neural network developed from PointNet, have demonstrated excellent performance in point cloud analysis. Point cloud neural networks directly process points to extract features, addressing the issues of disordered and discrete data. Furthermore, the mapping operations (including sampling and combination operations) within point cloud neural networks enable the network to possess multi-layered and powerful feature extraction capabilities, resulting in high inference accuracy and robustness. The use of shared weight MLP and pooling in point cloud neural networks makes the model compact and computationally efficient.

[0003] When deploying point cloud neural networks on edge devices such as wearables, mobile devices, and Internet of Things (IoT) devices, these devices need to interact with humans in real time and instantly perceive and understand the environment. Therefore, high inference accuracy and low latency and energy consumption are crucial. However, the high computational complexity, large number of parameters, and low data reuse rate of point cloud neural networks result in excessive energy consumption and long latency for computation and storage access, which cannot meet the device's requirements. Therefore, there is an urgent need to design a point cloud neural network that can minimize off-chip memory access while reducing computational complexity.

[0004] like Figure 1As shown in the figure, the input of the point cloud neural network is point cloud data, and the output is the analysis result, such as identifying the category of the point cloud, the location and occupied area of ​​the object in the point cloud, which object each point in the point cloud belongs to, etc. The backbone of the architecture consists of multiple levels (Stage). Each level generally contains a mapping layer, several consecutive multi-layer perceptron (MLP) layers and an aggregation layer. The disorder and discreteness of point cloud data pose considerable challenges to feature extraction. To meet these challenges, the mapping layer of the point cloud neural network uses sampling and grouping operations to construct the relationship between input points and output points, realizing powerful multi-level feature extraction, thereby achieving high accuracy and robustness of reasoning.

[0005] Sampling: Point cloud neural networks generally use the farthest point sampling (FPS) method to reduce the amount of data while enhancing the data representation ability. By selecting the point farthest from the previous sampling point in each iteration, a subset of the point cloud is gradually generated. The sampling process of the original FPS is to find all output points one by one from the input point cloud data to be analyzed. The process of sampling all output points is divided into two steps. The first step is to randomly select a point from the point cloud data as the initial sampling point, and then calculate the Euclidean distance between each unsampled point and the sampled point cloud. The second step is to add the point with the largest Euclidean distance to the sampled point cloud. The whole process of FPS is as follows Figure 1 As shown in (a), it is assumed that the initial sampling point a is randomly selected from the input point cloud data to be analyzed, and then the Euclidean distance between a and other points is calculated and compared. Since b has the largest Euclidean distance, b is selected as the second sampling point in the sampled point cloud. Then a new round of iteration begins, and after calculating the distance between each unsampled point (c, d, e, f) and the sampled point cloud (a, b), {a, b, c} is used as the sampled point cloud. In short, FPS discards some points of the input point cloud (IPC), and the discarded points are called DP (Discarded Points). The other points remain unchanged, and the retained points are called RP (Retained Points).

[0006] Combination: After FPS samples the point cloud, the combination operation uses each point in RP as the center point (Center Points, CP) and searches for the neighboring points (Neighbor Points, NP) of each CP in the point cloud IPC (including RP and DP) before FPS sampling. The combination operation does not change the coordinates of the point, but is only used to build the relationship between each point in RP and IPC, and obtain the mapping relationship table of all RP points and IPC, which is called NIT (Neighbor Index Table). Point cloud neural networks usually use KNN (K-Nearest-Neighbors) algorithm or BQ (Ball Query) algorithm to complete the combination operation. KNN will select the first K nearest points based on the distance from all points in IPC to CP. BQ further requires that these neighboring points need to be within a sphere with a radius of R based on KNN. For example Figure 1 As shown in (b), assuming the number of neighboring input points K is 3, for output point a', the BQ algorithm searches for four input points a, c, d, and e within a sphere of radius R centered on output point a'. It then selects the three points a, d, and e closest to output point a'. BQ considers points a, d, and e to be the neighboring point combination of output point a'. The relationship between output point a' and input points a, d, and e is called the mapping of output point a'. The combination operation also combines the features of points a, d, and e into a feature group, which is used by the subsequent MLP to calculate the features of output point a'.

[0007] MLP: MLP is a convolutional layer with a convolution kernel size of 1×1. In the MLP of the point cloud neural network, the filter is convolved with the features of all input points. Figure 1 (d) shows the changes in the point cloud within a layer, where p represents the coordinates of the input point, F represents the feature group (the combination operation combines the features of neighboring points), and R represents the partial sum group after executing multiple MLPs on the input feature group. The point cloud neural network stacks multiple MLPs within each layer to enhance the feature extraction capability.

[0008] Aggregation: Point cloud neural networks often use max pooling to aggregate parts and groups calculated by MLP from features of neighboring points. Figure 1 As shown in (d), the channel maximum of the partial and group R0, R2, R3, R4, R5 and R7 is calculated to obtain the feature F0' of the output point.

[0009] However, compared to two-dimensional convolutional neural networks, point cloud neural networks can only read / write the features of one point at a time in memory during point cloud data processing, rather than reading / writing an entire block of feature maps and filters from off-chip memory at once, as is the case when running two-dimensional convolutional neural networks. This results in lower interface bandwidth efficiency. In addition, after delayed aggregation, the output feature maps need to be combined into output feature map groups. At this time, the number of channels of the output feature maps is usually twice that of the input feature maps, so the number of feature channels that need to be combined in the combination operation is also doubled, significantly increasing the search domain, computational complexity, storage space usage, and off-chip storage access of the combination operation. Therefore, the combination and aggregation operations limit the acceleration effect and become a new bottleneck besides convolution. Summary of the Invention

[0010] The purpose of this application is to propose a point cloud analysis method, device and readable medium based on data reuse and calculation skipping to address the technical problems mentioned above.

[0011] In a first aspect, the present invention provides a point cloud analysis method based on data reuse and computation skipping, comprising the following steps:

[0012] Obtain point cloud data to be analyzed;

[0013] A point cloud analysis model based on data reuse and computation skipping is constructed and trained to obtain a trained point cloud analysis model. The point cloud neural network includes several layers, each of which includes a sampling layer, a combination layer, a multi-layer perceptron unit, an aggregation layer, and a pruning layer. The multi-layer perceptron unit includes multiple multi-layer perceptrons connected in sequence.

[0014] The point cloud data to be analyzed is input into the trained point cloud analysis model. The point cloud data to be analyzed is input into the first level of the trained point cloud analysis model. The sampling result is output through the sampling layer of the first level. The discarded points and retained points of the next level of sampling layer are determined based on the sampling result of the current level of sampling layer. The index and coordinates of the retained points of each level of sampling layer are established in sequence and output as the sampling result in combination with the features of the retained points. The sampling result of the current level of sampling layer is reused in the sampling process of the next level of sampling layer. The coordinates and features of the retained points in the sampling results adopt an inter-level data reuse method based on block-level pipeline delayed aggregation in the combination layer, multi-layer perceptron unit and aggregation layer of the current level, and skip the combination operation, convolution operation and aggregation operation of the discarded points of the current level in the previous level to obtain the aggregation features of the current level. The aggregation features of the current level are subjected to dynamic point pruning operation in the pruning layer to obtain the point cloud data output by the current level. The point cloud data output by the current level is used as the point cloud data input to the next level, and the point cloud data output by the last level is used as the analysis result.

[0015] In a specific embodiment, the coordinates and features of the retained points in the sampling results of the current level are reused in the combination layer, multi-layer perceptron unit, and aggregation layer of the current level using an inter-level data multiplexing method based on block-level pipeline delay aggregation, specifically including:

[0016] The coordinates and features of the retained points in the sampling results of the current level are divided into multiple point cloud blocks, and the following pipeline process is repeated for different point cloud blocks at different time steps:

[0017] The coordinates of the retained points in the point cloud block are input into the combination layer for combination operation to obtain the neighbor index table;

[0018] The features of the retained points in the point cloud block are input into the multi-layer perceptron unit for convolution operation to obtain the output feature map;

[0019] The neighbor index table and the output feature map are input into the aggregation layer for aggregation operation to obtain the aggregated features of the point cloud block;

[0020] The aggregate features of all point cloud blocks constitute the aggregate features of the current level.

[0021] In a specific embodiment, in the combination operation, each reserved point of the current level is used as a center point, and a neighborhood search process is used to search for neighboring points of each center point in the point cloud data input by the sampling layer of the current level, and a mapping relationship between all reserved points and their neighboring points in the point cloud data input by the sampling layer of the current level is established to obtain a neighbor index table;

[0022] In the aggregation operation, the index of the neighboring point in the neighbor index table is used as the address, and the feature maps corresponding to the K neighboring points of the center point are obtained in the output feature map and aggregated to obtain the aggregated features of the point cloud block.

[0023] In a specific embodiment, a mask is constructed based on the sampling results of the sampling layer at the current level, and the retained points in the mask are marked as 1 and the discarded points are marked as 0;

[0024] Before the combination operation of the first level combination layer, the distance set between all points in the point cloud data to be analyzed is calculated and sorted from small to large to form an index sequence of points with distances from the center point from small to large;

[0025] The mask obtained according to the sampling result of the sampling layer of the current level selectively filters out the index sequence of the current level in the index sequence of the previous level, and the distance in the index sequence of the current level is used in the neighborhood search process in the combination operation of the current level.

[0026] In a specific embodiment, it also includes:

[0027] Determining, in the currently input point cloud data to be analyzed, newly added point clouds relative to the last input point cloud data to be analyzed, and determining, in the currently input point cloud data to be analyzed, associated portions corresponding to the last input point cloud data to be analyzed;

[0028] In the current level, the calculation results of the newly added point cloud in the sampling layer, combination layer, multi-layer perceptron unit, and aggregation layer are calculated respectively, and combined with the calculation results of the associated parts of the last input point cloud data to be analyzed in the sampling layer, combination layer, multi-layer perceptron unit, and aggregation layer respectively, to obtain the aggregation features of the current input point cloud data to be analyzed at the current level, and then pass through the pruning layer to obtain the point cloud data output at the current level.

[0029] In a specific embodiment, the following process is performed in the pruning layer:

[0030] S41, sorting the aggregated features of the current level according to their numerical values, and determining the threshold of the current level;

[0031] S42, setting the values ​​of the aggregated features of the current level that are lower than the threshold of the current level to zero;

[0032] S43, determine whether the threshold of the current level meets the requirements. If so, obtain the point cloud data output by the current level. Otherwise, adjust the threshold of the current level and repeat steps S42-S43 until the threshold of the current level meets the requirements.

[0033] In a second aspect, the present invention provides a point cloud analysis device based on data reuse and calculation skipping, comprising:

[0034] a data acquisition module configured to acquire point cloud data to be analyzed;

[0035] a model building module configured to build and train a point cloud analysis model based on data reuse and computation skipping to obtain a trained point cloud analysis model, wherein the point cloud neural network includes several layers, each layer including a sampling layer, a combination layer, a multi-layer perceptron unit, an aggregation layer, and a pruning layer, and the multi-layer perceptron unit includes a plurality of multi-layer perceptrons connected in sequence;

[0036] The analysis module is configured to input the point cloud data to be analyzed into the trained point cloud analysis model, input the point cloud data to be analyzed into the first level of the trained point cloud analysis model, output the sampling results through the sampling layer of the first level, determine the discarded points and retained points of the next level of sampling layer based on the sampling results of the current level of sampling layer, establish the index and coordinates of the retained points of each level of sampling layer in sequence and output them as the sampling results in combination with the features of the retained points, reuse the sampling results of the current level of sampling layer in the sampling process of the next level of sampling layer, and The coordinates and features of the retained points in the sampling results of the current level adopt an inter-level data reuse method based on block-level pipeline delayed aggregation in the combination layer, multi-layer perceptron unit and aggregation layer of the current level, and skip the combination operation, convolution operation and aggregation operation of the discarded points of the current level in the previous level to obtain the aggregated features of the current level. The aggregated features of the current level are subjected to dynamic point pruning operation in the pruning layer to obtain the point cloud data output by the current level. The point cloud data output by the current level is used as the point cloud data input to the next level, and the point cloud data output by the last level is used as the analysis result.

[0037] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0039] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in any implementation manner in the first aspect when the computer program is executed by a processor.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The point cloud analysis method based on data reuse and computation skipping proposed in the present invention adopts inter-level data reuse based on block-level pipeline delay aggregation, which greatly reduces the search domain, hardware complexity and off-chip storage access of the combination operation, eliminates the combination operation bottleneck in hardware acceleration, and improves hardware utilization.

[0042] (2) The point cloud analysis method based on data reuse and calculation skipping proposed in the present invention adopts a low hardware complexity prior calculation skipping based on sampling, which can skip the retained points that have been calculated in the sampling layer, and skip the neighborhood search with the discarded point as the center point or the neighboring point, the convolution operation and aggregation operation related to the discarded point, which greatly reduces the amount of calculation without affecting the calculation results.

[0043] (3) The point cloud analysis method based on data reuse and calculation skipping proposed in the present invention reuses the calculation results of the associated parts between the two adjacent input point cloud data to be analyzed, which can effectively reduce the amount of calculation, thereby greatly improving the energy efficiency of the point cloud analysis model and reducing the inference delay. The mask obtained by the sampling layer filters the index sequence of the previous level, reducing the calculation operations and storage access of the neighborhood search of the combination operation, and further dynamically pruning the points with smaller values ​​in the aggregated features through the pruning layer, so that the point cloud analysis model can achieve the desired balance between accuracy and hardware energy efficiency and delay according to the actual application scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 The calculation process of the point cloud neural network in the prior art;

[0046] Figure 2 A schematic flow chart of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a block-level pipelined delayed aggregation (BPDA) calculation process of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0048] Figure 4 A schematic diagram of a pipeline for processing multiple point cloud blocks using convolution operations, combination operations, and aggregation operations of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0049] Figure 5 Schematic diagram of a process of prior calculation skipping based on farthest point sampling in a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0050] Figure 6A schematic diagram of a specific implementation example of a priori calculation skipping based on farthest point sampling in a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0051] Figure 7 A schematic diagram of reusing calculation results of associated parts in a single calculation unit of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0052] Figure 8 A schematic diagram of utilizing calculation results of associated parts in a complete calculation process of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0053] Figure 9 A schematic diagram of a combined inter-layer multiplexing of neighboring point indexes in a point cloud analysis method based on data multiplexing and calculation skipping according to an embodiment of the present application;

[0054] Figure 10 A schematic diagram of a point cloud analysis method based on data reuse and calculation skipping in an embodiment of the present application, in which a combination operation at a next level reuses neighboring point indexes of a combination operation at a previous level during the entire calculation process;

[0055] Figure 11 Schematic diagram showing a comparison before and after dynamic point pruning of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0056] Figure 12 A flowchart of a dynamic point pruning method of a point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0057] Figure 13 Schematic diagram of the position of the pruning layer in the entire point cloud analysis model of the point cloud analysis method based on data reuse and calculation skipping according to an embodiment of the present application;

[0058] Figure 14 Schematic diagram of a point cloud analysis device based on data reuse and calculation skipping according to an embodiment of the present application;

[0059] Figure 15 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0061] Figure 2 A point cloud analysis method based on data reuse and calculation skipping provided by an embodiment of the present application is shown, and is characterized by comprising the following steps:

[0062] S1, obtain the point cloud data to be analyzed.

[0063] Specifically, the point cloud data to be analyzed is first obtained. The point cloud data to be analyzed input in the reasoning task is continuous point cloud data. In applications such as autonomous driving, drones, AR and VR, there is a strong correlation between the continuously input three-dimensional point clouds. There is a correlation between the currently input point cloud data to be analyzed and the next input point cloud data to be analyzed, that is, part of the content of the currently input point cloud data to be analyzed also exists in the next input point cloud data to be analyzed. There are also new point clouds in the next input point cloud data to be analyzed. The point cloud data to be analyzed contains the coordinates and features of each point, and the features include information such as color.

[0064] S2, constructs and trains a point cloud analysis model based on data reuse and calculation skipping to obtain a trained point cloud analysis model. The point cloud neural network includes several layers, each layer includes a sampling layer, a combination layer, a multi-layer perceptron unit, an aggregation layer and a pruning layer, and the multi-layer perceptron unit includes multiple multi-layer perceptrons connected in sequence.

[0065] Specifically, the number of multi-layer perceptrons in the multi-layer perceptron unit is M, and the multi-layer perceptron includes a convolution layer with a convolution kernel size of 1×1.

[0066] S3, input the point cloud data to be analyzed into the trained point cloud analysis model, input the point cloud data to be analyzed into the first level of the trained point cloud analysis model, output the sampling results through the sampling layer of the first level, determine the discarded points and retained points of the next level of sampling layer based on the sampling results of the current level of sampling layer, establish the index and coordinates of the retained points of each level of sampling layer in sequence and output them as the sampling results in combination with the features of the retained points, reuse the sampling results of the current level of sampling layer in the sampling process of the next level of sampling layer, and discard the points of the current level of sampling layer. The coordinates and features of the retained points in the sampling results of the current level are reused between levels based on the block-level pipeline delayed aggregation method in the combination layer, multi-layer perceptron unit and aggregation layer of the current level, and the combination operation, convolution operation and aggregation operation of the discarded points of the current level in the previous level are skipped to obtain the aggregation features of the current level. The aggregation features of the current level are subjected to dynamic point pruning operation in the pruning layer to obtain the point cloud data output by the current level. The point cloud data output by the current level is used as the point cloud data input to the next level, and the point cloud data output by the last level is used as the analysis result.

[0067] In a specific embodiment, the coordinates and features of the retained points in the sampling results of the current level are reused in the combination layer, multi-layer perceptron unit, and aggregation layer of the current level using an inter-level data multiplexing method based on block-level pipeline delay aggregation, specifically including:

[0068] The coordinates and features of the retained points in the sampling results of the current level are divided into multiple point cloud blocks, and the following pipeline process is repeated for different point cloud blocks at different time steps:

[0069] The coordinates of the retained points in the point cloud block are input into the combination layer for combination operation to obtain the neighbor index table;

[0070] The features of the retained points in the point cloud block are input into the multi-layer perceptron unit for convolution operation to obtain the output feature map;

[0071] The neighbor index table and the output feature map are input into the aggregation layer for aggregation operation to obtain the aggregated features of the point cloud block;

[0072] The aggregate features of all point cloud blocks constitute the aggregate features of the current level.

[0073] In a specific embodiment, in the combination operation, each reserved point of the current level is used as a center point, and a neighborhood search process is used to search for neighboring points of each center point in the point cloud data input by the sampling layer of the current level, and a mapping relationship between all reserved points and their neighboring points in the point cloud data input by the sampling layer of the current level is established to obtain a neighbor index table;

[0074] In the aggregation operation, the index of the neighboring point in the neighbor index table is used as the address, and the feature maps corresponding to the K neighboring points of the center point are obtained in the output feature map and aggregated to obtain the aggregated features of the point cloud block.

[0075] In a specific embodiment, a mask is constructed based on the sampling results of the sampling layer at the current level, and the retained points in the mask are marked as 1 and the discarded points are marked as 0;

[0076] Before the combination operation of the first level combination layer, the distance set between all points in the point cloud data to be analyzed is calculated and sorted from small to large to form an index sequence of points with distances from the center point from small to large;

[0077] The mask obtained according to the sampling result of the sampling layer of the current level selectively filters out the index sequence of the current level in the index sequence of the previous level, and the distance in the index sequence of the current level is used in the neighborhood search process in the combination operation of the current level.

[0078] In a specific embodiment, it also includes:

[0079] Determining, in the currently input point cloud data to be analyzed, newly added point clouds relative to the last input point cloud data to be analyzed, and determining, in the currently input point cloud data to be analyzed, associated portions corresponding to the last input point cloud data to be analyzed;

[0080] In the current level, the calculation results of the newly added point cloud at the sampling layer, combination layer, multi-layer perceptron unit, and aggregation layer are calculated respectively, and combined with the calculation results of the associated parts of the last input point cloud data to be analyzed at the sampling layer, combination layer, multi-layer perceptron unit, and aggregation layer respectively, to obtain the aggregation features of the currently input point cloud data to be analyzed at the current level, and then pass through the pruning layer to obtain the point cloud data output at the current level.

[0081] In a specific embodiment, the following process is performed in the pruning layer:

[0082] S41, sorting the aggregated features of the current level according to their numerical values, and determining the threshold of the current level;

[0083] S42, setting the values ​​of the aggregated features of the current level that are lower than the threshold of the current level to zero;

[0084] S43, determine whether the threshold of the current level meets the requirements. If so, obtain the point cloud data output by the current level. Otherwise, adjust the threshold of the current level and repeat steps S42-S43 until the threshold of the current level meets the requirements.

[0085] Specifically, the point cloud analysis model constructed in the embodiments of the present application is improved on the basis of the point cloud neural network mentioned in the background technology, and the improved parts are divided into the following points.

[0086] 1. Inter-level data reuse based on block-level pipeline delay aggregation

[0087] Compared with two-dimensional convolution, the activation reuse of the multi-layer perceptron (MLP) of the point cloud neural network has only one dimension, and the activation reuse rate is low, resulting in greater bandwidth requirements. When the bandwidth is fixed, the utilization rate of the computing unit will be significantly reduced. Therefore, the point cloud analysis model proposed in the embodiment of the present application adopts delayed aggregation technology. The core idea of ​​delayed aggregation is to change the order of convolution operations and combination operations, and set the combination layer in each level of the point cloud neural network before the multi-layer perceptron unit instead of running the combination layer and the multi-layer perceptron unit in each level in parallel, so as to first calculate the convolution results of a single point and the combination results of each point separately, and then use the combination results to aggregate the convolution results, instead of the native algorithm's calculation process of first combining and then convolution.

[0088] The embodiment of the present application proposes a method for data reuse between layers based on block-wise pipeline delayed aggregation (BPDA). Figure 3 As shown, the following steps are included:

[0089] In step (1), the input data of each level includes the coordinates and features of the point cloud output by the previous level. The coordinates and features of the input point cloud of the current level are first divided into multiple point cloud blocks, and then the combination operation is performed within the point cloud block, that is, the neighboring point search is performed within the point cloud block to obtain the neighbor index table (NIT). Due to the delayed aggregation, the features of the points are first subjected to the convolution operation of the M layer, which increases the channel of the feature by 2. M times.

[0090] In step (2), after performing convolution calculation on an input point cloud block in the multi-layer perceptron unit of the current level, the output feature map is obtained and stored.

[0091] In step (3), the NIT pre-calculated by the combination layer is used in the aggregation layer. The index of the neighboring point in the NIT is used as the address of the output feature map. The feature map of the K neighboring points of the center point is read from the output feature map. Then, the features of the neighborhood are aggregated to obtain the aggregated features of each point cloud block. The aggregated features of all point cloud blocks are combined into the aggregated features of the current level.

[0092] In the original point cloud neural network, each partial sum output by the multi-layer perceptron (MLP) is only used once by the aggregation operation (for example, maximum pooling) to generate the features of the corresponding output point according to the mapping relationship. In the BPDA proposed in the embodiment of the present application, the output results of the multi-layer perceptron are used by multiple aggregation operations. Therefore, BPDA increases the number of reuses of the convolution results to K', reduces the amount of calculation to about 1 / K' of the original, and shortens the calculation time of the multi-layer perceptron unit. At the same time, BPDA performs block convolution on the input point cloud, which greatly reduces the search domain, hardware complexity and off-chip storage access of the combination operation. Theoretically, when the number of point cloud blocks is B, the BPDA's blocking method reduces the search domain of the combination operation and the amount of off-chip storage access data to 1 / B of the original, eliminating the bottleneck of the combination operation in hardware acceleration. As Figure 4 As shown in the figure, the convolution operations, combination operations, and aggregation operations of multiple point cloud blocks within the layer are executed on the hardware in a pipeline manner. The convolution operations, combination operations, and aggregation operations of each point cloud block are executed sequentially in the multi-layer perceptron unit, combination layer, and aggregation layer. The point cloud blocks are fed into these three layers at equal time intervals, thereby reducing the idle cycles of the convolution layer, combination layer, and aggregation layer and improving hardware utilization.

[0093] 2. Low hardware complexity prior calculation skipping based on sampling

[0094] The embodiment of the present application also proposes a sampling-based skipping method (Sampling-Based Skipping, SBS), such as Figure 5 As shown, there are 4 types of calculation skipping:

[0095] ① is to skip the calculated reserved points (RP) in the sampling operation;

[0096] ② is sparse skipping of discarded points (DP), skipping the group operation with the discarded point (DP) as the center point and skipping the search process with the discarded point (DP) as the neighboring point;

[0097] ③ is to skip the convolution operation related to the discarded point (DP);

[0098] ④ is to skip the aggregation operation related to the discard point (DP).

[0099] For ①, if Figure 5 As shown in the figure, by fixing the initial point of the farthest point sample (FPS) at each level to the same, for example, p1, because the coordinates of the points remain unchanged in all sampling layers of the point cloud neural network, only the number of points is reduced. Therefore, after fixing the initial point to the same, the output order of the sampling operation from the first point p1 to the subsequent sampling points p2, p3, p4, etc. at each sampling layer remains unchanged. The first sampling layer has a complete point cloud and will first output the sampling results of the last sampling layer (such as p1-p4 in the Lth layer in the figure), then output the sampling results of the second-to-last sampling layer (such as p1-p6 in the (L-1)th layer in the figure), until the sampling results of the first sampling layer are output, that is, the retained points (RP) of the output point cloud. Note that the sampling of the (L-1)th layer reuses the sampling results of p1-p4 already calculated in the Lth layer, that is, the subsequent sampling layers reuse the sampling results of the sampling layer of the previous layer. Overall, only the sampling results of the first sampling layer are calculated, and the sampling results of the subsequent sampling layers are output simultaneously.

[0100] Since some points will be discarded in the sampling layer of the next level in the point cloud neural network, the combination operation, aggregation operation and convolution operation of these points in the previous level can be skipped in advance. The BPDA proposed in Improvement Point 1 decouples mapping construction and feature extraction, so that the mapping relationship can be established in advance before feature extraction. Therefore, the input points and output points of the mapping operation, including the input points and output points of the sampling operation and the combination operation, can be calculated in advance. By first performing the sampling operation of the next layer, the discarded points (DP) of the next sampling layer are obtained, and then the combination operation, aggregation operation and convolution operation of the discarded points (DP) in the previous level are skipped. As Figure 5As shown in the figure, in the point set P, the result of the farthest point sampling (FPS) of the next level can be used to skip the neighborhood search operation of the discarded point (DP) of the current level, including two skipping cases: taking the discarded point (DP) as the center point and taking the discarded point (DP) as the neighboring point (② skip combination in the figure) and the aggregation operation guided by the combination operation (④ skip aggregation in the figure). In addition, since the convolution kernel size in the multi-layer perceptron is 1×1, there is no correlation between points during convolution, and SBS directly skipping the convolution operation of some points will not affect the convolution calculation of other points. Therefore, as Figure 5 As shown in the figure, SBS can skip the convolution operation of discarded points (DP) (③ skip convolution in the figure). Generally, when the FPS sampling rate is 1 / 2, the SBS method can reduce 3 / 4 of the neighborhood search and aggregation operations and 1 / 2 of the convolution calculations by skipping 1 / 2 of the center points and 1 / 2 of the neighboring points without affecting the calculation results. When the data set is large, such as the S3DIS data set, the FPS sampling rate is often only 1 / 4. The SBS method can reduce 15 / 16 of the neighborhood search and aggregation operations, which is a significant improvement.

[0101] Figure 6 A specific implementation example of the SBS method is given, but SBS is not limited to this implementation. Figure 6 As shown, only the sampling layer of the first level is calculated and the results of the sampling layers of all levels are output in order. After the sampling layer calculates the FPS of the next level, the index of the reserved point (RP) (Valid Index) is stored in the storage module. When the combination layer calculates the mapping relationship between the input point and the output point of the current level, the index of the reserved point (RP) is read from the storage module into the index buffer. Figure 6 As shown, when the combination module performs combination calculation, the neighborhood search process with the discarded point (DP) as the center point and the discarded point (DP) as the neighboring point is skipped.

[0102] The combination layer passes the calculated mapping relationship to the aggregation layer to aggregate the neighborhood features of the center point, and the aggregation layer also skips the aggregation operation with the sampled discard point (DP) of the next level as the center point. Figure 6 Therefore, the input features of the current layer to the multi-layer perceptron unit of the next layer only retain the features of valid points such as P1 and P2. The features of other points such as P5 and P6 will neither participate in the calculation of the multi-layer perceptron unit of the next layer nor be stored on the chip.

[0103] 3. Inference-level data reuse based on spatial locality

[0104] like Figure 7 As shown, the input point cloud data to be analyzed of the point cloud analysis model is usually composed of the associated part of the last input point cloud data to be analyzed ( Figure 7Therefore, during reasoning, it is only necessary to calculate and update the new point cloud on the edge, and then combine it with the calculation results of the associated part corresponding to the point cloud data to be analyzed last time to obtain the calculation results of the current point cloud. In a specific embodiment, the calculation results of the new point cloud are spliced ​​with the calculation results of the associated part to obtain the calculation results of the currently input point cloud data to be analyzed. At the same time, the calculation results of the current point cloud are stored in the storage unit to prepare for the reuse of the calculation results of the associated part in the calculation process of the next input point cloud data to be analyzed. Figure 8 As shown in the figure, during the entire calculation process of the point cloud analysis model, the calculation operations of the current point cloud, such as sampling, combination, convolution, aggregation, etc., only need to calculate the updated part of the point cloud, and then combine it with the associated part of the last input point cloud to form a complete calculation result of the current point cloud for reasoning, and store it for reuse when the next output point cloud is reasoned. Therefore, this method can directly reuse the calculation results of the associated part of the previous point cloud and the current input point cloud (such as mapping and convolution results), thereby achieving the purpose of reusing part of the results of the previous reasoning for the next reasoning. The stronger the correlation between the continuous input point clouds, the more the amount of calculation and data access storage can be reduced, thereby greatly improving the energy efficiency of the point cloud analysis model and reducing the reasoning delay.

[0105] 4. Inter-layer data reuse of search distance

[0106] The neighborhood search of the combined operation in the point cloud neural network needs to traverse the coordinates of all points multiple times and calculate the Euclidean distance between points. Therefore, the neighborhood search of the combined operation introduces a large number of computing operations and storage accesses. Figure 9 As shown, the embodiment of the present application pre-constructs a mask (Mask) corresponding to the input point cloud of each level of the combination layer in the sampling layer of the point cloud analysis model, that is, which points are retained points that need to be retained, and which points are discarded points that need to be discarded. In the mask, 1 represents a retained point, and 0 represents a discarded point, which will not participate in the neighboring point search of the next combination layer of the sampling layer. Therefore, the input point cloud of the combination layer of the next level is a subset of the input point cloud of the combination layer of the current level, and the calculation operation of the combination layer of the next level can directly reuse the distance set between the points calculated by the neighboring search of the combination layer of the current level. Moreover, the combination layer of the first level has previously calculated a complete set of distances between all points and sorted them from small to large, forming an index sequence of points with distances from the center point from small to large, such as Figure 9As shown. The combination layer of the subsequent layers only needs to selectively filter the index sequence according to the sampling result, that is, the mask output by the sampling layer, and then take the point index corresponding to the K minimum distances and output the mapping relationship of the combination operation. There is no need to recalculate the distance between points in the combination layer of the second layer. The combination layer of the subsequent layers is similar, and they only need to filter out the index sequence from the previous layer without calculating the distance between points. Figure 10 As shown, during the entire calculation process of the point cloud analysis model, the combined operation results of level 2 ( Figure 10 The green arrow in the figure is the mask of the sampling operation of level 2 ( Figure 10 The purple arrow in the figure filters the index sequence of level 1 ( Figure 10 Finally, the distance relationship is calculated once and reused across all combination layers, avoiding repeated distance calculations at each combination layer.

[0107] 5. Dynamic point pruning based on eigenvalue importance ranking

[0108] In terms of computational skipping, each point in the point cloud space has different importance for network reasoning. For example, points on the edge contour have stronger representation capabilities than points in the middle of an object, so points on the edge contour are more important. Figure 11 As shown in the figure, the blue points are larger in value, located at the outline of the point, with strong representation ability, and play a key role in feature extraction, while the light orange points are small in value, located inside the point cloud, with weak representation ability, and have limited effect on feature extraction. Figure 12 As shown in the figure, the numerical values ​​of all points in the aggregated features of each level in the statistical data set are counted, and the threshold of each level is set to prune these unimportant points with small values. Therefore, the convolution calculation of these pruned points is skipped in the calculation of the next level. At the same time, the size of the threshold is adjusted to achieve the desired balance between accuracy, hardware energy efficiency and latency according to the actual application scenario. Figure 13 As shown, in the entire calculation process of the point cloud neural network, the pruning operation ( Figure 13 The red box in the figure prunes the aggregated features of the previous level to obtain features with reduced size and input them to the next level.

[0109] Further references Figure 14 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a point cloud analysis device based on data reuse and calculation skipping. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0110] The embodiment of the present application provides a point cloud analysis device based on data reuse and calculation skipping, comprising:

[0111] A data acquisition module 1 is configured to acquire point cloud data to be analyzed;

[0112] a model building module 2 configured to build and train a point cloud analysis model based on data reuse and computation skipping to obtain a trained point cloud analysis model, wherein the point cloud neural network includes several layers, each layer including a sampling layer, a combination layer, a multi-layer perceptron unit, an aggregation layer, and a pruning layer, wherein the multi-layer perceptron unit includes a plurality of multi-layer perceptrons connected in sequence;

[0113] The analysis module 3 is configured to input the point cloud data to be analyzed into the trained point cloud analysis model, input the point cloud data to be analyzed into the first level of the trained point cloud analysis model, output the sampling results through the sampling layer of the first level, determine the discarded points and retained points of the next level of sampling layer based on the sampling results of the current level of sampling layer, establish the index and coordinates of the retained points of each level of sampling layer in sequence and output them as the sampling results in combination with the features of the retained points, reuse the sampling results of the current level of sampling layer in the sampling process of the next level of sampling layer, and The coordinates and features of the retained points in the sampling results of the current level adopt an inter-level data reuse method based on block-level pipeline delayed aggregation in the combination layer, multi-layer perceptron unit and aggregation layer of the current level, and skip the combination operation, convolution operation and aggregation operation of the discarded points of the current level in the previous level to obtain the aggregated features of the current level. The aggregated features of the current level are subjected to dynamic point pruning operation in the pruning layer to obtain the point cloud data output by the current level. The point cloud data output by the current level is used as the point cloud data input to the next level, and the point cloud data output by the last level is used as the analysis result.

[0114] Figure 15 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 15 As shown, the electronic device of this embodiment includes: a processor 1501 and a memory 1502; wherein the memory 1502 is configured to store computer-executable instructions; and the processor 1501 is configured to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above-described embodiment. For details, please refer to the relevant description of the aforementioned method embodiment.

[0115] Optionally, the memory 1502 may be independent or integrated with the processor 1501 .

[0116] When the memory 1502 is independently provided, the electronic device further includes a bus 1503 for connecting the memory 1502 and the processor 1501 .

[0117] An embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 1501 executes the computer execution instructions, the above method is implemented.

[0118] An embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 1501, the above method is implemented.

[0119] In the embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, which may be electrical, mechanical or other forms.

[0120] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these modules may be selected to implement the solution of this embodiment based on actual needs.

[0121] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each module may exist physically separately, or two or more modules may be integrated into a single unit. The units formed by the above modules may be implemented in the form of hardware or hardware plus software functional units.

[0122] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or processor 1501 to perform some steps of the methods of various embodiments of the present application.

[0123] It should be understood that the processor 1501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASIC). A general-purpose processor may be a microprocessor, or the processor 1501 may be any conventional processor 1501. The steps of the method disclosed in the present invention may be directly implemented by the hardware processor 1501, or implemented by a combination of hardware and software modules in the processor 1501.

[0124] The memory 1502 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0125] Bus 1503 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Bus 1503 can be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, the bus 1503 in the drawings of this application is not limited to a single bus 1503 or a single type of bus 1503.

[0126] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0127] An exemplary storage medium is coupled to the processor 1501, so that the processor 1501 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor 1501. The processor 1501 and the storage medium can be located in an application-specific integrated circuit (ASIC). Of course, the processor 1501 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0128] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A point cloud analysis method based on data reuse and computation skipping, characterized in that: The following steps are involved: Obtain point cloud data to be analyzed; A point cloud analysis model based on data reuse and computation skipping is constructed and trained to obtain a trained point cloud analysis model, wherein the point cloud neural network includes several layers, each layer including a sampling layer, a combination layer, a multi-layer perceptron unit, an aggregation layer, and a pruning layer, wherein the multi-layer perceptron unit includes a plurality of multi-layer perceptrons connected in sequence; The point cloud data to be analyzed is input into the trained point cloud analysis model, and the point cloud data to be analyzed is input into the first level of the trained point cloud analysis model. The sampling result is output through the sampling layer of the first level. The discarded points and retained points of the sampling layer of the next level are determined based on the sampling result of the sampling layer of the current level. The index and coordinates of the retained points of the sampling layer of each level are established in sequence and output as the sampling result in combination with the features of the retained points. The sampling result of the sampling layer of the current level is reused in the sampling process of the sampling layer of the next level. The coordinates and features of the retained points in the sampling results of the layer adopt an inter-layer data reuse method based on block-level pipeline delayed aggregation in the combination layer, multi-layer perceptron unit and the aggregation layer of the current layer, and skip the combination operation, convolution operation and aggregation operation of the discarded points of the current layer in the previous layer to obtain the aggregation features of the current layer. The aggregation features of the current layer are subjected to dynamic point pruning operation by the pruning layer to obtain the point cloud data output by the current layer. The point cloud data output by the current layer is used as the point cloud data input to the next layer, and the point cloud data output by the last layer is used as the analysis result.

2. The point cloud analysis method based on data reuse and calculation skipping according to claim 1, characterized in that: The coordinates and features of the retained points in the sampling results of the current level are reused in the combination layer, the multi-layer perceptron unit and the aggregation layer of the current level using an inter-level data multiplexing method based on block-level pipeline delay aggregation, specifically including: The coordinates and features of the retained points in the sampling results of the current level are divided into multiple point cloud blocks, and the following pipeline process is repeated for different point cloud blocks at different time steps: Input the coordinates of the retained points in the point cloud block into the combination layer for combination operation to obtain a neighbor index table; Inputting the features of the retained points in the point cloud block into the multi-layer perceptron unit for convolution operation to obtain an output feature map; Inputting the neighbor index table and the output feature map into the aggregation layer for aggregation operation to obtain the aggregation features of the point cloud block; The aggregate features of all point cloud blocks constitute the aggregate features of the current level.

3. The point cloud analysis method based on data reuse and calculation skipping according to claim 2, characterized in that: In the combination operation, each reserved point of the current level is used as a center point, and a neighborhood search process is used to search for neighboring points of each center point in the point cloud data input by the sampling layer of the current level, and a mapping relationship between all reserved points and their neighboring points in the point cloud data input by the sampling layer of the current level is established to obtain the neighbor index table; In the aggregation operation, the index of the neighboring point in the neighbor index table is used as the address, and the feature maps corresponding to the K neighboring points of the center point are obtained and aggregated in the output feature map to obtain the aggregated features of the point cloud block.

4. The point cloud analysis method based on data reuse and calculation skipping according to claim 3 is characterized in that: Construct a mask based on the sampling results of the current level of sampling layer, where the retained points in the mask are marked as 1 and the discarded points are marked as 0; Before the combination operation of the first level combination layer, a distance set between all points in the point cloud data to be analyzed is calculated and sorted from small to large to form an index sequence of points with distances from the center point from small to large; The mask obtained according to the sampling result of the sampling layer of the current level selectively filters out the index sequence of the current level in the index sequence of the previous level, and the distance in the index sequence of the current level is used in the neighborhood search process in the combination operation of the current level.

5. The point cloud analysis method based on data reuse and calculation skipping according to claim 1, characterized in that: Also includes: Determining, in the currently input point cloud data to be analyzed, newly added point clouds relative to the last input point cloud data to be analyzed, and determining, in the currently input point cloud data to be analyzed, associated portions corresponding to the last input point cloud data to be analyzed; In the current level, the calculation results of the newly added point cloud at the sampling layer, combination layer, multi-layer perceptron unit, and aggregation layer are calculated respectively, and combined with the calculation results of the associated parts of the last input point cloud data to be analyzed at the sampling layer, combination layer, multi-layer perceptron unit, and aggregation layer respectively, to obtain the aggregation features of the currently input point cloud data to be analyzed at the current level, and then pass through the pruning layer to obtain the point cloud data output at the current level.

6. The point cloud analysis method based on data reuse and calculation skipping according to claim 1, characterized in that: The following process is performed in the pruning layer: S41, sorting the aggregated features of the current level according to their numerical values, and determining the threshold of the current level; S42, setting the values ​​of the aggregated features of the current level that are lower than the threshold of the current level to zero; S43, determine whether the threshold of the current level meets the requirements, if so, obtain the point cloud data output by the current level, otherwise adjust the threshold of the current level, and repeat steps S42-S43 until the threshold of the current level meets the requirements.

7. A point cloud analysis device based on data reuse and calculation skipping, characterized in that: include: a data acquisition module configured to acquire point cloud data to be analyzed; a model building module configured to build and train a point cloud analysis model based on data reuse and computation skipping to obtain a trained point cloud analysis model, wherein the point cloud neural network includes several layers, each layer including a sampling layer, a combination layer, a multi-layer perceptron unit, an aggregation layer, and a pruning layer, wherein the multi-layer perceptron unit includes a plurality of multi-layer perceptrons connected in sequence; The analysis module is configured to input the point cloud data to be analyzed into the trained point cloud analysis model, input the point cloud data to be analyzed into the first level of the trained point cloud analysis model, output the sampling result through the sampling layer of the first level, determine the discarded points and retained points of the sampling layer of the next level based on the sampling result of the sampling layer of the current level, establish the index and coordinates of the retained points of the sampling layer of each level in sequence and output them as the sampling result in combination with the features of the retained points, and reuse the sampling result of the sampling layer of the current level in the sampling process of the sampling layer of the next level. As a result, the coordinates and features of the retained points in the sampling results of the current level are reused in the combination layer, multi-layer perceptron unit and the aggregation layer of the current level by an inter-level data reuse method based on block-level pipeline delayed aggregation, and the combination operation, convolution operation and aggregation operation of the discarded points of the current level in the previous level are skipped to obtain the aggregation features of the current level. The aggregation features of the current level are subjected to dynamic point pruning operation by the pruning layer to obtain the point cloud data output by the current level. The point cloud data output by the current level is used as the point cloud data input by the next level, and the point cloud data output by the last level is used as the analysis result.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images