Low-energy-consumption point cloud processing method and system

Abstract

The invention discloses a low-energy-consumption point cloud processing method. The method comprises the steps that original point cloud data are collected and preprocessed; processing the original point cloud data in a layering manner, generating a multi-level resolution point set, and establishing an interlayer mapping relationship; according to the multi-level resolution point set, a global graph is constructed in a low-resolution layer in the multi-level resolution point set, and a local graph is constructed in the region of interest; dynamic updating is carried out by adopting a gradual updating mechanism based on a characteristic change rate; a low-bit quantization technology is adopted, low-bit quantization features are generated, and event-driven feature aggregation is carried out in combination with a spiking neuron model; and outputting a segmentation result marked by the point cloud target based on a composite optimization strategy. According to the low-power-consumption point cloud processing method oriented to the embedded and edge computing devices, a quantized pulse graph convolutional network and a multi-resolution progressive updating mechanism are comprehensively applied.

Description

Technical Field

[0001] This invention relates to artificial intelligence, advanced manufacturing, and automation, specifically to a low-power point cloud processing method for embedded and edge computing devices. This method integrates quantized pulse graph convolutional networks and a multi-resolution progressive update mechanism, enabling high-precision, low-power, and real-time 3D point cloud segmentation and understanding tasks in key national development areas such as autonomous driving, industrial robots, intelligent security, and 3D reconstruction. Background Technology

[0002] With the rapid development of computer vision, intelligent sensors, edge computing and other fields, point cloud data, as a high-precision form of three-dimensional spatial information representation, has become an important foundation for target detection, scene understanding and object reconstruction.

[0003] Currently, in typical applications using neural networks such as autonomous driving, industrial inspection, and robot perception, the general processing flow of point cloud data from acquisition to final decision / output can be summarized into the following six steps:

[0004] 1. Data Collection

[0005] The system acquires raw point cloud frame sequences using a 3D LiDAR, structured light depth camera, ToF depth sensor, or binocular / multi-view vision system. It outputs raw 3D coordinates and additional attributes such as reflection intensity, RGB values, and timestamps.

[0006] 2. Pretreatment

[0007] Outliers are removed using statistical filtering and radius filtering, followed by spherical or cylindrical normalization. Finally, lightweight downsampling is achieved through voxel raster filtering or farthest-point sampling. The output is a clean, uniform coordinate system dense or sparse point cloud.

[0008] 3. Feature / Structured Representation

[0009] Convert point clouds into structured representations that can be efficiently processed by networks, outputting graphs, point sets, or voxel meshes.

[0010] 4. Deep Feature Extraction and Inference

[0011] This application uses neural networks to extract high-dimensional semantic features, enabling tasks such as object detection, semantic segmentation, instance segmentation, or registration. It outputs class probabilities or pose estimates for each point or each detection box.

[0012] 5. Post-processing and fusion

[0013] The network output is subjected to NMS, DBSCAN clustering, and multi-frame temporal fusion to improve robustness; point cloud results are fused with 2D images or other information features when necessary. Output 3D bounding boxes with IDs, semantic labels, or reconstructed meshes.

[0014] 6. Decision / Visualization Output

[0015] Decision-making instructions are generated based on specific applications, or the segmentation results are rendered to a visualization interface in real time. The final instructions or visualization results are output for use by upper-level systems or human-computer interaction.

[0016] In step 3 of the general processing flow, there are several common processing methods. Common techniques include:

[0017] 1. Graph Convolution Method Based on Neighborhood Aggregation

[0018] Methods such as Point-GCN, DGCNN, and EdgeConv can adapt to irregular structures by constructing K-nearest neighbor graphs or radius graphs to aggregate neighbor features, but their computational complexity increases quadratically with the number of nodes, resulting in a huge computational load under high-resolution point clouds.

[0019] 2. Hierarchical Networks Based on Multi-Scale Feature Extraction

[0020] Networks such as PointNet++ and multi-scale graph convolutional networks can take into account both global and local information through multi-level sampling and feature fusion, but they require multiple neighborhood searches, resulting in high computational overhead.

[0021] 3. Graph Neural Networks Based on Dynamic Neighborhood Update

[0022] For example, DGCNN (Dynamic Graph CNN) and SP-GCN reconstruct the neighborhood in the feature space at each layer to enhance the feature representation ability, but frequent KNN searches increase inference latency and power consumption.

[0023] 4. Point cloud processing method based on sparse convolution

[0024] Methods like SparseConvNet and MinkowskiNet map point clouds to a sparse voxel space and compute convolutions only on non-empty voxels, reducing computation but risking loss of geometric accuracy.

[0025] 5. Graph or Transformer models based on attention mechanisms

[0026] Applications such as Point Transformer and GAT for point clouds can capture long-range dependencies by dynamically adjusting the neighborhood aggregation contribution through attention weights, but the attention computation complexity is high and the memory consumption is large.

[0027] However, point cloud data is typically massive and sparse, and processing it directly at full resolution leads to enormous computational costs, high storage overhead, and significantly increased energy consumption, especially on embedded and edge computing devices, where real-time processing is extremely challenging. Although the methods mentioned above have achieved good results in terms of accuracy and feature extraction capabilities, they still have significant shortcomings in high-resolution point cloud processing, low-power inference, and real-time performance.

[0028] Taking L4 autonomous driving real-time environmental perception as a typical scenario: the vehicle is equipped with a 128-line LiDAR, with a single frame point cloud scale of 2.3 million points and a frame rate of 10 FPS; the point cloud segmentation task is run on the automotive embedded platform NVIDIA Jetson AGX Orin (with a total system power budget of less than 30 W), requiring an end-to-end latency of ≤100 ms, corresponding to a single frame energy consumption of ≤3J, and a processing rate of ≥23M points / s. Especially in embedded and edge computing environments, global neighborhood construction, dynamic neighborhood updates, or global attention mechanisms will bring huge computational loads and energy consumption, making it difficult to meet the requirements of real-time processing.

[0029] In terms of model compression and low-power inference, quantization techniques can reduce computational load and storage bandwidth by lowering computational precision, but simple quantization is insufficient for utilizing highly sparse event data. Spiking Neural Networks (SNNs), as event-driven low-power neural computing models, can utilize sparse pulse signals to avoid redundant computation and naturally possess low-power characteristics.

[0030] In terms of computing resource allocation, multi-resolution point cloud processing technology can effectively reduce the scale of one-time computation through layer-by-layer downsampling and feature transfer. However, most existing multi-resolution schemes adopt fixed global graph construction and unified update strategies, without performing progressive local topology updates based on the degree of feature change in different regions. This leads to repeated computation in static regions and untimely response in dynamic regions, limiting the potential for computational efficiency and energy consumption optimization.

[0031] In summary, existing technologies have the following problems in low-energy point cloud processing:

[0032] 1. The computational cost and energy consumption of full-image GNN inference under high-resolution point clouds are too high, making it difficult to meet real-time requirements.

[0033] 2. Quantization and event-driven reasoning techniques have not yet been deeply integrated into graph convolutional structures, and the advantages of sparse computing have not been fully utilized.

[0034] 3. Multi-resolution methods lack a progressive topology update mechanism for dynamic regions, resulting in low utilization of computational resources.

[0035] 4. Existing methods are not adequately optimized for deployment on embedded and edge devices, and cannot balance the requirements for accuracy, latency, and power consumption.

[0036] Therefore, there is an urgent need for a low-energy processing method that combines quantization with pulse graph convolution and integrates multi-resolution point cloud input with progressive graph update strategy, so as to significantly reduce computational overhead and energy consumption while maintaining high accuracy and improving real-time processing capabilities on resource-constrained platforms. Summary of the Invention

[0037] To address the problems of high computational cost, high energy consumption, insufficient integration of quantization and event-driven inference, and lack of progressive update mechanisms in multi-resolution point cloud processing in existing technologies, this invention provides a low-energy-consumption point cloud processing method based on quantized pulse graph convolution and multi-resolution progressive updates. This method achieves high-precision, low-power, real-time point cloud processing capabilities on embedded and edge computing platforms through collaborative optimization of multi-resolution point cloud representation, sparse graph construction, low-bit quantization, and spiking neuron inference.

[0038] In a first aspect, embodiments of this application provide a low-power point cloud processing method, which is applied to embedded and edge computing platforms, and includes:

[0039] Point cloud data acquisition and preprocessing steps: Acquire raw point cloud data and perform preprocessing;

[0040] Multi-resolution point cloud construction steps: process the original point cloud data in layers, generate multi-level resolution point sets and establish inter-layer mapping relationships;

[0041] Graph construction and progressive update steps: Based on a multi-level resolution point set, a global graph is constructed at the low-resolution layer, and a local graph is constructed in the region of interest; and a progressive update mechanism based on the feature change rate is used for dynamic updating.

[0042] Quantized pulse graph convolutional inference steps: Based on the constructed global and local graphs, low-bit quantization technology is used to generate low-bit quantized features, which are then combined with the spiking neuron model for event-driven feature aggregation;

[0043] Task output steps: Based on the aggregated features, output the segmentation results labeled with point cloud targets using a composite optimization strategy.

[0044] In this embodiment of the invention, the low-energy point cloud processing method further includes:

[0045] Model training steps: After completing the forward inference of quantized impulses, the graph neural network model is trained by backpropagation with propagation in both time and space domains.

[0046] In this embodiment of the invention, the above-mentioned point cloud data acquisition and preprocessing steps include:

[0047] The raw point cloud data of the target scene is acquired by the acquisition device; the i-th point cloud data Including: the first point cloud data The coordinates of a point in three-dimensional space, and the first Additional features of each point;

[0048] Point cloud data The point cloud coordinates are normalized based on the mean vector and standard deviation vector of the point cloud coordinates.

[0049] Point cloud data Statistical filtering is used for noise reduction, and preprocessed point cloud data is output.

[0050] In this embodiment of the invention, the above-mentioned multi-resolution point cloud construction step includes:

[0051] Multi-level sampling is performed on the normalized and denoised point cloud. A far-point sampling combined with a curvature-based interest point sampling strategy is used to generate multiple point sets with different resolutions. ,in, For a low-resolution point set, For medium to high resolution point sets, For high-resolution point sets, low-resolution point sets Preserving global structure, high-resolution point set Preserve local details.

[0052] In this embodiment of the invention, the above-mentioned graph construction and progressive update steps include:

[0053] In low-resolution point sets Construct a global sparse graph and edge weight matrix on top of it;

[0054] In medium-high resolution point sets In this process, local subgraphs are constructed only in regions of interest, which are determined by feature entropy or point motion estimation.

[0055] When the rate of change of node features in the graph exceeds a preset threshold, the reconstruction of the node neighborhood is triggered; otherwise, the neighborhood connection of the previous frame will be used.

[0056] In this embodiment of the invention, the above-mentioned quantized pulse graph convolutional inference step includes:

[0057] On the constructed graph structure, quantized pulse graph convolutional inference is used, and graph neural networks are used for feature aggregation.

[0058] The weight parameters of the graph are symmetrically quantized, the activation values ​​are asymmetrically quantized, and the model accuracy is maintained through quantization-aware training.

[0059] The quantized features are input into a spiking neural network. Using a neuron model and sparse pulse signals, calculations are performed when a valid event is received to generate event-driven feature aggregation.

[0060] In this embodiment of the invention, the above-mentioned task output step includes:

[0061] The task output is point cloud target label segmentation. The class label of each point is predicted. A composite loss function is used to construct a composite loss according to the target task. The composite loss function includes: weighted cross-entropy function, Dice loss function and impulse sparsity regularization loss function.

[0062] Calculate the total loss function value and construct a composite loss function according to the target task. The total loss function value is the sum of the weighted cross-entropy function, the Dice loss function, and the impulse sparsity regularization loss function.

[0063] Secondly, embodiments of this application provide a low-power point cloud processing system, employing the low-power point cloud processing method described above. The system is applied to embedded and edge computing platforms and includes:

[0064] Point cloud data acquisition and preprocessing module: Acquires raw point cloud data and performs preprocessing;

[0065] Multi-resolution point cloud construction module: Processes raw point cloud data in layers, generates multi-level resolution point sets, and establishes inter-layer mapping relationships;

[0066] The graph construction and progressive update module: Based on a multi-level resolution point set, it constructs a global graph at the low-resolution layer and a local graph in the region of interest; and uses a progressive update mechanism based on the feature change rate to perform dynamic updates.

[0067] Quantized pulse graph convolutional inference module: Based on the constructed global and local graphs, low-bit quantization technology is used to generate low-bit quantized features, which are then combined with the spiking neuron model for event-driven feature aggregation;

[0068] Task output module: Based on the aggregated features, outputs segmentation results labeled with point cloud targets using a composite optimization strategy.

[0069] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a low-power point cloud processing method.

[0070] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the low-power point cloud processing method as described above.

[0071] Compared with existing technologies, it has the following outstanding advantages:

[0072] 1) The method and system of this invention propose a multi-resolution progressive point cloud processing framework, which aims to solve the problem of huge computational load and energy consumption caused by the massive scale of high-resolution point cloud data. This invention constructs point sets of different resolutions from the original point cloud data through multi-level sampling. The low-resolution point set preserves the global structure of the point cloud, while the high-resolution point set preserves local details. This hierarchical processing method reduces the huge computational load generated by processing at full resolution, thereby balancing processing performance and energy efficiency.

[0073] Technical benefits: This framework processes point cloud data in layers, performing global analysis at low resolution and local optimization at high resolution, thereby reducing unnecessary computation and achieving the goal of effectively reducing energy consumption while ensuring performance.

[0074] 2) The method and system of this invention propose a graph construction and dynamic update mechanism. This invention employs different graph construction strategies on point sets of different resolutions. A global sparse graph is constructed on low-resolution point sets, while on medium- and high-resolution point sets, local subgraphs are constructed only within regions of interest determined by feature entropy or point motion estimation. Furthermore, this method introduces a progressive update mechanism. During inference, the reconstruction of a node's neighborhood is only triggered when the feature change rate of a node exceeds a preset threshold. Otherwise, the neighborhood connections from the previous frame are reused. This dynamic update mechanism avoids redundant calculations of static regions, reduces unnecessary reconstruction overhead, thereby improving processing speed and reducing power consumption.

[0075] Technical Effects: This invention avoids redundant computation of static regions by constructing a global sparse graph at a low-resolution layer and a region-of-interest subgraph at a high-resolution layer, combined with a dynamic update strategy based on feature change rate. This mechanism effectively reduces unnecessary reconstruction overhead, thereby improving processing speed and reducing power consumption.

[0076] 3) The method and system of this invention propose quantized graph convolution operations. To achieve efficient operation on embedded hardware, this method employs low-bit quantization technology. Specifically, the model weights are quantized using 8-bit symmetric quantization, and the activation function is quantized using 4-bit asymmetric quantization. Simultaneously, quantization-aware training (QAT) is incorporated during training to maintain model accuracy. This quantization strategy effectively reduces model storage space and computational bit width, thereby reducing computational load and storage bandwidth, making it particularly suitable for resource-constrained platforms.

[0077] Technical Effects: This invention employs low-bit quantization technology, performing 8-bit quantization on weights and 4-bit quantization on activations. This quantization strategy reduces computational bit width and storage bandwidth, enabling the model to run efficiently on embedded hardware with limited computing resources.

[0078] 4) The method and system of this invention propose spiking neuron event-driven reasoning. This invention deeply integrates spiking neural networks (SNNs) with graph convolutional structures to realize an event-driven reasoning mechanism. This invention uses a LIF (Leaky Integrate-and-Fire) neuron model to process quantized features. This mechanism utilizes sparse pulse signals, performing calculations only when a valid event is received, avoiding redundant computations, and thus naturally possessing low power consumption characteristics.

[0079] Technical Effects: This invention inputs quantized features into a spiking neural network and employs a LIF neuron model for event-driven feature aggregation. This method utilizes sparse pulse signals, avoids redundant computation, and achieves low-power inference.

[0080] 5) The method and system of this invention propose a composite loss function design. To improve the accuracy of point cloud segmentation while also considering low power consumption, this method designs a composite loss function. This loss function consists of weighted cross-entropy loss, Dice loss, and impulse sparsity regularization loss. The weighted cross-entropy loss is used to ensure the overall classification accuracy of the model. The Dice loss aims to improve the segmentation performance for sparse or imbalanced classes. The impulse sparsity regularization loss is used to encourage impulse sparsity, thereby reducing power consumption. This composite loss function integrates multiple objectives during the optimization process, which can improve the segmentation accuracy of the model while meeting the requirement of low power consumption.

[0081] Technical Effects: This invention employs a composite optimization strategy combining weighted cross-entropy, Dice loss, and impulse sparsity regularization. This design balances overall accuracy and the performance of sparse classes, while reducing power consumption through sparsity regularization, thereby improving segmentation accuracy.

[0082] 6) The method and system of this invention propose low-power embedded deployment optimization. This invention achieves synergistic optimization through key technologies such as multi-resolution progressive updates, low-bit quantization, and spiking neuron inference. These technologies work together to address the shortcomings of existing methods in deployment optimization on embedded and edge computing devices. This method can maintain high performance in resource-constrained environments while significantly reducing computational overhead and energy consumption, and improving real-time processing capabilities, thereby effectively solving the bottlenecks of latency and power consumption while maintaining high accuracy.

[0083] Technical Effects: This invention utilizes multi-resolution progressive updates and synergistic optimization of quantization and pulsed processing, specifically designed for deployment on embedded and edge computing platforms. This enables the method to maintain high performance in resource-constrained environments, achieving reduced power consumption and latency while maintaining low accuracy. Attached Figure Description

[0084] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0085] Figure 1 This is a schematic diagram of the low-energy point cloud processing method of the present invention;

[0086] Figure 2 This is a schematic diagram of a low-energy point cloud processing workflow based on quantized pulse map convolution and multi-resolution progressive update according to an embodiment of the present invention.

[0087] Figure 3 This is a schematic diagram of the multi-resolution progressive graph update mechanism according to an embodiment of the present invention;

[0088] Figure 4 This is a schematic diagram of the hidden layer calculation process of the quantized spiking neural network according to an embodiment of the present invention;

[0089] Figure 5 This is a schematic diagram of the training process of a quantized spiking neural network according to an embodiment of the present invention;

[0090] Figure 6 This is a schematic diagram of the low-energy point cloud processing system of the present invention;

[0091] Figure 7 This is a schematic diagram of the computer hardware of the present invention. Detailed Implementation

[0092] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0093] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0094] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0095] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0098] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0100] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0101] This invention aims to propose a low-energy processing method that combines quantization with pulse graph convolution and integrates multi-resolution point cloud input with a progressive graph update strategy. This method significantly reduces computational overhead and energy consumption while maintaining high accuracy, thereby improving real-time processing capabilities on resource-constrained platforms.

[0102] This invention provides a low-power point cloud processing method based on quantized pulse graph convolution and multi-resolution progressive updates. This method achieves high-precision, low-power, real-time point cloud processing capabilities on embedded and edge computing platforms through collaborative optimization of multi-resolution point cloud representation, sparse graph construction, low-bit quantization, and spiking neuron inference.

[0103] The system of this application embodiment will be described in detail below with reference to specific embodiments:

[0104] like Figure 1 and Figure 2 As shown, the present invention relates to a low-energy point cloud processing method based on quantized pulse map convolution and multi-resolution progressive update.

[0105] like Figure 1 As shown, this invention proposes a low-power point cloud processing method, which is applied to embedded and edge computing platforms. The method includes:

[0106] Point cloud data acquisition and preprocessing step 101: Acquire raw point cloud data and perform preprocessing;

[0107] Multi-resolution point cloud construction step 102: Process the original point cloud data in layers to generate multi-layer resolution point sets and establish inter-layer mapping relationships;

[0108] Graph construction and progressive update step 103: Based on the multi-level resolution point set, construct a global graph in the low-resolution layer and a local graph in the region of interest; and use a progressive update mechanism based on feature change rate to perform dynamic updates.

[0109] Quantization pulse graph convolutional inference step 104: Based on the constructed global and local graphs, low-bit quantization technology is used to generate low-bit quantized features, which are then combined with the spiking neuron model for event-driven feature aggregation.

[0110] Task output step 105: Based on the aggregated features, output the segmentation results labeled with point cloud targets using a composite optimization strategy.

[0111] like Figure 2 As shown, specifically, in a specific embodiment of the present invention, a low-energy point cloud processing method based on quantized pulse map convolution and multi-resolution progressive update includes:

[0112] S100, point cloud data acquisition and preprocessing: acquire raw point clouds and perform normalization and noise reduction processing.

[0113] S200, multi-resolution point cloud construction, generating multi-level resolution point sets and establishing inter-layer mapping relationships.

[0114] S300, graph construction and progressive updates: build a global graph at a low-resolution layer, and build and dynamically update a local graph in the region of interest.

[0115] S400, quantized pulse graph convolutional inference, combines low-bit quantization with spiking neurons for event-driven feature aggregation.

[0116] S500, task output and loss function design, based on a composite optimization strategy of cross-entropy, Dice loss and impulse sparsity regularization to output segmentation results.

[0117] In this embodiment of the invention, the low-energy point cloud processing method further includes:

[0118] Model training steps: After completing the forward inference of quantized impulses, the graph neural network model is trained by backpropagation with propagation in both time and space domains.

[0119] In this embodiment of the invention, the point cloud data acquisition and preprocessing step 101 includes:

[0120] The raw point cloud data of the target scene is acquired by the acquisition device; the i-th point cloud data Including: the first point cloud data The coordinates of a point in three-dimensional space, and the first Additional features of each point;

[0121] Point cloud data The point cloud coordinates are normalized based on the mean vector and standard deviation vector of the point cloud coordinates.

[0122] Point cloud data Statistical filtering is used for noise reduction, and preprocessed point cloud data is output.

[0123] like Figure 2 As shown, specifically, in a specific embodiment of the present invention, S100 specifically includes: acquiring the original point cloud data of the target scene by acquisition devices such as 3D LiDAR, structured light camera, or depth camera, denoted as:

[0124]

[0125] in , is the first in the point cloud data The coordinates of a point in three-dimensional space. Additional features (such as reflection intensity, color information, or normal vectors).

[0126] Point cloud data normalization processing:

[0127]

[0128] in, The mean vector of point cloud coordinates. is the standard deviation vector of the point cloud coordinates.

[0129] Statistical filtering is used for noise reduction, and the average distance of its k nearest neighbors is calculated for each point. If the following conditions are met:

[0130]

[0131] These are then identified as outliers and removed. and These are the global mean distance and standard deviation, respectively. This is the noise threshold factor.

[0132] In this embodiment of the invention, the multi-resolution point cloud construction step 102 includes:

[0133] Multi-level sampling is performed on the normalized and denoised point cloud. A far-point sampling combined with a curvature-based interest point sampling strategy is used to generate multiple point sets with different resolutions. ,in, For a low-resolution point set, For medium to high resolution point sets, For high-resolution point sets, low-resolution point sets Preserving global structure, high-resolution point set Preserve local details.

[0134] like Figure 2 As shown, specifically, in a specific embodiment of the present invention, S200 specifically includes: constructing a multi-resolution point cloud. This involves multi-level sampling of the normalized and denoised point cloud, employing a far-point sampling combined with a curvature-based interest point sampling strategy to generate multiple point sets of different resolutions. Among them, low resolution Preserve global structure, high resolution Preserve local details.

[0135] In this embodiment of the invention, the graph construction and progressive update step 103 includes:

[0136] In low-resolution point sets Construct a global sparse graph and edge weight matrix on top of it;

[0137] In medium-high resolution point sets In this process, local subgraphs are constructed only in regions of interest, which are determined by feature entropy or point motion estimation.

[0138] When the rate of change of node features in the graph exceeds a preset threshold, the reconstruction of the node neighborhood is triggered; otherwise, the neighborhood connection of the previous frame will be used.

[0139] like Figure 3 As shown, specifically, in a specific embodiment of the present invention, S300 specifically includes: graph construction and progressive update, on a low-resolution point set... Construct a global sparse graph and edge weight matrix The formula for calculating edge weights is:

[0140]

[0141] in Connecting points in the edge weight matrix and The weight of the edges. This represents the bandwidth parameter of the Gaussian kernel.

[0142] In medium-high resolution point sets In this process, local subgraphs are constructed only in regions of interest. The region of interest is determined by feature entropy or point motion estimation.

[0143] During the reasoning process, if the rate of change of a certain node's features is:

[0144]

[0145] If the neighboring node is not reconstructed, then the reconstructed neighborhood of that node is triggered; otherwise, the neighboring connections from the previous frame are used. For nodes exist The eigenvectors of time.

[0146] In this embodiment of the invention, the above-mentioned quantized pulse graph convolutional inference step 104 includes:

[0147] On the constructed graph structure, quantized pulse graph convolutional inference is used, and graph neural networks are used for feature aggregation.

[0148] The weight parameters of the graph are symmetrically quantized, the activation values ​​are asymmetrically quantized, and the model accuracy is maintained through quantization-aware training.

[0149] The quantized features are input into a spiking neural network. Using a neuron model and sparse pulse signals, calculations are performed when a valid event is received to generate event-driven feature aggregation.

[0150] like Figure 4 As shown, specifically, in a specific embodiment of the present invention, S400 specifically includes: quantizing pulse graph convolutional inference on the constructed graph structure, using a graph neural network for feature aggregation, the basic formula of which is:

[0151]

[0152] in, For nodes of 3D feature vectors For nodes The set of neighbors retrieved by KNN, This is a learnable weight matrix. The normalization coefficient is... It is a non-linear activation function.

[0153] The weights are quantized using 8-bit symmetric quantization, and the activations are quantized using 4-bit asymmetric quantization. Quantization-aware training (QAT) is then used to maintain model accuracy. The quantized features are input into a spiking neural network using a LIF neuron model, and the membrane potential update formula is as follows:

[0154]

[0155] when A pulse is delivered periodically, resetting the membrane potential. For nodes At the point of time membrane potential, Leakage coefficient, For input current, To determine whether discharge occurred at the previous moment, This is the threshold voltage.

[0156] In this embodiment of the invention, the above-mentioned task output step 105 includes:

[0157] The task output is point cloud target label segmentation. The class label of each point is predicted. A composite loss function is used to construct a composite loss according to the target task. The composite loss function includes: weighted cross-entropy function, Dice loss function and impulse sparsity regularization loss function.

[0158] Calculate the total loss function value and construct a composite loss function according to the target task. The total loss function value is the sum of the weighted cross-entropy function, the Dice loss function, and the impulse sparsity regularization loss function.

[0159] like Figure 2 As shown, specifically, in a specific embodiment of the present invention, S500 specifically includes: using point cloud target label segmentation as the task output, predicting the category label of each point. A composite loss function is used:

[0160] S501, Weighted Cross-Entropy:

[0161]

[0162] S502, Dice loss:

[0163]

[0164] S503, Pulse sparsity regularization loss:

[0165]

[0166] The total loss function is:

[0167] .

[0168] The following details specific implementation examples of the present invention:

[0169] This invention proposes a low-energy point cloud processing method based on quantized pulse map convolution and multi-resolution progressive update, comprising the following steps:

[0170] Step 1: Acquire the raw point cloud data of the target scene using acquisition devices such as 3D LiDAR, structured light cameras, or depth cameras, and record it as follows:

[0171]

[0172] in Let these be the spatial coordinates of the point. Additional features (such as reflection intensity, color information, or normal vectors).

[0173] By calculating the mean vector of the point cloud and standard deviation Complete the normalization process for the collected point cloud data:

[0174]

[0175] The normalized coordinate range is limited to the interval [-1, 1], which facilitates the numerical stability of the network input.

[0176] For noise reduction, this invention preferably uses statistical filtering, calculating the average distance of its k nearest neighbors for each point. If the following conditions are met:

[0177]

[0178] These are then identified as outliers and removed. and These are the global average distance and standard deviation, respectively, and α is the noise threshold factor, which generally ranges from [0.5, 2.0].

[0179] Step 2: Perform multi-level sampling on the normalized and denoised point cloud. A combination of far-point sampling and curvature-based interest point sampling strategy is used to generate multiple point sets at different resolutions. ;

[0180] Low-resolution point sets are obtained by far-point sampling on the low-resolution layer. In this example, the sampling ratio is 20% of the original number of points. In high-resolution layers... In this process, curvature-based point of interest sampling is adopted, prioritizing the retention of points with curvature greater than 0.05 to capture geometric detail features.

[0181] This embodiment constructs a three-layer resolution: a low-resolution layer P_1 (global structure), a medium-resolution layer P_2 (local supplementation), and a high-resolution layer P_3 (detail optimization).

[0182] Step 3: Graph construction and progressive update, on the low-resolution point set. The above is based on KNN to construct a global sparse graph. The formula for calculating edge weights is:

[0183]

[0184] in This is the median distance between adjacent points.

[0185] In medium-high resolution point sets In this process, local subgraphs are constructed only in regions of interest. The determination of regions of interest is based on the feature change rate:

[0186]

[0187] If the neighbor reconstruction of the node is triggered, then the neighbor connection of the previous frame will be used; otherwise, the neighbor connection of the previous frame will be retained. The neighborhood reconstruction is triggered at that time.

[0188] This progressive update mechanism ensures that static regions retain the graph structure of the previous frame, reducing unnecessary graph reconstruction calculations and thus lowering computational latency and energy consumption.

[0189] Step 4: Quantization of Pulse Graph Convolutional Inference. Based on the constructed graph structure, a graph neural network (Point-GCN, DGCNN, EdgeConv, etc.) is used for feature aggregation. The basic formula for graph convolution is:

[0190]

[0191] in, This is the normalization coefficient.

[0192] The weight parameters use 8-bit symmetric quantization, and the activation values ​​use 4-bit asymmetric quantization, combined with quantization-aware training (QAT) to reduce accuracy loss. The quantization mapping formula is:

[0193]

[0194] in The quantized value. The original value, Indicates the scaling factor. This is the offset. In symmetric quantization, the scaling factor and offset are calculated using the following formulas:

[0195]

[0196]

[0197] and These are the quantized maximum and minimum values. and These represent the maximum and minimum values ​​of the actual data. In asymmetric quantization, the scaling factor and offset are calculated using the following formulas:

[0198]

[0199]

[0200] The quantized features are input into a spiking neural network, which uses a LIF neuron model. Its membrane potential update formula is as follows:

[0201]

[0202] when A pulse is emitted and the membrane potential is reset at a given time. The node emits a pulse and resets the membrane potential as follows: .

[0203] Step 5: The task is to segment the point cloud target and predict the category label for each point. A composite loss function is used:

[0204] Weighted cross-entropy:

[0205]

[0206] Category weight Set according to the reciprocal of the category percentage in the training set.

[0207] Dice loss:

[0208]

[0209] Impulse sparsity regularization loss:

[0210]

[0211] The total loss function is:

[0212] .

[0213] The output of the quantized pulse graph convolutional network lies in the interval [0,1], and a composite loss is constructed according to the target task. , The entire calculation is performed on an 8-bit fixed-point firing rate, with no floating-point tensors involved.

[0214] Step Six, as Figure 5 As shown, to enable quantized pulse DGCNN to maintain end-to-end training capabilities with extremely low power consumption, after completing quantized-pulse forward inference, this embodiment employs STBP (Spatial-Time Propagation Backpropagation), a technique specifically designed for SNNs, to perform end-to-end training of the network. Since the pulse function is non-differentiable, this embodiment can use surrogate gradients to bypass the step.

[0215] As described above, the method of the present invention can be implemented well.

[0216] Compared with the prior art, the present invention has the following outstanding advantages and beneficial effects:

[0217] Compared with existing point cloud processing methods, the present invention has the following advantages:

[0218] Multi-resolution progressive update: This method reduces redundant computations globally and dynamically responds to changes in the region of interest. This invention constructs a global sparse graph on a low-resolution point set, and only constructs local subgraphs in the region of interest on medium- and high-resolution point sets. Furthermore, during inference, the reconstruction of a node's neighborhood is only triggered when the feature change rate of a node exceeds a preset threshold; otherwise, the neighborhood connections from the previous frame are reused. This mechanism solves the problem of low computational resource utilization in existing methods.

[0219] Quantization and Pulsation Dual Optimization: This method combines low-bit quantization with spiking neural networks, reducing computational bit width and number of operations, and achieving low-power inference. This invention performs 8-bit symmetric quantization on weights and 4-bit asymmetric quantization on activations, and combines this with spiking neurons for event-driven feature aggregation. This solves the problems in existing technologies where quantization and event-driven inference are not fully integrated, and the advantages of sparse computation are not fully utilized.

[0220] Composite Loss Function: This invention employs a composite optimization strategy that combines weighted cross-entropy loss, Dice loss, and impulse sparsity regularization loss. This strategy balances the overall accuracy of the model with the performance of sparse classes, thereby achieving more optimized segmentation results.

[0221] Embedded Adaptability: The method of this invention achieves high-performance operation on resource-constrained embedded and edge computing platforms by co-optimizing multi-resolution point cloud representation, sparse graph construction, low-bit quantization, and spiking neuron inference. It addresses the shortcomings of existing methods in terms of insufficient optimization for deployment on these devices and the difficulty in balancing accuracy, latency, and power consumption requirements.

[0222] Secondly, such as Figure 6As shown in the figure, this application provides a low-power point cloud processing system, which adopts the low-power point cloud processing method described above. The system is applied to embedded and edge computing platforms and includes:

[0223] Point cloud data acquisition and preprocessing module 201: Acquires raw point cloud data and performs preprocessing;

[0224] Multi-resolution point cloud construction module 202: Processes raw point cloud data in layers, generates multi-level resolution point sets, and establishes inter-layer mapping relationships;

[0225] Graph construction and progressive update module 203: Based on a multi-level resolution point set, a global graph is constructed in the low-resolution layer, and a local graph is constructed in the region of interest; and a progressive update mechanism based on feature change rate is used for dynamic updating;

[0226] Quantized Pulse Graph Convolutional Inference Module 204: Based on the constructed global and local graphs, low-bit quantization technology is used to generate low-bit quantized features, which are then combined with the spiking neuron model for event-driven feature aggregation.

[0227] Task output module 205: Based on the aggregated features, outputs segmentation results labeled with point cloud targets using a composite optimization strategy.

[0228] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the low-power point cloud processing method described above.

[0229] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the low-power point cloud processing method as described above.

[0230] In addition, combined Figure 1 The low-power point cloud processing method described in this application embodiment can be implemented by an electronic device, such as a computer device. Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application.

[0231] In some embodiments, the computer device may further include a communication interface 83 and a bus 80. For example, Figure 7 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.

[0232] Specifically, the processor 81 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0233] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 81.

[0234] The processor 81 reads and executes computer program instructions stored in the memory 82 to implement any of the low-power point cloud processing methods in the above embodiments.

[0235] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0236] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A low-energy point cloud processing method, characterized in that, The method is applied to embedded and edge computing platforms, and the method includes: Point cloud data acquisition and preprocessing steps: Acquire raw point cloud data and perform preprocessing; Multi-resolution point cloud construction steps: The original point cloud data is processed in layers to generate multi-layer resolution point sets and establish inter-layer mapping relationships; Graph construction and progressive update steps: Based on the multi-level resolution point set, a global graph is constructed in the low-resolution layer, and a local graph is constructed in the region of interest; and a progressive update mechanism based on feature change rate is used for dynamic updating. Quantized pulse graph convolutional inference steps: Based on the constructed global graph and local graph, low-bit quantization technology is used to generate low-bit quantized features, and event-driven feature aggregation is performed in combination with the spiking neuron model; Task output steps: Based on the aggregated features, output the segmentation results labeled with point cloud targets using a composite optimization strategy.

2. The low-energy point cloud processing method according to claim 1, characterized in that, The method further includes: Model training steps: After completing the forward inference of quantized impulses, the graph neural network model is trained by backpropagation with time and spatial domains.

3. The low-energy point cloud processing method according to claim 1 or 2, characterized in that, The point cloud data acquisition and preprocessing steps include: The raw point cloud data of the target scene is acquired by the acquisition device; the i-th point cloud data Including: the first point cloud data The coordinates of a point in three-dimensional space, and the first Additional features of each point; For the point cloud data The point cloud coordinates are normalized based on the mean vector and standard deviation vector of the point cloud coordinates. For the point cloud data Statistical filtering is used for noise reduction, and the preprocessed point cloud data is output.

4. The low-energy point cloud processing method according to claim 1 or 2, characterized in that, The multi-resolution point cloud construction steps include: Multi-level sampling is performed on the normalized and denoised point cloud. A far-point sampling combined with a curvature-based interest point sampling strategy is used to generate multiple point sets with different resolutions. ,in, For a low-resolution point set, For medium to high resolution point sets, For high-resolution point sets, low-resolution point sets Preserving global structure, high-resolution point set Preserve local details.

5. The low-energy point cloud processing method according to claim 1 or 2, characterized in that, The graph construction and progressive update steps include: In the low-resolution point set Construct a global sparse graph and edge weight matrix on top of it; In the medium-to-high resolution point set In this process, local subgraphs are constructed only in regions of interest, which are determined by feature entropy or point motion estimation. When the rate of change of node features in the graph exceeds a preset threshold, the reconstruction of the node neighborhood is triggered; otherwise, the neighborhood connection of the previous frame will be used.

6. The low-energy point cloud processing method according to claim 1 or 2, characterized in that, The quantized pulse graph convolutional inference step includes: On the constructed graph structure, quantized pulse graph convolutional inference is used, and graph neural networks are used for feature aggregation. The weight parameters of the graph are symmetrically quantized, the activation values ​​are asymmetrically quantized, and the model accuracy is maintained through quantization-aware training. The quantized features are input into a spiking neural network. Using a neuron model and sparse pulse signals, calculations are performed when a valid event is received to generate event-driven feature aggregation.

7. The low-energy point cloud processing method according to claim 1 or 2, characterized in that, The task output steps include: The task output is to segment point cloud targets and predict the category label of each point. A composite loss function is used to construct a composite loss function according to the target task. The composite loss function includes: weighted cross-entropy function, Dice loss function and impulse sparsity regularization loss function. Calculate the total loss function value and construct a composite loss according to the target task. The total loss function value is the sum of the weighted cross-entropy function, the Dice loss function, and the impulse sparsity regularization loss function.

8. A low-energy point cloud processing system, employing the low-energy point cloud processing method as described in any one of claims 1-7, characterized in that, The system is applied to embedded and edge computing platforms, and the system includes: Point cloud data acquisition and preprocessing module: Acquires raw point cloud data and performs preprocessing; Multi-resolution point cloud construction module: processes the original point cloud data in layers, generates multi-level resolution point sets, and establishes inter-layer mapping relationships; Graph construction and progressive update module: Based on the multi-level resolution point set, a global graph is constructed in the low-resolution layer, and a local graph is constructed in the region of interest; and a progressive update mechanism based on feature change rate is used for dynamic updating; Quantized pulse graph convolutional inference module: Based on the constructed global graph and the local graph, low-bit quantization technology is used to generate low-bit quantized features, and event-driven feature aggregation is performed in combination with the spiking neuron model; Task output module: Based on the aggregated features, output segmentation results labeled with point cloud targets using a composite optimization strategy.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the low-power point cloud processing method according to any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the low-power point cloud processing method as described in any one of claims 1-7.

Images