A millimeter wave radar point cloud compression method based on heterogeneous representation and implicit neural network

Abstract

The application discloses a millimeter wave radar point cloud compression method based on heterogeneous representation and implicit neural network. First, the point cloud fine geometric features are extracted through a point-based compression network, and then the global semantics are aggregated through a voxel-based compression network to realize point-voxel heterogeneous joint representation and preliminary compression. Secondly, an adaptive octree is constructed to encode the compressed data space structure to generate a partition bit stream, and a context-aware entropy coding is used to statistically compress the voxel-level high-dimensional features to generate a feature bit stream. Then, the partition bit stream, the feature bit stream and the neural network weight for implicit representation are jointly transmitted. Finally, the coarse-grained point cloud is decoded and recovered at the receiving end, and is input into a point-based synthesis network based on implicit neural representation together with the neural network weight, and through continuous surface modeling and detail compensation, a high-fidelity point cloud is finally reconstructed. The method significantly improves the compression efficiency and reconstruction quality of the millimeter wave radar point cloud at a low code rate.

Description

Technical Field

[0001] This invention relates to the technical fields of three-dimensional data processing, point cloud compression, autonomous driving environmental perception, and neural network applications, and in particular to a point cloud compression method based on heterogeneous representation and implicit neural networks for millimeter-wave radar point cloud data. Background Technology

[0002] In the rapid development of autonomous driving, robot navigation, and intelligent transportation systems, high-precision, real-time environmental perception is crucial for achieving system safety and intelligent decision-making. Millimeter-wave radar, due to its stable detection capabilities even in adverse weather conditions such as rain, fog, and darkness, has become an indispensable key sensor in vehicle-mounted perception systems. However, the point cloud data generated by millimeter-wave radar is characterized by sparse density, irregular distribution, and significant noise. Furthermore, with the improvement of sensor performance, the amount of point cloud data is increasing dramatically, placing enormous pressure on vehicle communication bandwidth and storage computing resources. Therefore, efficiently compressing millimeter-wave radar point clouds is a key technical challenge for reducing data transmission and storage costs in vehicle-road-cloud collaborative systems and ensuring real-time processing and interaction.

[0003] Traditional point cloud compression methods, such as those based on Geometric Information Coding (G-PCC) or Video Coding (V-PCC), while achieving some degree of compression, have limited efficiency in addressing the sparsity and irregularity inherent in millimeter-wave radar point clouds, and suffer significant loss of geometric details at low bit rates. In recent years, deep learning-based point cloud compression methods have demonstrated great potential, enabling more efficient capture of the semantic and geometric structure of point clouds through data-driven feature learning. However, existing learning-based methods still have significant limitations. First, most methods rely on a single point-based or voxel-based representation. While point-based methods can accurately represent point locations, they are inefficient at modeling the global context; voxel-based methods, while convenient for convolution operations, introduce quantization errors during the regularization process, resulting in loss of details, and consume enormous memory for high-resolution point clouds. This single representation makes it difficult to achieve the optimal balance between compression efficiency and reconstruction accuracy. Secondly, in the feature encoding stage, existing methods do not adequately utilize the statistical redundancy of high-dimensional feature vectors. They typically employ simple scalar quantization and general entropy coding, failing to fully leverage the spatial locality of point clouds and the correlation between feature channels, resulting in room for bitrate optimization. Finally, in the decoding and reconstruction stage, existing methods often use explicit upsampling or simple fully connected networks for point coordinate regression, which struggles to effectively recover the geometric details lost during compression, especially high-frequency details. This leads to a rough surface in the reconstructed point cloud, affecting the accuracy of subsequent perception tasks.

[0004] Therefore, breaking through the limitations of existing methods in the representation, encoding, and reconstruction stages, and exploring a new end-to-end cloud compression framework that can deeply integrate the advantages of irregular and regular representations, achieve efficient statistical compression, and possess strong detail recovery capabilities, has theoretical value and practical significance for promoting the real-time, efficient, and high-fidelity processing and transmission of 3D perception data in fields such as autonomous driving. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a millimeter-wave radar point cloud compression method based on heterogeneous representation and implicit neural networks. This method aims to achieve efficient compression and high-fidelity reconstruction of millimeter-wave radar point clouds through innovative heterogeneous representation encoding, context-aware entropy encoding, and implicit neural representation reconstruction. The specific technical solution is as follows:

[0006] A millimeter-wave radar point cloud compression method based on heterogeneous representation and implicit neural network includes the following steps:

[0007] Step S1: Collect raw point cloud data using millimeter-wave radar to form an input point cloud; firstly, input a point-based compression network to encode and downsample the point-level geometry to obtain an intermediate point cloud representation; then input the intermediate point cloud representation into a voxel-based compression network, map it to a regular voxel space, perform feature aggregation and compression, and output compressed point cloud data.

[0008] Step S2: Generate spatial partitioning information based on the compressed point cloud data, and extract voxel-level high-dimensional features through a parametric feature analysis network; wherein, the spatial partitioning information is structured and encoded by a tree-based encoder to generate a partitioned bitstream, and the voxel-level high-dimensional features are statistically compressed by an entropy encoder to generate a feature bitstream.

[0009] Step S3: Jointly transmit the partitioned bitstream, the feature bitstream, and the neural network weights used for implicit neural representation;

[0010] Step S4: At the receiving end, tree-based decoding and entropy decoding are performed on the partitioned bit stream and the feature bit stream respectively to recover the coarse-grained point cloud representation; then the coarse-grained point cloud representation and the neural network weights are input into a point-based synthesis network based on implicit neural representation to perform continuous surface modeling and geometric detail compensation, and finally reconstruct high-quality point cloud data.

[0011] Further, step S1 specifically includes: Step S1.1: Inputting the original point cloud data into the point-based compression network, selecting a representative point set through the farthest point sampling algorithm, constructing a local neighborhood on each sampling point, extracting local features through multilayer perceptron and pooling operations, and generating an intermediate point cloud representation composed of the sampling point coordinates and their corresponding high-dimensional features; Step S1.2: Inputting the intermediate point cloud representation into the voxel-based compression network, discretizing it into a sparse voxel feature map through voxelization operations; then using a three-dimensional sparse convolutional neural network to perform deep feature learning and downsampling on the sparse voxel feature map to obtain a downsampled point set; Step S1.3: Introducing a feature aggregation module to perform cross-layer connections and adaptive weighted fusion on the multi-level features extracted by the three-dimensional sparse convolutional neural network to generate aggregated features; combining the aggregated features with the downsampled point set to form the final compressed point cloud data.

[0012] Further, step S2 specifically includes: Step S2.1: After mapping the compressed point cloud data to voxel space, a nonlinear transformation is performed through a parameterized feature analysis network to extract a voxel-level high-dimensional feature map containing rich semantic and geometric information; Step S2.2: The voxel-level high-dimensional feature map is input into the tree-based encoder, and the space is recursively partitioned using an adaptive octree structure; whether to split is determined based on the amount of feature information and depth within a node, and an occupancy code sequence indicating the occupancy status of child nodes is generated; the occupancy code sequence is arithmetically encoded using a context-based learning probabilistic model to generate the partitioned bitstream; Step S2.3: The voxel-level high-dimensional features associated with the leaf nodes of the adaptive octree are quantized, and the quantized features are statistically compressed using an entropy coding algorithm based on context modeling to generate the feature bitstream.

[0013] Furthermore, in step S2.3, the context-based entropy coding algorithm employs deep context adaptive binary arithmetic coding, and its context probability model is a hybrid probability model used to predict the conditional probability distribution of each quantization coefficient to be encoded.

[0014] Furthermore, in step S3, the neural network weights are parameters of a multilayer perceptron; before transmission, the neural network weights are quantized and then compressed using the same deep context adaptive binary arithmetic encoding as in step S2.3 to generate a weight bitstream; subsequently, the partition bitstream, feature bitstream, and weight bitstream are combined into a unified transmission bitstream for transmission.

[0015] Further, step S4 specifically includes: Step S4.1: The receiving end demultiplexes the transmitted bitstream, separating the partition bitstream and the feature bitstream; reconstructs the octree spatial structure through tree-based decoding, and recovers the voxel-level high-dimensional features associated with the leaf nodes through entropy decoding and dequantization; Step S4.2: The recovered voxel-level high-dimensional features are used to form a sparse voxel feature map according to their spatial location, and are processed sequentially through a parameterized feature synthesis network and a voxel-based synthesis network; the parameterized feature synthesis network upsamples and enhances the features through deconvolution operations, and the voxel-based synthesis network upsamples and enhances the features through deconvolution operations. The network predicts the occupancy state and coordinate offset of voxels based on the enhanced features, generating the coarse-grained point cloud representation; Step S4.3: Input the coarse-grained point cloud representation and the neural network weights into the point-based synthesis network based on implicit neural representation; For a spatial query point, its high-frequency encoded coordinates are concatenated with the local features interpolated from the coarse-grained point cloud, and input into a multilayer perceptron parameterized by the neural network weights to predict the signed distance value from the query point to the implicit surface. Surface reconstruction and detail compensation are achieved through implicit field modeling, ultimately generating a high-quality reconstructed point cloud.

[0016] Furthermore, in step S4.3, the point-based synthesis network based on implicit neural representation introduces an Eikonal regularization term during training to constrain the gradient of the predicted signed distance field to be close to the unit length, thereby ensuring that the learned function is an effective signed distance function, thus generating a geometrically reasonable smooth surface.

[0017] The advantages and beneficial effects of this invention are as follows:

[0018] 1. This invention proposes a heterogeneous cascaded encoding architecture of point-based and voxel-based networks. By combining the synergistic effect of point-based networks preserving fine geometric structures with voxel-based networks efficiently aggregating global semantics, it solves the inherent contradiction between compression efficiency and geometric fidelity in single representation methods, providing a more compact and robust feature representation foundation for irregular point cloud data.

[0019] 2. This invention designs a compression strategy that combines tree-based coding and contextual entropy coding. It uses an adaptive octree to structurally represent spatial sparsity and utilizes a deep learning-based contextual model to accurately characterize the statistical dependencies of high-dimensional features. This overcomes the key technical bottlenecks of traditional coding methods in point cloud feature compression, such as high redundancy and imprecise bitrate allocation, and achieves higher compression efficiency.

[0020] 3. This invention introduces a point cloud detail reconstruction mechanism based on implicit neural representation (INR), which combines coarse-grained point cloud with continuous implicit field modeling. Through coordinate-driven neural networks, it performs fine repair and completion of the three-dimensional surface, which significantly improves the geometric integrity and detail fidelity of the reconstructed point cloud under low bit rate conditions, and overcomes the limitation of insufficient detail recovery capability of explicit decoding methods. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method in an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of the encoder-decoder module structure in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the parametric feature network and INR-Based point synthesis module structure in an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of the dot matrix element network structure in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0026] like Figures 1-4 As shown, the present invention provides a millimeter-wave radar point cloud compression method based on heterogeneous representation and implicit neural network, aiming to achieve efficient compression and high-fidelity reconstruction of millimeter-wave radar point clouds, specifically including the following steps:

[0027] Step 1: Collect raw point cloud data using millimeter-wave radar to form the input point cloud. First, a point-based compression network is input to encode the point-level geometry, resulting in an intermediate point cloud representation. ; then Input a voxel-based compression network to map the point cloud to a regular voxel space, achieving a heterogeneous joint representation of point basis and voxel basis, and output compressed point cloud data. ;

[0028] Step 1.1: Input raw point cloud data A point-based compression network is proposed. This network first encodes the geometric structure of the unordered point set. Its core operations are farthest point sampling and feature extraction, aiming to extract more representative and robust structural features from the original high-density point cloud and generate a coarse-grained point cloud. .

[0029] First of all, By applying farthest-point sampling and iteratively selecting a representative point set based on Euclidean distance, the number of point clouds is significantly reduced while preserving the spatial distribution structure as much as possible. The sampled point cloud is denoted as... .in, This represents the three-dimensional coordinate vector of the j-th point in the sampled point cloud. This represents the number of points in the point cloud after sampling, specifically the number of points retained after sampling from the farthest point. This represents the total number of points in the original point cloud data. The sampling process can be formalized as follows:

[0030]

[0031] in, It is a candidate point, representing the original point cloud. A point that has not yet been selected into the sampling point set. It is a point in the selected point set, where k is an index (k j), representing the k-th selected point.

[0032] Subsequently, A local neighborhood is constructed at each point, and local features are extracted using a multilayer perceptron with shared parameters. For each point... Its neighborhood feature aggregation formula is:

[0033]

[0034] in, This represents vector concatenation. express The K-nearest neighbor set is an integer hyperparameter representing the number of nearest neighbors used to control the size of the neighborhood during local feature extraction. This represents the element-wise max pooling operation. MLP (Multilayer Perceptron) is a feedforward neural network model. Finally, the extracted high-dimensional local features are combined with the coordinates of the sampling points to obtain the intermediate point cloud representation. .

[0035] Step 1.2: Convert the point cloud Input a voxel-based compression network to achieve heterogeneous representation transformation from points to voxels. This step maps irregular point cloud data to a regular voxel space, and further compresses and abstracts it to output a highly compressed point cloud representation. .

[0036] Step 1.2.1: First, perform voxelization to discretize the continuous 3D space into a regular mesh, and aggregate the features of points falling within the same voxel grid. Let the voxel resolution be... The voxelization process can be represented as:

[0037]

[0038] in, The coordinates of the minimum boundary of the point cloud. Representing the intermediate point cloud The coordinates of a three-dimensional point, Is and point The associated high-dimensional feature vectors. Voxel lattice size, Mean pooling is used to obtain sparse voxel feature maps. R represents the set of real numbers, that is, the set of all real numbers.

[0039] Step 1.2.2: Subsequently, The input is fed into a 3D sparse convolutional neural network for deep feature learning. Sparse convolution, through an efficient implementation based on spatial hashing, only performs computation on non-empty voxels; its core computation can be represented as:

[0040]

[0041] in, Indicates the first l In the +1 convolutional layer, the output feature value located at voxel position u This represents the input feature value located at the relative position u+v in the l-th convolutional layer. Therefore, the body element The set of non-empty voxels in the neighborhood of the convolution kernel centered at the center. and For weights and biases.

[0042] Next, the extracted voxel features are downsampled, and the spatial resolution is reduced by strided sparse convolution to obtain the downsampled voxel feature map. Then, back-project it back onto the point cloud to obtain the downsampled point set. .

[0043] Step 1.3: Introduce the IRN feature aggregation module to integrate multi-level sparse convolutional features to enhance expressive power. The IRN module receives features from multiple levels. As input, aggregation is performed through cross-layer connections and an adaptive weighting mechanism.

[0044] Features at different levels Application of Channel Attention Weight Generation Network Obtain the weight vectors of each layer. Subsequently, channel-level weighted fusion is performed:

[0045]

[0046] Where L is an integer representing the number of feature levels. It is an activation function, and a nonlinear transformation function. This indicates channel-by-channel multiplication. Used to unify the number of channels. Finally, aggregate features. The output is a compressed point cloud after passing through a lightweight projection head. The feature part, and the coordinates of the downsampling point. Combined, the final compressed point cloud data is formed. ,in For the aggregated features, N2 is an integer representing the total number of points in the compressed point cloud X2. The final output... As a compressed point cloud representation with high semantics and low redundancy, it is fed into the subsequent entropy coding and structured coding modules.

[0047] Step 2: As Figure 2 As shown, based on compressed point cloud Spatial partitioning information is generated, and voxel-level high-dimensional features are extracted through a parametric feature analysis network. The partitioning structure is structured by a tree-based encoder, and the parametric features are statistically compressed by an entropy encoder to generate partition bitstreams and feature bitstreams, respectively.

[0048] Step 2.1: Utilize a parametric feature analysis network to process the input compressed point cloud data. The process involves extracting voxel-level high-dimensional features for encoding.

[0049] Input downsampled point cloud data First, it is mapped to voxel space, where each non-empty voxel... Features It is obtained by aggregating features from all points within it. These features are then subjected to a non-linear transformation through a parameterized network to enhance their expressive power. The core of this network consists of multiple cascaded convolution-normalization-activation units, ultimately outputting a voxel-level high-dimensional feature map containing rich semantic and geometric information. This process can be formally represented as:

[0050]

[0051] in, This indicates a voxelization operation. The internal transformation of the parametric feature analysis network can be described as follows:

[0052]

[0053] here, For the first Features of the layer Represents sparse convolution. This indicates a normalization operation. This represents the activation function.

[0054] Step 2.2: Extract the voxel-level high-dimensional features from Step 2.1 The input tree-based encoder performs structured encoding of spatial partitioning information to generate a partitioned bitstream. This step involves two core operations: octree construction and arithmetic encoding.

[0055] First, the octree construction recursively partitions the space using a 3D bounding box containing all non-empty voxels as the root node, generating an adaptive octree T. Partitioning conditions... The information content I of the features within a node and the node depth d determine:

[0056]

[0057] Among them, node information volume (node) It is the sum of the L2 norms of all voxel eigenvectors within this node. Information threshold This represents the maximum depth. If the condition is true, the node is divided into 8 child nodes, and an 8-bit occupancy code is generated. This indicates whether each child node contains a feature. The formula for calculating each bit of the occupancy code is:

[0058]

[0059] Where 1 is an indicator function, Indicates the first Each node has a child node. This process generates a sequence containing the occupancy codes of all nodes. and the set of feature vectors stored in the leaf points .

[0060] Secondly, the occupancy code sequence of the octagonal tree node. Compress it. Each occupancy code The encoding is based on a probability model This model relies on contextual information. Its conditional probability distribution can be modeled as follows:

[0061]

[0062] in, Indicates in a given context Time The probability that a child node is occupied. The arithmetic encoder uses this probability model to process the sequence. Encode the data to generate the final partitioned bitstream. Its codeword length is theoretically close to the Shannon information entropy of the sequence:

[0063]

[0064] Step 2.3: Extract the voxel-level high-dimensional features from Step 2.1 and associate them with the octagonal leaf nodes. The input entropy encoder performs quantization and statistical compression to generate a feature bitstream. This process includes three sub-steps: quantization, context modeling, and DeepCABAC encoding.

[0065] First, for continuous eigenvectors Uniform scalar quantization is performed to map the quantization values ​​to integer values ​​for entropy encoding. The quantization and dequantization formulas are as follows:

[0066]

[0067]

[0068] Round is a function that represents the "rounding" operation. To quantize the step size, For quantized integer value features, These are the quantized features that can be reconstructed at the decoding end.

[0069] Then, an accurate probability model is constructed for each quantization coefficient to be encoded to achieve efficient encoding. The contextual probability model is based on a mixture model, and the quantization residuals follow a predictive value. A Gaussian distribution centered at the center, with scale parameter Context information through parameterized network predict:

[0070]

[0071]

[0072] Here, ctx is an abbreviation for "context," representing contextual information. This represents a uniform distribution, used to simulate the uncertainties introduced by quantization. This represents a convolution operation. This model provides conditional probabilities for each coefficient. .

[0073] Finally, based on the aforementioned contextual probability model, the DeepCABAC algorithm is used to perform element-wise entropy encoding on the quantized feature Q. For each coefficient q, its encoded code length is approximately its information entropy:

[0074]

[0075] Encode all coefficients in Q to generate the final feature bitstream. This bitstream and the partitioned bitstream Together with the output of step 2, they are used for subsequent transmission or storage.

[0076] Step 3: Generate INR network weights for implicitly representing point cloud geometry, and then jointly transmit the partition bit stream obtained by tree-based encoding, the feature bit stream obtained by entropy encoding, and the INR network weights to achieve efficient transmission of point cloud structural information and implicit continuous representation parameters.

[0077] Step 3.1: Generate a weight bitstream using quantization and entropy encoding of the INR network weights, which will be used as the weight parameters for the point-based synthesis network of the implicit neural representation. It also needs to be transmitted. It is a parameter set of a multilayer perceptron. First, the continuous weights are quantized:

[0078]

[0079] in, The step size for weight quantization is then determined. Subsequently, DeepCABAC entropy coding, similar to step 2.3, is used to... Compress to generate a weighted bitstream. During encoding, its context These can be adjacent weights within the same layer or statistical priors from pre-training.

[0080] Step 3.2: Divide the bitstream into partitions Feature bit stream and weighted bitstream Bitstream multiplexing is performed to form a unified transmission bitstream. The multiplexing process typically involves adding a short header to the bitstream, indicating the length of each substream and necessary decoding parameters (such as quantization step size). This ensures that the decoding end can be correctly separated. Its structure can be represented as:

[0081]

[0082] Here, Header is the transmitted bit stream. The header information segment, also known as the header or metadata segment, These represent the lengths of each sub-stream. This indicates bitstream concatenation. Ultimately, this composite bitstream... Transmission via a channel enables efficient joint transmission of point cloud structural information (partition bitstream), explicit features (feature bitstream), and implicit continuous representation parameters (weight bitstream).

[0083] Step 4: As Figure 3 As shown, at the receiving end, tree-based decoding and entropy decoding are performed on the partitioned bitstream and feature bitstream, respectively, and coarse-grained point clouds are recovered through a parameterized feature synthesis network and a voxel-based synthesis network. ; then By inputting the weights of the INR network into the INR-Based point synthesis network, continuous modeling and detail compensation of the point cloud geometry are performed, ultimately reconstructing a high-quality point cloud. ;

[0084] Step 4.1: After receiving the multiplexed bit stream, the receiving end first demultiplexes it to separate the partitioned bit streams. and feature bit stream .

[0085] First, tree-based decoding will partition the bitstream. Arithmetic decoding is performed to reconstruct the spatial structure of the octree T established by the sending end. The decoding process is the reverse of the encoding process, starting from the root node and utilizing the same parameterized context probability model as the encoding end. Iteratively decode the 8-bit occupancy code of each node. Specifically, for the current node to be decoded, its parent node and its already decoded sibling nodes constitute the context. The decoder determines the occupancy code of the current node based on the following formula:

[0086]

[0087] in, It is an arithmetic decoding algorithm, a lossless data decoding technique based on probability and statistics. Based on the decoded... Determine child nodes Does it exist ( (Indicates existence). If the child node exists and the maximum depth of the tree has not been reached. If a node is not found, it is added to the decoding queue. This process is repeated recursively until all nodes have been processed, ultimately reconstructing the complete octree structure T′ with its set of leaf nodes. Spatial partitioning of the compressed point cloud was defined.

[0088] Then, entropy decoding will process the feature bitstream. Perform DeepCABAC entropy decoding. The decoder uses the same context model as the encoder. To predict each scalar to be decoded conditional probability distribution And perform arithmetic decoding:

[0089]

[0090] Among them, context The construction is symmetrical to the encoding end. After decoding all the quantized integer feature scalars, they are reassembled into a set of quantized feature vectors corresponding to the octagonal leaf nodes. Then, dequantization is performed to restore the continuous floating-point feature vector:

[0091]

[0092] in, To be consistent with the quantization step size vector at the encoder end, This represents element-wise multiplication. The final result is a reconstructed set of high-dimensional feature vectors at the voxel level. Each feature vector is associated with a leaf node of the octree T′.

[0093] Step 4.2: This step involves inputting the decoded feature vectors into a parameterized feature synthesis network and a voxel-based synthesis network to recover the coarse-grained point cloud. .

[0094] Step 4.2.1: First, parametric feature synthesis will reconstruct the voxel features. Based on the position of its associated octagonal leaf node, it is placed into the corresponding 3D voxel mesh, forming a sparse voxel feature map. Subsequently, the sparse feature map is fed into a parameterized feature synthesis network. This network typically consists of multiple deconvolutional or sparse deconvolutional layers, with the aim of recovering denser intermediate-level voxel features from the compressed leaf node features. A deconvolution operation can be formalized as:

[0095]

[0096] in, Representative at the l Voxel feature map after +1 layer upsampling It is the first l The sparse voxel characteristics of the layer This represents a (sparse) deconvolution operation used for upsampling and increasing spatial resolution. Indicates normalization, This represents the activation function. After processing by this network, a voxel feature map with higher resolution and richer features is obtained. .

[0097] Step 4.2.2: Next, the voxel-based synthesis network will make the feature map The data is input into a voxel-based synthesis network, which is responsible for generating explicit 3D point sets from voxel features. For example... Figure 4 As shown, firstly, the network further refines features through a 3D sparse convolutional layer and predicts the geometric existence probability and coordinate offset of each voxel. Let the input features be... After processing with a 3x3x3 sparse convolution kernel K (representing a 3D convolution kernel, weight tensor), preliminary geometric features are output. :

[0098]

[0099] Here, K is a 3×3×3 sparse convolution kernel. This represents the sparse convolution operation. For bias. Then, for... An IRN feature aggregation module is applied, which fuses feature residuals from corresponding levels at the encoding end to enhance details. The fusion formula can be expressed as:

[0100]

[0101] in, These are features passed from and decoded by the corresponding layer of the encoder. These are the aggregated weights for adaptive learning. Finally, the upsampled aggregated features are... The process involves processing each voxel predicted to exist (i.e., "Occupancy > threshold") based on its predicted coordinate offset. and its voxel center coordinates Calculate the three-dimensional coordinates of the point inside the voxel. By combining all the calculated point coordinates, we obtain the reconstructed coarse-grained point cloud. .

[0102] Step 4.3: Finally, the... By inputting the weights of the INR network into the INR-Based point synthesis network, continuous modeling and detail compensation of the point cloud geometry are performed, ultimately reconstructing a high-quality point cloud. .

[0103] Although coarse-grained point clouds It provides a global shape but may lack fine geometric details. INR-Based Point-Based Synthesis Networks are used to perform continuous implicit function modeling of the point cloud surface to compensate for the lack of detail.

[0104] Step 4.3.1: Therefore, network input and coordinate encoding are performed first for any three-dimensional spatial query point. Its input is a concatenation of its location information and the information from... Local context features obtained through interpolation. First, the coordinates of the query point... Perform high-frequency position coding To help MLP learn high-frequency details:

[0105]

[0106] in, This represents a high-frequency positional coding function, whose function is to map low-dimensional three-dimensional coordinate points to a high-dimensional feature space. Simultaneously, in Search in Find the K nearest neighbors and aggregate the features of these neighbors. Encoded coordinates With local features splicing, as a coordinate basis MLP network The input, where This refers to the transmitted INR network weights.

[0107] Step 4.3.2: Then perform implicit field prediction, which predicts a signed distance field (SDF) value. This represents the occupancy probability, implicitly defining a 3D shape surface. Its core formula is:

[0108]

[0109] in, Point Signed distance to the hidden surface ( On the surface, Internally, (on the outside) h It is an intermediate feature vector.

[0110] Step 4.3.3: Finally, perform detail compensation and surface reconstruction. To obtain accurate surface points, we need to find the conditions that meet the requirements. The isosurface points. To enhance the geometric quality of the reconstructed surface, especially in areas with sharp features and large curvature changes, the network incorporates an Eikonal regularization term during training, requiring the gradient of the predicted SDF function to be unit length:

[0111]

[0112] Here, Concat is an abbreviation for "concatenate," representing a vector concatenation operation. Through the network For input The automatic differentiation is obtained. This constraint ensures that the learned implicit field is an effective signed distance function, thus producing a smooth and geometrically reasonable surface.

[0113] Step 4.5: Through the above process, a high-quality surface defined in continuous space can be obtained. Uniform or importance sampling of this surface will then generate the final reconstructed point cloud. This point cloud Not only inherited The global structure also compensates for the geometric details lost during compression through continuous implicit representations, achieving high-fidelity reconstruction.

[0114] In summary, the millimeter-wave radar point cloud compression method based on heterogeneous representation and implicit neural networks of this invention includes four key steps: First, raw point cloud data is acquired by millimeter-wave radar and input into a point-based compression network to encode and downsample the point-level geometric structure. Then, the intermediate point cloud representation is input into a voxel-based compression network and mapped to a regular voxel space to efficiently aggregate global semantics, achieving heterogeneous joint representation and compression of point-based and voxel-based data, and outputting compressed point cloud data. Next, spatial partitioning information is generated based on the compressed point cloud, and voxel-level high-dimensional features are extracted through a parametric feature analysis network. The partitioning structure is structured and encoded by a tree-based encoder to generate a partitioned bitstream, and the parametric features are statistically compressed using an entropy encoder based on context modeling (such as DeepCABAC) to generate a feature bitstream. Finally, the partitioned bitstream, feature bitstream, and INR network weights used for implicit representation are jointly transmitted to achieve efficient transmission of point cloud structure, attributes, and continuous representation parameters. Finally, at the receiving end, tree-based decoding and entropy decoding are performed on the partitions and feature bitstreams respectively, and coarse-grained point clouds are recovered through parameterized feature synthesis network and voxel-based synthesis network. Then, the coarse-grained point cloud data is input into the INR-Based point-based synthesis network along with the INR network weights. By performing implicit continuous modeling and detail compensation on the point cloud surface, high-quality point cloud data is finally reconstructed.

[0115] This method is particularly suitable for scenarios with stringent requirements for real-time performance and perception accuracy, such as autonomous driving and robot navigation. It can effectively solve the core contradiction between efficient compression and high-fidelity reconstruction of millimeter-wave radar point clouds, and is of great significance for improving the three-dimensional environmental perception capability and reliability of intelligent systems under limited bandwidth.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for compressing millimeter-wave radar point clouds based on heterogeneous representation and implicit neural networks, characterized in that, Includes the following steps: Step S1: Collect raw point cloud data using millimeter-wave radar to form an input point cloud; firstly, input a point-based compression network to encode and downsample the point-level geometry to obtain an intermediate point cloud representation; then input the intermediate point cloud representation into a voxel-based compression network, map it to a regular voxel space, perform feature aggregation and compression, and output compressed point cloud data. Step S2: Generate spatial partitioning information based on the compressed point cloud data, and extract voxel-level high-dimensional features through a parametric feature analysis network; wherein, the spatial partitioning information is structured and encoded by a tree-based encoder to generate a partitioned bitstream, and the voxel-level high-dimensional features are statistically compressed by an entropy encoder to generate a feature bitstream. Step S3: Jointly transmit the partitioned bitstream, the feature bitstream, and the neural network weights used for implicit neural representation; Step S4: At the receiving end, tree-based decoding and entropy decoding are performed on the partitioned bit stream and the feature bit stream respectively to recover the coarse-grained point cloud representation; then the coarse-grained point cloud representation and the neural network weights are input into a point-based synthesis network based on implicit neural representation to perform continuous surface modeling and geometric detail compensation, and finally reconstruct high-quality point cloud data. Step S4.1: The receiving end demultiplexes the transmitted bit stream to separate the partition bit stream and the feature bit stream; it reconstructs the octree spatial structure through tree-based decoding, and recovers the voxel-level high-dimensional features associated with the leaf nodes through entropy decoding and dequantization. Step S4.2: The recovered voxel-level high-dimensional features are used to form a sparse voxel feature map based on their spatial location, and then processed sequentially through a parameterized feature synthesis network and a voxel-based synthesis network. The parameterized feature synthesis network upsamples and enhances the features through deconvolution operations, and the voxel-based synthesis network predicts the voxel occupancy state and coordinate offset based on the enhanced features to generate the coarse-grained point cloud representation. Step S4.3: Input the coarse-grained point cloud representation and the neural network weights into the point-based synthesis network based on the implicit neural representation; for a spatial query point, concatenate its high-frequency encoded coordinates with the local features interpolated from the coarse-grained point cloud, input the concatenation into a multilayer perceptron parameterized by the neural network weights, predict the signed distance value from the query point to the implicit surface, realize surface reconstruction and detail compensation through implicit field modeling, and finally generate a high-quality reconstructed point cloud; The point-based synthesis network based on implicit neural representation introduces an Eikonal regularization term during training to constrain the gradient of the predicted signed distance field to be close to the unit length, thereby generating a geometrically reasonable smooth surface.

2. The millimeter-wave radar point cloud compression method according to claim 1, characterized in that, Step S1 specifically includes: Step S1.1: Input the original point cloud data into the point-based compression network, select a representative point set through the farthest point sampling algorithm, construct a local neighborhood on each sampling point, extract local features through multilayer perceptron and pooling operation, and generate an intermediate point cloud representation composed of the sampling point coordinates and their corresponding high-dimensional features; Step S1.2: Input the intermediate point cloud representation into the voxel-based compression network, and discretize it into a sparse voxel feature map through a voxelization operation; then use a three-dimensional sparse convolutional neural network to perform deep feature learning and downsampling on the sparse voxel feature map to obtain a set of downsampled points; Step S1.3: Introduce a feature aggregation module to perform cross-layer connections and adaptive weighted fusion on the multi-level features extracted by the three-dimensional sparse convolutional neural network to generate aggregated features; combine the aggregated features with the downsampled point set to form the final compressed point cloud data.

3. The millimeter-wave radar point cloud compression method according to claim 1, characterized in that, Step S2 specifically includes: Step S2.1: After mapping the compressed point cloud data to voxel space, perform nonlinear transformation through a parameterized feature analysis network to extract voxel-level high-dimensional feature maps containing rich semantic and geometric information; Step S2.2: Input the voxel-level high-dimensional feature map into the tree-based encoder, and recursively partition the space using an adaptive octree structure; determine whether to split based on the amount of feature information and depth within a node, and generate an occupancy code sequence indicating the occupancy status of child nodes; use a probabilistic model based on context learning to perform arithmetic encoding on the occupancy code sequence to generate the partitioned bitstream; Step S2.3: Quantize the voxel-level high-dimensional features associated with the adaptive octagonal leaf nodes, and statistically compress the quantized features using an entropy coding algorithm based on context modeling to generate the feature bitstream.

4. The millimeter-wave radar point cloud compression method according to claim 3, characterized in that, In step S2.3, the context-based entropy coding algorithm uses deep context adaptive binary arithmetic coding, and its context probability model is a hybrid probability model, which is used to predict the conditional probability distribution of each quantization coefficient to be encoded.

5. The millimeter-wave radar point cloud compression method according to claim 4, characterized in that, In step S3, the neural network weights are parameters of a multilayer perceptron. Before transmission, the neural network weights are quantized and then compressed using the same deep context adaptive binary arithmetic encoding as in step S2.3 to generate a weight bitstream. The partitioned bitstream, feature bitstream, and weighted bitstream are then combined into a unified transmission bitstream for transmission.

Images