Dual-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features

Abstract

The invention discloses a dual-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features. The method comprises the following steps: extracting representative skeleton anchor points from original point clouds by adopting farthest point sampling; predicting an octree node occupancy state by using an anchor encoder, and performing entropy coding to generate a skeleton bit stream; a skeleton condition feature encoder is introduced, probability distribution of local features is predicted by using skeleton context priori, cross-path context association is constructed, and a feature bit stream is generated; in the decoding stage, the sparse skeleton point set is restored from the skeleton bit stream, local features are recovered from the feature bit stream in combination with the skeleton structure context, fine-grained reconstruction is carried out on the point cloud through a geometric fusion point cloud up-sampling module, and finally the high-fidelity point cloud is obtained. According to the invention, the point cloud compression effect and reconstruction quality can be improved. The method can be widely applied to the point cloud field.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of point cloud compression, in particular to a double-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features. BACKGROUND

[0002] In recent years, with the rapid development of three-dimensional perception technology, the wide application of sensing devices such as laser radar in the fields of unmanned driving, intelligent manufacturing, digital twin, etc. makes the use of high-precision, high-resolution point cloud data become more and more common. The point cloud data is large in number and sparse, in order to meet the efficiency requirements of storage and transmission, point cloud compression technology has become a research hotspot. Traditional point cloud geometry compression methods mostly use octree structure to divide the space in layers, and realize compression through occupancy map and residual coding. However, this method performs well in low complexity scenarios, but the utilization of context information is limited, and it lacks end-to-end learning ability. SUMMARY

[0003] Therefore, in order to solve the technical problem that the current point cloud compression method lacks global modeling ability of the hierarchical relationship between nodes in the octree structure and the geometric relationship, resulting in insufficient context awareness, and further resulting in low compression effect and reconstruction quality, the present application proposes a double-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features, which comprises the following steps:

[0004] First, representative skeleton anchor points are extracted from the original point cloud using farthest point sampling;

[0005] Taking each skeleton point as the center, a multi-scale local window is constructed by finding the nearest multiple neighbor points for each skeleton anchor point to represent local geometric details;

[0006] The skeleton anchor points are organized into a hierarchical structure via octree coding, and the spatial consistency is improved through non-overlapping window division. Then, the anchor point encoder is used to predict the occupancy state of the octree node and perform entropy coding to generate the skeleton bit stream. At the same time, the features extracted from the local window are input into the graph attention local feature extractor to gradually aggregate the multi-scale geometric context through multi-layer graph convolution and attention mechanism, and extract rich local feature representation;

[0007] In order to realize efficient cooperation between the structure path and the detail path, the present application further introduces a skeleton conditional feature encoder, which uses the skeleton context prior to predict the probability distribution of local features, constructs a cross-path context association, and generates a feature bit stream;

[0008] In the decoding stage, the sparse skeleton point set is first restored from the skeleton bit stream, and then the local features are recovered from the feature bit stream in combination with the skeleton structure context. Through the learnable geometric fusion point cloud upsampling module, the point cloud is reconstructed in fine granularity, and finally a high-fidelity point cloud is obtained.

[0009] Based on the above scheme, the application provides a dual-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features, which realizes high-fidelity reconstruction of point cloud data at a high compression rate and improves compression efficiency and reconstruction accuracy by constructing a dual-path laser radar point cloud compression framework fusing multi-scale geometric features, introducing an anchor point coding network of octree structure, a graph-driven multi-scale feature aggregation mechanism and a structure prior guided context modeling module. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a step flowchart of the dual-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features of the application;

[0011] Figure 2 is a data flow direction schematic diagram of skeleton octree construction and skeleton bit stream generation in the specific embodiment of the application;

[0012] Figure 3 is a data flow direction schematic diagram of the dual-path three-dimensional point cloud compression reconstruction method fusing multi-scale geometric features of the application;

[0013] Figure 4 is a result schematic diagram of the simulation experiment of the application; DETAILED DESCRIPTION

[0014] In addition to the defects mentioned in the background, the defects of the prior art also include: the existing point cloud compression methods based on octree or sparse tensor fail to fully utilize the multi-layer spatial context information, usually adopt fixed voxel division, the neighborhood receptive field is limited, it is difficult to capture long-distance dependence, resulting in a decline in reconstruction quality at an extremely low bit rate, and poor adaptability to complex geometric structures; the main structure and local details of the point cloud lack effective joint modeling, most of the existing methods adopt a single path coding strategy, which cannot optimize the structure level and local accuracy at the same time, and is prone to cause low compression efficiency, more redundant information and low prediction accuracy; the existing voxel method usually relies on a large number of three-dimensional convolution operations or multi-head self-attention mechanism, the model structure is complex, the calculation overhead is high, the resource consumption is large, and the inference time is long, which is difficult to deploy in real-time processing scenarios.

[0015] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the application.

[0016] It should be noted that only parts related to the present application are shown in the drawings for the convenience of description. The embodiments in the present application and the features in the embodiments can be combined with each other in the case of no conflict.

[0017] It should be understood that the terms “system”, “apparatus”, “unit” and / or “module” used in the present application are a method for distinguishing different components, elements, parts, sections or assemblies of different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0018] As indicated in the present application and claims, unless the context clearly indicates otherwise, the words “one”, “an”, “a” and / or “the” do not specify a singular number, but can include a plural number. Generally, the terms “comprise” and “include” only indicate the inclusion of the steps and elements explicitly identified, and these steps and elements do not constitute an exclusive list, and the method or device can also include other steps or elements. The element defined by the phrase “comprising a” does not exclude the presence of additional identical elements in the process, method, product or device including the element.

[0019] In the description of the embodiments of the present application, “a plurality of” means two or more than two. The following terms “first”, “second” are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first”, “second” can explicitly or implicitly include one or more of the features.

[0020] In addition, flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or subsequent operations are not necessarily performed in sequence. On the contrary, each step can be processed in reverse order or simultaneously. Meanwhile, other operations can be added to these processes, or one or more steps of operation can be removed from these processes.

[0021] Reference Figure 1 The flowchart of an optional example of the fusion multi-scale geometric feature dual-path three-dimensional point cloud compression reconstruction method proposed in the present application can be applied to a computer device. The compression reconstruction method proposed in the present embodiment can include but is not limited to the following steps:

[0022] Step S1, skeleton anchor point extraction;

[0023] Step S2, skeleton octree construction;

[0024] Step S3, predicting the occupancy state of each node of the octree by using an anchor point encoder based on an attention mechanism;

[0025] Step S4, entropy encoding the octree structure of the skeleton anchor points based on the prediction result to generate a skeleton bitstream;

[0026] Step S5, local feature extraction and quantization;

[0027] Step S6, feature modeling and entropy encoding of the quantized features to generate a feature bitstream;

[0028] Step S7, point cloud reconstruction.

[0029] In some feasible embodiments, step S1 specifically includes:

[0030] In this embodiment, the original point cloud data is down-sampled to reduce the data size and improve the calculation efficiency.

[0031] The down-sampling operation is to select representative anchor points from the original point cloud to maintain the feature information of the overall spatial structure of the point cloud. The farthest point sampling algorithm is used to realize the down-sampling. This algorithm selects the point farthest from the sampled point set from the unsampled point set through iteration, so as to ensure that the sampling result is uniformly distributed in three-dimensional space and avoid local aggregation of the sampling points. The point set obtained through the above down-sampling is defined as the skeleton anchor point, which is used to represent the global structural features of the original point cloud.

[0032] The K nearest neighbors in the original point cloud construct the local neighborhood of the skeleton anchor point. K is the window size. The above neighborhood relationship provides local context information support for the anchor points for the subsequent modules.

[0033] In summary, the extracted skeleton anchor points and their local neighborhoods jointly constitute the input basis for the subsequent structure path and detail path encoding, supporting effective modeling of global structure and local details in the compression and reconstruction process.

[0034] In some feasible embodiments, step S2 specifically includes:

[0035] This embodiment constructs an octree structure based on the skeleton anchor points to realize the hierarchical organization and context window division of the point cloud space, and provides structural support for the subsequent encoding and context modeling. The above octree construction adopts a recursive space division method to divide the three-dimensional point cloud into multiple non-overlapping voxel blocks, and stores the information of each level in a tree structure.

[0036] Specifically, first, the entire point cloud is wrapped in a bounding box of a cube, i.e. a voxel, and the cube is taken as the root node. Then the voxel is recursively divided, and if the current voxel is not empty, it is divided into 8 sub-voxels of the same size. This process continues until the edge length of the leaf node voxel reaches the preset quantization accuracy.

[0037] For each non-leaf node voxel, it is marked as 1 for occupied or 0 for unoccupied according to whether its 8 child voxels contain point cloud data or not. Therefore, the occupancy state of each non-leaf node voxel is represented by an 8-bit binary code, and each bit in the code corresponds to the occupancy of a child voxel.

[0038] After the octree is constructed, each node contains the following attributes: the spatial center coordinates of the node, the hierarchical depth of the node, the index position of the node in the parent node, and the occupancy state of the parent node.

[0039] After the above attributes are flattened, they are input as node features into the octree anchor encoder network of the present application. The above network is used to predict the child voxel occupancy state distribution of each node, and the output is a probability vector of length 256, corresponding to all possible 8-bit binary occupancy combinations, and the sum of its elements is 1. This probability distribution is used to guide the subsequent arithmetic entropy coding to achieve efficient compression.

[0040] In addition, during the construction of the octree, the real occupancy state of each non-leaf node can be obtained synchronously, denoted as an integer array with a value range of [0, 255]. The above occupancy state label can be used as a supervision signal to guide the optimization of the skeleton encoder during the training process, and can also be used as an input basis for entropy coding for symbol coding in the actual compression process.

[0041] In some possible embodiments, step S3 specifically comprises:

[0042] S3.1, high-dimensional embedding expression of node embedding attributes.

[0043] The present application extracts 6-dimensional attributes including spatial coordinates, hierarchical depth, etc. as input features for each node in the octree. To enhance the model's ability to express the spatial characteristics of the node, the present application uses word embedding mapping technology to map the above 6-dimensional input features to a high-dimensional hidden space, usually 128-dimensional. The above word embedding mapping is a method of converting low-dimensional discrete or continuous features to high-dimensional dense vectors through a learnable matrix, which can extract more complex feature representation relationships. Through this technology, the model can more effectively capture the complex contextual relationships between nodes such as spatial directionality and local density, which are difficult to directly reflect from the original low-dimensional data.

[0044] S3.2, global structure modeling based on absolute position coding.

[0045] After obtaining the embedded features, in order to enhance the modeling ability of the model to the structural position information in the input node sequence, the application introduces an absolute position encoding mechanism. Specifically, for the embedded node feature sequence, a position vector corresponding to the node arrangement order is introduced, and the position encoding vector and the node feature vector are added element by element to form an enhanced feature representation containing position information. The above-mentioned method can explicitly provide the model with the absolute order information of each node in the whole sequence, so that the model has stronger global structure perception ability, which is helpful to capture the long-range dependence relationship and spatial organization rule between nodes.

[0046] S3.3, local position encoding mechanism based on convolution residual structure.

[0047] After introducing the absolute position encoding to enhance the global sequence perception, in order to further improve the modeling ability of the model to the local spatial structure relationship, the application introduces a local position encoding mechanism. The mechanism performs local modeling processing on the channel dimension of the embedded features in the node sequence in the form of convolution, aiming to capture the spatial proximity relationship between nodes and its local pattern features.

[0048] Specifically, each dimension of the embedded feature sequence is input to the convolution network as a one-dimensional signal, and the local structure pattern between adjacent nodes is extracted and encoded through the sliding window method without destroying the node arrangement order. The convolution operation has translational invariance, which can effectively enhance the perception ability of the model to the local geometric structure of the point cloud.

[0049] In order to preserve the original feature information and realize deeper feature fusion, the application introduces a residual connection structure in the local position encoding module, that is, the input features and the convolution output are added element by element, so as to realize feature enhancement while avoiding information loss. The above design helps to improve the expression ability of the model to the key spatial relationship in the local context and strengthens the modeling effect of the node-level features to the detailed structure of the point cloud.

[0050] S3.4, octree node occupancy state modeling based on multi-head attention and feedforward network.

[0051] S3.4.1, processing of the normalization layer.

[0052] In order to improve the stability of model training and prevent gradient vanishing or explosion, the application introduces a layer normalization before the input of each layer substructure. The above-mentioned method normalizes the distribution of each feature dimension, so that the mean and variance are kept consistent, thereby accelerating the convergence speed of the model and improving the overall training effect.

[0053] Specifically, the feature representation of a node is denoted as a feature vector with dimension d, and the normalization process includes the following steps: first, the mean value μ of all dimensions of the vector is calculated, which is used to describe the overall center position of the node feature; then the standard deviation σ is calculated to measure the fluctuation degree of each dimension feature value relative to the mean value, i.e. the dispersion. The standard deviation is defined as the square root of the average of the square of the difference between each dimension feature value and the mean value. The smaller the standard deviation, the more concentrated the feature distribution, and the more consistent the overall. Then the node feature is normalized by subtracting the mean value and multiplying by the standard deviation.

[0054] After the above standardization operation is completed, in order to improve the adaptability of the model to different tasks, two learnable parameters, a scaling factor γ and an offset factor β, are further introduced to perform linear transformation on the normalized result, so as to enhance the expression flexibility and generalization ability of the model. In order to avoid numerical instability problems such as zero standard deviation, a small positive number ε is introduced in the normalization calculation, so as to ensure the numerical stability of the whole process.

[0055] Through the above operation, layer normalization can effectively adjust the feature distribution and alleviate the distribution offset problem between different layers, thereby improving the stability and convergence speed of the neural network in the training process.

[0056] S3.4.2, multi-head self-attention mechanism processing.

[0057] Attention mechanism is used to model the correlation between each node and other nodes in the input sequence. In order to improve the modeling ability of the model for different semantic subspaces, the present application introduces a multi-head mechanism on the basis of the standard self-attention mechanism.

[0058] Specifically, the input features are sent into multiple parallel self-attention submodules, each submodule being an attention head, having independent linear transformation parameters to extract different context representations. For each attention head, the node feature vector X is mapped to the query vector (Q), the key vector (K) and the value vector (V) through different learnable linear mapping matrices W Q , W K , W V , respectively. This process can be formally represented as:

[0059] Q=XW Q ,K=XW K ,V=XW V

[0060] Where X is the input feature matrix, W Q , W K , W V are the corresponding linear transformation weight matrices, respectively.

[0061] The self-attention weight obtained above is obtained by scaled dot-product computation:

[0062]

[0063] where Attention(Q, K, V) denotes the computation result, Q, K, and V are query vector, key vector, and value vector, K T is the transpose of K, d k is the dimension of the value vector, is a scaling factor, which can avoid the result being too large to cause unstable gradient. Softmax is a normalization function, which is used to convert a real number vector into a probability distribution.

[0064] The outputs of multiple attention heads are concatenated in the feature dimension to form the final multi-head attention representation. The above design enables the model to capture information from multiple representation subspaces in parallel, improving the understanding of complex geometric context, while enhancing the diversity of model representation and training stability.

[0065] S3.4.3, processing of the feedforward network.

[0066] The feedforward network module combines a fully connected layer and a one-dimensional convolutional layer, aiming to enhance the non-linear representation ability and local structure perception ability of node features.

[0067] The fully connected layer, as a basic neural network structure, realizes global weighted combination and complex non-linear mapping of input features through linear transformation of input vector and weight matrix and bias addition combined with activation function. The activation function introduces non-linear transformation to improve the representation ability of the model and avoid the limitation of multi-layer network only representing linear functions.

[0068] The one-dimensional convolutional layer is mainly used for processing sequence data. It extracts local continuous feature patterns by weighted sum of features in local neighborhood through sliding convolution kernel, thereby strengthening the model's ability to capture local structures between adjacent nodes. The convolution operation slides in the sequence dimension, which has translational invariance and effectively improves the perception of local details.

[0069] Specifically, first, the input feature vector is linearly mapped to a high-dimensional space by the first fully connected layer to generate more rich feature representations; then, the mapped features are adjusted in shape to adapt to one-dimensional convolution operation (usually set the feature dimension as the channel dimension), and one-dimensional convolution is performed along the sequence dimension to capture the local features of adjacent nodes; finally, the convolution output is mapped back to the original dimension by the second fully connected layer to generate the final output features of the feedforward network.

[0070] This module combines linear transformation and local convolution, and has both global non-linear representation and local correlation modeling ability, effectively improving the encoding ability of the encoder to extract node features.

[0071] S3.4.4, multi-layer perceptron prediction.

[0072] After the above multi-head self-attention mechanism and local feedforward network fully fuse the node context information, the model has strong global modeling and local feature expression ability. To realize accurate prediction of the occupancy state of the octree node, the present application designs a multi-layer perceptron containing two layers of structure as an output module.

[0073] First, the node feature sequence is input into the first layer of the multi-layer perceptron for nonlinear transformation to enhance its abstract expression ability; then, the intermediate features are mapped from the hidden space to the final output space in the second layer of the perceptron to generate a probability vector with a length of 256. The vector corresponds to 256 possible voxel occupancy state combinations of each non-leaf node of the octree, and the sum of all elements is 1, satisfying the probability distribution constraint. The probability distribution is used to describe the probability mode of generating the next layer of eight child nodes for each node, and is the direct input basis for the subsequent entropy coding stage. Through deep feature learning and end-to-end modeling, the model can accurately predict the real occupancy state of each node on the basis of context modeling, providing an efficient and accurate statistical prior for compression.

[0074] In some possible embodiments, step S4 specifically comprises:

[0075] Based on the prediction result, the octree structure of the skeleton anchor point is entropy coded to generate a skeleton bitstream.

[0076] In the compression process of the present embodiment, the anchor point encoder models the occupancy state of each octree node and outputs a probability distribution vector with a length of 256 to constitute a probability distribution array corresponding to the node. The distribution is used to represent the prediction probability of the node under 256 possible occupancy states and can be directly input into the arithmetic entropy encoder.

[0077] During the octree construction process, the real occupancy state of each non-leaf node is represented by an integer with a value range of [0, 255], and each bit of the binary representation corresponds to the occupancy of a child voxel. The occupancy states of all nodes form an occupancy state array in sequence.

[0078] To realize efficient compression of the array, the present application uses an arithmetic entropy coding method to encode the real occupancy state node by node according to the aforementioned prediction probability distribution. The process is based on the initial coding interval [0, 1), and the coding interval is recursively scaled in node order. At each step, the interval is updated according to the cumulative probability boundary of the current symbol in the corresponding probability model, and finally the entire sequence is mapped to a unique real number in a certain sub-interval, realizing the generation of a skeleton bitstream with extremely high compression efficiency.

[0079] Arithmetic entropy coding is a lossless compression method based on probability model, which is different from traditional Huffman coding. Instead of assigning discrete code words to individual symbols, it encodes the entire symbol sequence as a whole in continuous intervals. Since this method can more accurately depict the probability structure in non-integer entropy scenarios, it is theoretically closer to the source entropy limit, and is particularly suitable for encoding tasks with uneven symbol probability distribution or strong context dependence.

[0080] To further illustrate the process, the following examples are explained: Assume that the true occupancy states of the first four nodes of the first layer in the occupancy state array are known, and the corresponding probability distribution array has been output by the model. The initial encoding interval is set to [0, 1). Take the first node as an example, its occupancy state is 17 (binary is 00010001), which means that its 0th and 4th child nodes are occupied. If the probability of state 17 in the prediction probability is 0.6, and the cumulative probability of states 0-16 is 0.3, then the encoding interval corresponding to this node is [0.3, 0.9). Continue to process the second node, based on the current interval [0.3, 0.9), further narrow the interval according to its true state and probability distribution (such as to [0.54, 0.72)). Repeat this process, and perform the same operation on all nodes in turn, and the final encoding interval will gradually converge. Any real number in the final interval (such as the upper bound) can be selected as the bit stream representation, thereby completing the overall compression encoding process.

[0081] In steps S2-S4, the skeleton anchor points are first constructed by octree coding to build a hierarchical structure, and the spatial consistency is enhanced by non-overlapping window division, then the node occupancy state is predicted by the anchor point encoder, and finally the skeleton bit stream is generated by entropy coding. The overall processing process is referred to Figure 2 .

[0082] In some possible embodiments, step S5 specifically includes:

[0083] A multi-scale local window is constructed by the skeleton anchor points, and the features in the local window are sent to the graph attention local feature extractor. Through multi-layer graph convolution and attention mechanism, multi-scale geometric context information is gradually aggregated, so as to extract local feature representation with rich expression ability.

[0084] S5.1, local window alignment.

[0085] To improve the comparability and processing consistency of different local area features, the present application performs a normalization alignment operation on each local window, which is centered on the anchor point in space and unified in scale. Specifically, it includes the following two steps:

[0086] Position centering: Taking the anchor point of the skeleton corresponding to the window as the reference, the coordinates of all points in the local region are subtracted from the anchor point position, so as to realize the alignment centered on the anchor point. This processing can effectively eliminate the interference of global displacement on feature learning and enhance the attention of the model to local geometric structure.

[0087] Scale normalization: The average nearest neighbor distance of the anchor point set is calculated, and the coordinates of all points in the window are divided by the scale factor to realize the scale consistency. This step helps to eliminate the scale difference between different regions and avoid affecting the stability and convergence of model training due to large scale changes.

[0088] Through the above local window alignment operation, each window is mapped to a unified center and scale space, which facilitates subsequent model alignment, comparison and fusion of multi-window features, thereby enhancing the consistency and generalization ability of feature learning.

[0089] S5.2, multi-scale window construction.

[0090] To enhance the multi-scale perception ability of the model to the local structure of the point cloud, the present application divides the normalized local region into two sub-windows with complementary resolutions based on local window alignment using an odd-even interval sampling strategy. Specifically, the odd-indexed points and even-indexed points in the local point set are alternately selected in the manner of "sampling one point every two intervals" to construct two sub-sampling windows.

[0091] The above two sub-windows are respectively input into a structure-consistent feature extraction network to encode their local geometric features through multi-layer graph convolution and attention mechanism. Subsequently, the two feature vectors obtained by extraction are respectively subjected to global representation through maximum pooling operation, and are spliced in the feature dimension to form fused multi-scale local features.

[0092] The double-path window construction method has structural complementarity and can capture the geometric changes and topological relationships of the local region from different granularity levels, significantly improving the expression ability and robustness of the model to complex point cloud structures.

[0093] S5.3, local feature extraction based on multi-layer graph convolution.

[0094] To fully exploit the local geometric structure information of the point cloud, the present embodiment proposes a local feature extraction method based on multi-layer graph convolution and attention mechanism. This method effectively improves the expression ability and generalization performance of the model to complex structures through multi-scale neighborhood modeling and feature fusion.

[0095] S5.3.1, K-neighbor extraction and multi-scale modeling.

[0096] In point cloud and graph neural network tasks, K-Nearest Neighbor search is a key step to construct graph structure. By extracting the nearest K neighbors for each point, local connection relationship can be constructed, which is convenient for subsequent feature aggregation and structure modeling. The present application adopts step set {1, 2, 4} to perform K-Nearest Neighbor search, thereby constructing multi-scale neighborhood windows. These windows are input into different levels of feature extraction networks to capture geometric feature information at different granularities, and finally fused and integrated in the feature dimension.

[0097] S5.3.2, input encoding of geometric attribute enhancement.

[0098] To improve the expression ability of the model on local geometric structure, the present application performs enhancement processing on the input features. Specifically, before entering each layer of the feature extraction network, the following information is spliced in the last dimension: the original features of the neighbor points; the spatial coordinates of the neighbor points; the relative position difference (L1 norm) between the center point and the neighbor points; the Euclidean distance (L2 norm) between the center point and the neighbor points.

[0099] The L1 norm (Manhattan distance) is the sum of the absolute values of the coordinate differences, which can reflect the local changes in each dimension; the L2 norm (Euclidean distance) emphasizes the overall distance in space, which is suitable for global scale geometric modeling. Through the splicing method of geometric enhancement, the model can obtain richer semantics and spatial priori in the feature space, thereby improving the modeling ability of complex neighborhood relationships.

[0100] S5.3.3, local feature aggregation.

[0101] In each layer of the feature extraction network, the spliced input data is first sent to the first multi-layer perceptron block, and after layer-by-layer linear transformation and activation function processing, rich neighbor features are extracted. Then, for each center point, its corresponding features are extracted, and the center point features and neighbor features are spliced as the input of the attention mechanism and input into the second multi-layer perceptron block. Through the attention module, the network can dynamically calculate the neighbor weights based on the relationship between the neighbors and the center point. Finally, the neighbor features are weighted using the calculated attention weights, and the sum is calculated along the neighbor dimension to obtain the aggregated feature representation, realizing effective encoding of local neighborhood information. This design extracts neighbor features layer by layer, combines center point feature input attention module, so that the network can adaptively assign neighbor weights, emphasizing neighbor information with stronger relevance to the center point. Weighted summation realizes effective aggregation of neighbor features, improves the accuracy and discriminability of local feature expression, thereby enhancing the model's ability to capture local geometry and semantic relationships in point cloud or graph structure, and promoting the performance improvement of downstream tasks (such as classification, segmentation, compression, etc.).

[0102] S5.3.4, multi-layer feature fusion and embedding output.

[0103] The output features of all layer feature extraction networks are spliced in the feature dimension, fusing multi-scale neighborhood information extracted at different levels. In this way, not only the detailed features of the shallow layer are retained, but also the abstract semantics of the deep layer are combined, enhancing the richness and discriminability of feature expression, thereby improving the modeling capability of the model for complex geometric relationships in point clouds or graph structures.

[0104] The spliced multi-layer features are linearly transformed and mapped through a fully connected layer to obtain the final embedding feature representation. This step integrates the fused multi-scale information into a unified feature vector, providing efficient and discriminative input for subsequent tasks.

[0105] S5.3.5, max-pooling and quantization operation.

[0106] The max-pooling operation is used to downsample the feature tensor. Max-pooling selects the maximum value in each local neighborhood, effectively retaining the most representative response in the region and enhancing the model's ability to perceive key features. It helps to compress the feature dimension, reduce computational overhead, and has certain translation invariance and noise resistance, which can improve the robustness and generalization ability of the model.

[0107] To further compress data and reduce storage and transmission costs, a quantization operation is introduced based on the continuous features obtained by max-pooling. This process maps continuous high-precision feature values to a predefined set of discrete numerical values, obtaining a discrete representation that is easy to encode. Specifically, the quantization process can be viewed as dividing the feature space into intervals and representing all feature values in each interval with the center value of that interval, thereby achieving a trade-off between numerical precision and information redundancy. This operation not only significantly reduces the data size, but also provides a basis for subsequent entropy coding or bit rate estimation, making it suitable for point cloud compression, low bit rate representation, and other application scenarios.

[0108] In some possible embodiments, step S6 specifically includes:

[0109] The present embodiment discloses a local geometric feature modeling and entropy coding method based on skeleton anchor points, i.e., a skeleton conditional feature encoder. This method uses the spatial distribution features of skeleton anchor points and their neighborhood anchor points to construct multi-scale context information, which is used to guide the probability distribution modeling of quantized local geometric features. The neighborhood anchor point distribution of the above-mentioned skeleton anchor points reflects the spatial topological information of the sparse skeleton structure, and then approximately describes the geometric distribution features of the local window of the real point cloud. Based on this context information, the system performs entropy coding on the local geometric features, significantly improving the feature compression efficiency and coding performance.

[0110] S6.1、Firstly, the system takes the set of skeleton anchor points extracted from the point cloud as input, which constitutes the global topological skeleton of the point cloud. Based on this set of anchor points, the system extracts context information in a multi-scale range and constructs probability distribution parameters suitable for compression encoding.

[0111] S6.2、To achieve efficient probability modeling, the system designs a distribution parameter prediction network based on graph attention convolution. This network maps the high-dimensional semantic features of each skeleton anchor point into two key parameters: one is the mean vector of the probability distribution, and the other is the scale parameter vector representing the distribution range. Among them, the scale parameter is positively constrained by the exponential function, ensuring that the model output conforms to the mathematical definition of the probability distribution. This network fully integrates the spatial information of the anchor points and their neighborhood anchor structure, thereby enhancing the context modeling capability of the feature representation.

[0112] S6.3、In the probability modeling stage, the system introduces a Laplace mixture distribution to model the quantized features of the skeleton anchor points. For each dimension of each anchor point, the system calculates its probability density value under the Laplace distribution based on the predicted mean and scale parameters. Further, the system uses the cumulative distribution function of this probability density function to discretize the continuous distribution, obtaining a discrete probability distribution suitable for arithmetic entropy encoders.

[0113] S6.4、In the feature encoding stage, the system inputs the above discretized probability distribution into the arithmetic entropy encoder. The encoder dynamically allocates bit resources to each quantized feature component based on the distribution and compresses it into a compact bit stream representation. Compared with traditional encoding methods with fixed bit rate, this scheme fully utilizes the prior probability of feature values, achieving more fine-grained bit resource allocation and compression efficiency optimization.

[0114] In addition, since the skeleton anchor points integrate the local geometric structure information of the original point cloud in their neighborhood during extraction, the probability distribution learned by the system in the joint modeling process of anchor points and their neighborhood anchor points can accurately reflect the statistical characteristics of the actual point cloud local structure. Therefore, this probability modeling mechanism not only improves the encoding efficiency, but also enhances the reconstructability of local geometric features.

[0115] Finally, the feature bit stream generated by the above process can be used as the feature decoding input in point cloud reconstruction, providing high-fidelity semantic support for subsequent dense point cloud restoration.

[0116] In some feasible embodiments, step S7 specifically includes:

[0117] S7.1、Skeleton recovery.

[0118] At the beginning of the decoding stage of the skeleton bitstream, the application first loads the root node initialization information generated in the encoding process. The information includes the center coordinates of the root node in the three-dimensional space, the spatial range size of the root node, and the occupation state marker of the eight child nodes corresponding to the root node. The application constructs the root node data structure of the octree based on the above parameters, and based on the root node, recursively completes the decoding and reconstruction of the multi-layer octree from top to bottom.

[0119] In the specific decoding process, the application starts from the root node and expands the octree nodes layer by layer using a layer sequence strategy. For each layer, the application first unfolds all the nodes of the current layer in turn to generate corresponding child nodes. Then, the application extracts the node features of all the child nodes of the layer. The node features include the center coordinates of the three-dimensional space where the node is located, the spatial range size, the level to which the node belongs, and the occupation state marker of the parent node. After the extraction is completed, the application constructs all the node features of the current layer into a fixed-length sequence. If the number of nodes is insufficient, zero padding is performed. Then, the application inputs the sequence into the pre-trained neural network model for forward reasoning to output the occupation state probability distribution of each node. Finally, the application decodes the skeleton bitstream using an arithmetic entropy decoder in combination with the occupation state probability distribution to obtain the actual occupation state marker of each node.

[0120] The above process is repeated from top to bottom until the preset maximum depth of the octree is reached, thereby obtaining a complete octree structure. At the bottom layer of the octree, the application extracts the spatial center coordinates of all the leaf nodes as the reconstructed skeleton point cloud data, and restores the point cloud skeleton coordinates in the original three-dimensional space scale through the inverse normalization operation consistent with the encoding stage.

[0121] S7.2, feature decoding.

[0122] After the skeleton is recovered, the application constructs a multi-scale neighborhood index for each skeleton point and inputs it into the trained entropy modeling network to predict the probability distribution parameters of the corresponding local window, including the mean and scale parameters of the Gaussian distribution. Based on the above probability distribution, the application decodes the feature bitstream using an arithmetic entropy decoder to obtain the quantized local geometric features corresponding to each skeleton point.

[0123] S7.3, geometry fusion point cloud upsampling.

[0124] First, the quantized local geometric features obtained by decoding are input into a multi-layer graph convolutional network. The above graph convolutional network combines the three-dimensional spatial coordinates of the skeleton points and the topological structure of their local neighborhood to complete cross-level feature extraction and fusion, thereby obtaining a high-dimensional embedding representation containing rich spatial semantics.

[0125] After obtaining the high-dimensional embedding features described above, the application introduces a learnable coordinate generation module for generating a plurality of reconstruction sub-points centered on each skeleton point, realizing the local detail recovery and density expansion of the point cloud. Specifically, the coordinate generation module first maps the high-dimensional embedding to a higher dimension through a multi-layer perception machine, and combines a learnable folding mapping mechanism to up-sample the features, to preliminarily generate a plurality of offset vectors relative to the skeleton points. Subsequently, the application replicates and expands the original embedding features along the point dimension, and splices and fuses the offset features, as input to enter the subsequent multi-layer perception machine and up-sampling module again, to further generate more detailed local point cloud structures.

[0126] After completing the window-level point cloud generation, the application performs a coordinate anti-alignment operation, that is, applies an anti-normalization operation consistent with the encoding stage to the generated coordinates, to restore the position and scale of each local window in the original space. Through the above processing, high-fidelity laser radar point cloud data recovery is finally realized.

[0127] Based on the above method, the purpose of the application is to realize high-fidelity reconstruction of point cloud data at a high compression rate by constructing a dual-path laser radar point cloud compression framework that integrates multi-scale geometric features, introducing an anchor point encoding network based on an octree structure, a graph-driven multi-scale feature aggregation mechanism, and a structure prior-guided context modeling module, thereby improving compression efficiency and reconstruction accuracy. The overall data flow is shown in Figure 3 .

[0128] The main difficulty of current point cloud compression technology lies in how to effectively model and jointly optimize global structure information and local detail features under extremely low bit rate conditions, while ensuring the control of compression overhead and computing resources. The application hopes to start from the collaborative expression of skeleton structure and local context, and optimize the global structure-local detail modeling capability through a split-path processing strategy.

[0129] Existing methods generally adopt a unified path compression strategy, which is difficult to model point cloud structure hierarchy and detail information, and has problems such as insufficient context utilization and reconstruction distortion. The application introduces an octree anchor point encoding module based on an attention mechanism, which models the long-range dependency relationship between structures through a multi-layer self-attention mechanism, thereby effectively predicting the occupancy state of the octree node.

[0130] Existing methods often lack an efficient context aggregation strategy when modeling point cloud details, and cannot balance local details and global consistency. The application proposes a graph convolution driven local feature embedding network and an expanded residual graph attention module, which integrates feature information under different receptive fields through a multi-scale context modeling mechanism, to enhance the geometric modeling capability of complex regions.

[0131] The existing method lacks a guided modeling strategy when compressing point clouds, resulting in high feature redundancy and low compression efficiency. The present application constructs a skeleton conditional feature encoder, uses skeleton prior information to predict local feature distribution, and compresses global structure information and local geometric features through a structure-detail decoupled dual-path architecture, thereby effectively reducing redundant information encoding.

[0132] The existing method lacks a guided modeling strategy when compressing point clouds, resulting in high feature redundancy and low compression efficiency. The present application constructs a skeleton conditional feature encoder, uses skeleton prior information to predict local feature distribution, and compresses global structure information and local geometric features through a structure-detail decoupled dual-path architecture, thereby effectively reducing redundant information encoding.

[0133] In summary, the present application realizes the overall optimization of point cloud compression and reconstruction accuracy, efficiency and complexity through the organic synergy of dual-path encoding, structure guidance, graph convolution enhancement and local refinement reconstruction modules, meeting the high-quality, low-bandwidth and high-efficiency application requirements of laser radar point clouds in autonomous driving and other scenarios.

[0134] Based on the above method, the present application also gives related simulation examples:

[0135] Compression performance evaluation was carried out on two public laser radar point cloud datasets, SemanticKITTI and Ford. Figure 4 The code rate-distortion curves of each method are shown, and the evaluation indicators include D1 PSNR, D2 PSNR and Chamfer distance (CD).

[0136] Each point bit rate (bpp), an indicator of compression efficiency, represents the average number of bits used after encoding for each point. The smaller the value, the higher the compression rate and the lower the storage and transmission cost.

[0137] D1 PSNR (point-to-point peak signal-to-noise ratio), used to measure the distance error between the original point cloud and the reconstructed point cloud. The specific method is to find the nearest point in the reconstructed point cloud for each original point, and then calculate the average error of these distances and convert it into a peak signal-to-noise ratio (PSNR) value. The larger the value, the more accurate the reconstruction.

[0138] D2 PSNR (point-to-surface peak signal-to-noise ratio), which considers the local surface structure of the point cloud. The reconstructed point cloud is constructed into a mesh, and the distance from the original point to the surface is evaluated. Compared with D1 PSNR, it pays more attention to surface smoothness and structure preservation ability.

[0139] Chamfer distance (CD), used to measure the overall geometric difference between two sets of point clouds. This indicator calculates the nearest point distance from both the original point cloud and the reconstructed point cloud, and takes the average. The smaller the value, the closer the shapes of the two.

[0140] A dual-path three-dimensional point cloud compression reconstruction system fusing multi-scale geometric features comprises:

[0141] A skeleton anchor point encoding unit is configured to perform steps S1-S4.

[0142] A local window encoding unit is configured to perform steps S5 and S6.

[0143] A reconstruction unit is configured to perform step S7.

[0144] The contents in the above method embodiments are all applicable to the system embodiments, the system embodiments specifically realize the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0145] A dual-path three-dimensional point cloud compression reconstruction device fusing multi-scale geometric features comprises:

[0146] At least one processor;

[0147] At least one memory is configured to store at least one program;

[0148] When the at least one program is executed by the at least one processor, the at least one processor implements the above method for fusing multi-scale geometric features in a dual-path three-dimensional point cloud compression reconstruction method.

[0149] The contents in the above method embodiments are all applicable to the device embodiments, the device embodiments specifically realize the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0150] A storage medium has processor-executable instructions stored therein, and the processor-executable instructions are used to implement the above method for fusing multi-scale geometric features in a dual-path three-dimensional point cloud compression reconstruction method when executed by a processor.

[0151] The contents in the above method embodiments are all applicable to the storage medium embodiments, the storage medium embodiments specifically realize the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0152] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A dual-path 3D point cloud compression reconstruction method fusing multi-scale geometric features, characterized in that, The method comprises the following steps: extracting skeleton anchor points based on original point cloud data; constructing an octree structure based on the skeleton anchor points to obtain a skeleton anchor point octree; using an anchor point encoder based on an attention mechanism to predict the occupancy state of each node in the skeleton anchor point octree and to encode, to obtain a skeleton bit stream; constructing a multi-scale local window based on the skeleton anchor points; processing the original point cloud data based on the multi-scale local window to obtain preliminary features; processing the preliminary features based on a graph attention local feature extractor to obtain quantized features; encoding the quantized features based on a skeleton conditional feature encoder to obtain a feature bit stream; decoding and reconstructing based on the skeleton bit stream and the feature bit stream to obtain recovered point cloud data.

2. The method of claim 1, wherein the method further comprises: The step of constructing an octree structure based on the skeleton anchor points to obtain a skeleton anchor point octree specifically comprises: using a recursive spatial division method to divide the skeleton anchor points into a plurality of non-overlapping voxel blocks and storing hierarchical information in a tree structure to obtain a skeleton anchor point octree.

3. The method of claim 1, wherein the method further comprises: The step of using an anchor point encoder based on an attention mechanism to predict the occupancy state of each node in the skeleton anchor point octree and to encode, to obtain a skeleton bit stream specifically comprises: performing feature extraction and word embedding mapping processing based on each node in the skeleton anchor point octree to obtain embedded features; introducing an absolute position encoding mechanism and a local position encoding mechanism to model and process the embedded features to obtain position-encoded features; processing the position-encoded features based on a multi-attention mechanism and a feedforward network to obtain a prediction result; node-by-node encoding the occupancy state of each node in the skeleton anchor point octree based on the prediction result to obtain a skeleton bit stream.

4. The method of claim 3, wherein the method further comprises: The step of processing the position-encoded features based on a multi-attention mechanism and a feedforward network to obtain a prediction result specifically comprises: processing the position-encoded features based on a plurality of parallel self-attention submodules and concatenating in the feature dimension to obtain multi-head attention representations; processing the multi-head attention representations based on a feedforward network with one-dimensional convolution to obtain the output of the feedforward network; processing the output of the feedforward network based on a multi-layer perceptron to obtain a prediction result.

5. The method of claim 4, wherein the method further comprises: Further comprising: introducing a normalization layer before the plurality of parallel self-attention submodules and the feedforward network with one-dimensional convolution.

6. The method of claim 1, wherein the method further comprises: The step of processing the preliminary features based on a graph attention local feature extractor to obtain quantized features specifically comprises: enhancing the preliminary features through a geometry-enhanced concatenation method to obtain concatenated features; performing feature aggregation based on the concatenated features to obtain aggregated features; performing maximum pooling and quantization operations on the aggregated features to obtain quantized features.

7. The method of claim 1, wherein the method further comprises: The step of encoding the quantized features based on a skeleton conditional feature encoder to obtain a feature bit stream specifically comprises: constructing a probability distribution parameter based on the skeleton anchor points; introducing a Laplace mixture distribution to model the quantized features based on the probability distribution parameter to obtain a discrete probability distribution; Entropy encoding the quantized features based on the discrete probability distribution to obtain a feature bitstream.

8. The method of claim 1, wherein the method further comprises: The step of decoding reconstruction based on the skeleton bitstream and the feature bitstream to obtain the recovered point cloud data specifically comprises: restoring a sparse skeleton point set based on the skeleton bitstream; combining the coefficient skeleton set to recover local features from the feature bitstream and performing fine-grained reconstruction on the point cloud through a learnable geometric fusion point cloud upsampling module to obtain the recovered point cloud data.

9. A dual-path 3D point cloud compression reconstruction system fusing multi-scale geometric features, characterized in that, It comprises: a skeleton anchor point encoding unit that extracts skeleton anchor points based on original point cloud data; constructing an octree structure based on the skeleton anchor points to obtain a skeleton anchor point octree; using an anchor point encoder based on an attention mechanism to predict the occupancy state of each node in the skeleton anchor point octree and encode it to obtain a skeleton bitstream; a local window encoding unit that constructs a multi-scale local window based on the skeleton anchor points; processing the original point cloud data based on the multi-scale local window to obtain preliminary features; processing the preliminary features based on a graph attention local feature extractor to obtain quantized features; encoding the quantized features to obtain a feature bitstream; a reconstruction unit that performs decoding reconstruction based on the skeleton bitstream and the feature bitstream to obtain the recovered point cloud data.

Images