Point cloud context neighborhood selection method for entropy model
By optimizing the neighborhood selection method in learning-based point cloud compression, and employing checkerboard grouping and 1-swap replacement techniques, the redundant neighbor problem caused by fixed geometric rules is solved, improving the prediction accuracy and compression efficiency of the entropy model. This method is suitable for end-to-end point cloud compression systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-02-11
- Publication Date
- 2026-05-05
AI Technical Summary
In existing learning-based point cloud compression, the context neighborhood constructed by fixed geometric rules is prone to introducing redundant or mismatched neighbors, which leads to reduced prediction accuracy and increased bit rate of the entropy model, and the neighborhood construction lacks an optimization mechanism.
Point cloud data is divided using a checkerboard grouping rule. Candidate neighbor sets are retrieved in the decoded first coding feature space, and the neighbor sets are optimized by a 1-swap replacement operation to form a final neighbor set, which is used for conditional probability prediction and entropy coding of the second coding features.
It improves the prediction accuracy of the entropy model, reduces the bitrate, and maintains neighborhood consistency between the encoder and decoder without additional bitstream overhead. It is suitable for practical deployment and is compatible with existing backbone encoders and entropy model structures.
Smart Images
Figure CN121985129A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of learning-based point cloud compression and entropy coding probabilistic modeling technology, and is applied in end-to-end point cloud compression encoding and decoding systems. Specifically, it relates to a point cloud context neighborhood selection method for entropy models. Background Technology
[0002] Point clouds are typically composed of geometric coordinates and attributes (such as color). They are large in scale, sparse, and locally irregular, making compression difficult. Traditional point cloud compression standards (such as schemes based on octrees, prediction, transformation, and entropy coding) often require a trade-off between bit rate and complexity in complex scenarios.
[0003] In recent years, learning-based point cloud compression has increasingly adopted end-to-end networks and probabilistic models for encoding. A typical framework includes: an encoder / decoder obtains latent variable representations, combines a hyperprior and a context model to predict the conditional probability of the latent variable symbols, and then uses arithmetic encoding to achieve bit output. To improve the accuracy of probabilistic prediction, existing methods typically construct contextual neighborhoods (e.g., based on KNN, octree neighborhoods, or fixed geometric rules) and input the neighbor features of the decoded regions into a contextual aggregation network.
[0004] However, neighborhoods constructed using fixed geometric rules have inherent limitations: the same geometric proximity does not equate to being "useful" for predicting the probability of the target symbol. Weakly correlated, redundant, or mismatched neighbors are often mixed into the neighborhood, causing the conditional distribution to become "scattered," thus increasing the bit rate. More importantly, in many parallelized entropy models (such as grouping / chessboard modeling), the source of the neighborhood is often limited to the already decoded groups, making neighborhood construction more prone to being "fixed by default" and lacking optimizable mechanisms. Summary of the Invention
[0005] To address the problem that existing learning-based point cloud compression methods directly determine the context neighborhood using fixed geometric rules, which is not optimizable and introduces redundant / mismatched neighbors, thus reducing the prediction accuracy of entropy models, this invention proposes a point cloud context neighborhood selection method for entropy models. Applied to a learning-based point cloud compression system, this method transforms point cloud data into latent features and quantizes them to obtain latent variables. These latent variables are divided into first-coded features and second-coded features according to a checkerboard grouping rule. The first-coded features are entropy-encoded first and then recovered at the decoding end, serving as decoded information. The conditional probability of the second-coded features is jointly determined by the decoded first-coded feature's context neighborhood features and the prior information. The process of retrieving neighborhood features of the second-coded features from the decoded feature space of the first-coded features includes:
[0006] 101. For any position n to be encoded in the second encoded feature, a candidate neighbor set C is obtained in the feature space of the decoded first encoded feature according to the deterministic geometric nearest neighbor rule.n The size of the candidate neighbor set is K. cand ;
[0007] 102. Select K from the candidate neighbor set kept Each element is used as the retained neighbor set S. n Perform T 1-swap replacement operations on the set to obtain the final neighbor set S' of the position to be encoded n;
[0008] 103. During the t-th 1-swap replacement, from complement C... n \S t-1 The set of neighbors S obtained by selecting one element from the previous 1-swap replacement is called the set of neighbors that is retained. t-1 The elements in the set minimize the local bitrate change, and the complement C n \S t-1 Let the candidate neighbor set C be n In addition to retaining subset S t-1 A set consisting of elements other than those mentioned above;
[0009] 104. The elements in the final neighbor set S' are used as the neighborhood context input of the position n to be encoded in the second encoded feature, for conditional probability prediction of the second encoded feature and entropy encoding / entropy decoding.
[0010] This invention also proposes a point cloud context neighborhood selection system for entropy models, which implements a point cloud context neighborhood selection method for entropy models, including an offline module and an online module, wherein:
[0011] The offline module is used to collect samples for training. Each sample includes features and labels. The features of the sample consist of the feature information of the retained subset and the discarded subset of each encoded position, the geometric relationship features between the retained subset and the discarded subset and the current encoding position n, and the prior side information. The neighbor set obtained through steps 101 to 104 is used as the label.
[0012] The online module is used to train the perceptron network composed of multilayer perceptrons based on the sample data obtained by the offline module, and then predicts the label of the trained perceptron network based on the features of the location to be encoded.
[0013] The present invention also proposes a computer device, the device comprising:
[0014] Memory;
[0015] processor;
[0016] The memory stores computer-executed instructions;
[0017] The processor executes computer execution instructions stored in the memory to implement a point cloud context neighborhood selection method for an entropy model.
[0018] The present invention also proposes a computer storage medium storing computer execution instructions, which, when executed by a processor, are used to implement a point cloud context neighborhood selection method for an entropy model.
[0019] Compared with the prior art, the present invention has at least the following beneficial effects:
[0020] 1. Improve the predictive concentration of the entropy model and reduce the code rate: This invention reduces the uncertainty of conditional distribution by removing redundant / mismatched neighbors from the candidate pool, thereby reducing the number of encoded bits.
[0021] 2. Zero signaling, consistent deployment at both ends: In the online phase, this invention adopts deterministic iteration and a scorer input that relies only on publicly available information. The encoder and decoder ends can reproduce the same neighborhood subset without additional bitstream overhead.
[0022] 3. Controllable complexity and effective with few steps: This invention employs 1-swap and local refinement with a small number of iterations (TTT) to avoid combinatorial explosion; in practice, the main benefits can be obtained with fewer steps, making it suitable for actual deployment.
[0023] 4. Strong compatibility: This invention does not change the overall structure of the backbone encoder / decoder and the existing entropy model, and can be integrated into various learning-based point cloud compression frameworks as a plug-in module of the context path. Attached Figure Description
[0024] Figure 1 This is a flowchart of a point cloud context neighborhood selection method for an entropy model according to the present invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] This invention proposes a point cloud context neighborhood selection method for entropy models, applied to a learning-based point cloud compression system. This system transforms point cloud data into latent features and quantizes them to obtain latent variables. The latent variables are divided into first-coded features and second-coded features according to a checkerboard grouping rule. The first-coded features are entropy-encoded first and then recovered at the decoding end, serving as decoded information. The conditional probability of the second-coded features is jointly determined by the decoded first-coded feature's context neighborhood features and the prior information. Figure 1 The process of retrieving neighborhood features of the second encoded feature in the feature space where the first encoded feature has been decoded includes:
[0027] 101. For any position n to be encoded in the second encoded feature, a candidate neighbor set C is obtained in the feature space of the decoded first encoded feature according to the deterministic geometric nearest neighbor rule. n The size of the candidate neighbor set is K. cand ;
[0028] 102. Select K from the candidate neighbor set kept Each element is used as the retained neighbor set S. n Perform T 1-swap replacement operations on the set to obtain the final neighbor set S' of the position to be encoded n;
[0029] 103. During the t-th 1-swap replacement, from complement C... n \S t-1 The set of neighbors S obtained by selecting one element from the previous 1-swap replacement is called the set of neighbors that is retained. t-1 The elements in the set minimize the local bitrate change, and the complement C n \S t-1 Let the candidate neighbor set C be n In addition to retaining subset S t-1 A set consisting of elements other than those mentioned above;
[0030] 104. The elements in the final neighbor set S' are used as the neighborhood context input of the position n to be encoded in the second encoded feature, for conditional probability prediction of the second encoded feature and entropy encoding / entropy decoding.
[0031] This invention addresses the "context entropy modeling" stage in learning-based point cloud compression (neural network compression). (For example, in the existing document "Joint Geometry-Attribute Point Cloud Compression with Spatial ContextMining and Dual-Class Attribute Loss," this invention can be applied before Spatial ContextMining in that document, selecting certain features as decoded information to perform subsequent operations on that location for encoding and decoding.) It proposes a technical solution to optimize the neighborhood context used for conditional probability prediction without altering the original backbone network structure. This solution maintains completely consistent decision results between the encoder and decoder, eliminating the need to additionally transmit information about "which neighbors were selected" in the bitstream, thus stably reducing the number of encoded bits and improving compression efficiency.
[0032] In learning-based compression, to further reduce the bitstream size, an "entropy model" (also called a probabilistic model) is typically added to predict the most likely value for each symbol to be encoded, allowing the entropy encoder to complete the encoding with fewer bits. The more accurate the entropy model, the fewer bits are needed for encoding. To improve prediction accuracy, the entropy model uses already encoded and decoded information as "contextual clues," which is called contextual entropy modeling. Intuitively, this means that before encoding a symbol, it refers to nearby already encoded content to help predict the current symbol.
[0033] Many advanced methods employ a "group coding" approach to improve parallelism: the quantized latent features are divided into two groups, denoted as the first coded feature y1 and the second coded feature y2. The system first encodes and decodes the first coded feature y1, and then uses the decoded first coded feature y1 as side information to assist in predicting the coding relationship of the second coded feature y2, thus saving more bits when encoding the second coded feature y2. At the same time, the system usually generates a super-prior side information h (which can be understood as a global statistical cue predicted by an additional network) to further assist in the probability prediction of y2. In other words, the probability prediction of the second coded feature y2 usually depends on two types of information: (1) the neighborhood features related to the current position in the decoded first coded feature y1; and (2) the super-prior side information h.
[0034] Existing technologies mostly employ fixed geometric rules, such as KNN (K-Nearest Neighbors), when constructing the neighborhood context of the second encoded feature y2. This involves finding the closest neighbors to the current position in the first encoded feature y1 and then directly inputting these K neighbors into a context aggregation network to obtain the conditional probability distribution of the second encoded feature y2. While this method is simple to implement, it has significant drawbacks: selecting neighbors solely based on geometric distance can easily include neighbors that appear close but offer little or no help in prediction, or even interfere with it. These redundant or mismatched neighbors degrade the context quality, making conditional probability predictions less accurate, increasing the actual number of bits output by the entropy encoder, and limiting compression efficiency.
[0035] Therefore, in learning-based point cloud compression, the conditional probability prediction of the second encoded feature y2 usually depends on the neighborhood context of the first encoded feature y1. However, existing methods directly select neighbors using fixed geometric rules, which easily introduces redundant or mismatched neighbors, leading to inaccurate probability predictions and an increase in encoded bits. A method is needed that, while maintaining the original network structure, makes the selection of "which neighbors make up the neighborhood" optimizable, while ensuring complete consistency between the neighbors selected by the encoder and decoder to avoid decoding errors.
[0036] This invention applies to a learning-based point cloud compression system employing grouped latent variable modeling, the system comprising the following modules:
[0037] An encoder network is used to map point clouds into latent feature representations and quantize them;
[0038] The advanced prior network is used to generate advanced prior information h to assist in probability prediction;
[0039] Entropy models (probabilistic models) are used to predict the conditional probability distribution of symbols to be encoded.
[0040] Entropy encoder / decoder encodes / decodes symbols based on probability distributions;
[0041] The decoder network recovers the point cloud from the latent features obtained from decoding.
[0042] This invention only applies to the context path of the second encoded feature y2: it does not change the structure of the encoder network, decoder network, super-prior network and entropy model, but only changes the step of "selecting which neighbors from the first encoded feature y1 as the context input of the second encoded feature y2", thereby ensuring compatibility and deployability.
[0043] For each position n that needs to be encoded in the second encoding feature y2, the neighborhood construction of this invention is divided into two steps:
[0044] Step 1: Candidate Neighbor Retrieval. This involves retrieving a large set of candidate neighbors, denoted as C, from the decoded first encoded feature y1 using a deterministic geometric nearest neighbor rule (e.g., KNN). n The size of the candidate neighbor set is K. cand The candidate set here can be understood as a "pool of alternative neighbors".
[0045] Step 2: Select a fixed number of retainable neighbors from the candidate pool, i.e., from the candidate set C. n Select a fixed-size subset from the set, denoted as S. n The size of the retained subset is K kept (Stabilized) n This will be used as input to the context aggregation network to predict the probability distribution of the second encoded feature y2 at the position n to be encoded, since K kept It is fixed and has the same input dimension as the original system, so no network structure needs to be modified.
[0046] To ensure process reproducibility and consistency between the encoder and decoder, the initial reserved subset S0 adopts the simplest deterministic rule: from C n Select the nearest K from the middle kept The candidate neighbors are selected as the initial reserved subset S0.
[0047] From candidate set C n Select K kept The problem of finding a neighbor subset is essentially a combinatorial optimization problem. Enumerating all possible subsets and trying them one by one would be computationally extremely expensive, making it impractical for real-world coding or large-scale experiments. Therefore, this invention employs a short-step iterative replacement refinement method to quickly approximate a better neighbor subset while keeping computational costs under control.
[0048] The refining process starts from the initial subset S0 and then proceeds through T iterations. Each iteration performs only one simple "single replacement." During the t-th 1-swap replacement, the set is changed from the complement C... n \S t-1 The set of neighbors S obtained by selecting one element from the previous 1-swap replacement is called the set of neighbors that is retained. t-1 One element is used to obtain the set of retained neighbors S obtained by the t-th 1-swap replacement. t To minimize the local bit rate change, the complement C n \S t-1 Let the candidate neighbor set C be n In addition to retaining subset S t-1 The set consists of elements other than those in the set. This achieves the goal of "replacing a less suitable neighbor with a more suitable one".
[0049] To determine whether a replacement is helpful, this invention introduces the concept of "local coding cost." Intuitively, the more useful the hint information and the more accurate the prediction, the lower this cost. This cost is expressed as:
[0050]
[0051] in, This indicates the use of a reserved subset S. t The encoding cost of the second encoded feature at position n, using a set as the context neighborhood, is that when encoding the second encoded feature y2 at position n, if the context uses a subset S. t The probability distribution predicted by the entropy model determines how many bits are needed to encode this position; S t S represents the set of retained neighbors obtained by performing the t-th 1-swap replacement, which is the set of retained neighbors obtained by performing the (t-1)-th 1-swap replacement. t-1 Replace one element with its complement C n \S t-1 The set obtained by taking one element.
[0052] To save computation time, this invention employs a perceptron network composed of a multilayer perceptron to predict the local bitrate change at the position n to be encoded. During prediction, the feature information of the retained subset and the discarded subset (referring to latent variable features, such as the feature vector of any candidate neighbor point that can be directly read from the decoded y1 feature map), the geometric relationship features between the retained subset and the discarded subset and the current position n to be encoded (relative displacement, Euclidean distance, etc. can be used as geometric relationship features between the red neighbor points of the subset and the current position), and the prior side information are taken as input. The multilayer perceptron outputs the local bitrate change at the position n to be encoded under this input state.
[0053] When constructing training data, in order to accurately analyze the source of benefits and obtain high-quality training data, the output of the full entropy model can be directly called to calculate the real local coding cost difference for each candidate replacement, thereby obtaining the real local bitrate change.
[0054] In this embodiment, the perceptron network composed of a multilayer perceptron is a three-layer fully connected network (3-layer MLP). The input of the scorer consists of three parts: the first part is the feature information of the removed neighbors and the newly added candidate neighbors; the second part is the relative geometric relationship information between them and the current position (e.g., distance or relative position encoding); and the third part is the cue features related to the prior side information h. The scorer outputs a local bit rate change prediction value.
[0055] The scorer is trained using supervised learning. Training data comes from real computation results in the offline phase: for a large number of positions and candidate substitution operations, the true local bitrate changes are calculated using a full entropy model, and the "substitution input features" are combined with the corresponding true local bitrate changes to form training samples. The training task of the scorer is a regression prediction problem, namely, making the network output as close as possible to the true ∆R. The training loss function uses mean squared error (MSE) to measure the difference between the predicted and true values, enabling the scorer to learn to quickly evaluate the "bit-saving effect" of different substitution operations.
[0056] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A point cloud context neighborhood selection method for entropy models, applied to a learning-based point cloud compression system, wherein the system transforms point cloud data into latent features and quantizes them to obtain latent variables, and the latent variables are divided into first encoded features and second encoded features according to a checkerboard grouping rule, wherein the first encoded features are first entropy encoded and then recovered at the decoding end as decoded information; characterized in that, The conditional probability of the second encoded feature is jointly determined by the context neighborhood features of the decoded first encoded feature and the prior information. The process of retrieving neighborhood features of the second encoded feature in the feature space of the decoded first encoded feature includes:
101. For any position n to be encoded in the second encoded feature, a candidate neighbor set C is obtained in the feature space of the decoded first encoded feature according to the deterministic geometric nearest neighbor rule. n The size of the candidate neighbor set is K. cand ; 102. Select K from the candidate neighbor set kept Each element is used as the retained neighbor set S. n Perform T 1-swap replacement operations on the set to obtain the final neighbor set S' of the position to be encoded n; 103. During the t-th 1-swap replacement, from complement C... n \S t-1 The set of neighbors S obtained by selecting one element from the previous 1-swap replacement is called the set of neighbors that is retained. t-1 The elements in the set minimize the local bitrate change, and the complement C n \S t-1 Let the candidate neighbor set C be n In addition to retaining subset S t-1 A set consisting of elements other than those mentioned above; 104. The elements in the final neighbor set S' are used as the neighborhood context input of the position n to be encoded in the second encoded feature, for conditional probability prediction of the second encoded feature and entropy encoding / entropy decoding.
2. The point cloud context neighborhood selection method for an entropy model according to claim 1, characterized in that, The candidate neighbor set C is obtained by retrieving the deterministic geometric nearest neighbor rule. n That is, selecting K, which is the geometric distance from the location n to be encoded, from all neighbors. cand C is a candidate neighbor set. n Select K from the candidate neighbor set kept Each element is used as the retained neighbor set S. n When selecting a position n, choose the K position that is geometrically closest to the position n to be encoded. kept A neighbor.
3. The point cloud context neighborhood selection method for an entropy model according to claim 1, characterized in that, The local bitrate change at position n to be encoded during the t-th 1-swap replacement is represented as: ； in, This indicates the use of a reserved subset S. t The encoding cost of the second encoded feature at position n is calculated using a set as the context neighborhood; S t S represents the set of retained neighbors obtained by performing the t-th 1-swap replacement, which is the set of retained neighbors obtained by performing the (t-1)-th 1-swap replacement. t-1 Replace one element with its complement C n \S t-1 The set obtained by taking one element.
4. The point cloud context neighborhood selection method for an entropy model according to claim 3, characterized in that, Using the reserved subset S t Using the set as the context neighborhood, the encoding cost of the second encoded feature at the position n to be encoded is calculated. Represented as: ； in, The conditional probability output by the entropy model of the learning-based point cloud compression system is obtained by preserving a subset of neighbors. And the prior side information h is input into the entropy model of the learning point cloud compression system, and the probability output by the entropy model; Indicates the second coding feature The encoded symbol at position n to be encoded.
5. A point cloud context neighborhood selection method for an entropy model according to claim 1, characterized in that, The perceptron network composed of multilayer perceptrons predicts the local bit rate change at the position n to be encoded. During prediction, the feature information of the retained subset and the discarded subset, the geometric relationship features of the retained subset and the discarded subset with the current position n to be encoded, and the prior side information are taken as inputs. The multilayer perceptron outputs the local bit rate change at the position n to be encoded under the input state.
6. A point cloud context neighborhood selection method for an entropy model according to claim 5, characterized in that, When making predictions for the perceptron network, the parameters of the perceptron network are updated by minimizing the mean square error between the perceptron network's predicted values and the training data.
7. A point cloud context neighborhood selection system for entropy models, characterized in that, A method for selecting a point cloud context neighborhood for an entropy model as described in claim 1 includes an offline module and an online module, wherein: The offline module is used to collect samples for training. Each sample includes features and labels. The features of the sample consist of the feature information of the retained subset and the discarded subset of each encoded position, the geometric relationship features between the retained subset and the discarded subset and the current encoding position n, and the prior side information. The neighbor set obtained through steps 101 to 104 is used as the label. The online module is used to train the perceptron network composed of multilayer perceptrons based on the sample data obtained by the offline module, and then predicts the label of the trained perceptron network based on the features of the location to be encoded.
8. A computer device, characterized in that, The device includes: Memory; processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement a point cloud context neighborhood selection method for an entropy model as described in any one of claims 1-6.
9. A computer storage medium, characterized in that, The computer storage medium stores computer execution instructions, which, when executed by a processor, are used to implement a point cloud context neighborhood selection method for an entropy model as described in any one of claims 1-6.