Method and apparatus for binary entropy encoding of point cloud

Abstract

Methods and apparatus for binary entropy encoding of point clouds. Binary entropy encoding is used to encode a bit sequence that signals the occupancy pattern of sub-volumes used for a volume. Context can be based on neighbor configuration and a partial sequence of previously encoded bits of the bit sequence. A determination is made as to whether to apply a context reduction operation, and if so, the operation reduces the number of available contexts. Exemplary context reduction operations include reducing neighbor configuration based on: masking of sub-volumes associated with previously encoded bits, special handling of empty neighbor configurations, and statistically based context merging.

Description

[0001] Related application citation

[0002] This application is a divisional application of the invention patent application with international application number PCT / CA2019 / 050399, international application date of April 3, 2019, entry into the Chinese national phase date of December 8, 2020, Chinese national application number 201980039083.6, and invention title "Method and apparatus for binary entropy encoding of point clouds". Technical Field

[0003] This application relates generally to point cloud compression, and more particularly to methods and apparatus for binary entropy encoding of point clouds. Background Technology

[0004] Data compression is used in communications and computer networking to efficiently store, transmit, and reproduce information. There is a growing interest in representing three-dimensional objects or spaces, as this can involve large datasets for which efficient and effective compression is extremely useful and valuable. In some cases, point clouds can be used to represent three-dimensional objects or spaces. A point cloud is a collection of points, each with three coordinate positions (X, Y, Z), and in some cases, additional properties such as color data (e.g., brightness and chromaticity), transparency, reflectivity, normal vectors, etc. Point clouds can be static (a snapshot of a fixed object or environment / object at a single point in time) or dynamic (a time series of point clouds).

[0005] Example applications of point clouds include terrain and mapping applications. Autonomous vehicles and other machine vision applications may rely on point cloud sensor data in the form of 3D scans of the environment, such as those from LiDAR scanners. Virtual reality simulations may also depend on point clouds.

[0006] It will be recognized that point clouds can involve large amounts of data, and that it is crucial to compress (encode and decode) this data quickly and accurately. Therefore, it would be advantageous to provide methods and devices for compressing point cloud data more efficiently and / or effectively. Furthermore, it would be beneficial to find methods and devices for encoding point clouds that can be implemented using context-adaptive binary entropy coding without managing an excessive amount of context. Attached Figure Description

[0007] Reference will now be made to the accompanying drawings, which illustrate exemplary embodiments of this application, in which:

[0008] Figure 1 A simplified block diagram of an example point cloud encoder is shown;

[0009] Figure 2 A simplified block diagram of an example point cloud decoder is shown;

[0010] Figure 3 Example partial subvolumes and associated tree structures used for encoding are shown;

[0011] Figure 4 The recursive partitioning and encoding of an octree are illustrated;

[0012] Figure 5 An example scan pattern from an octree is shown within an example cube;

[0013] Figure 6 The example occupancy pattern within the example cube is shown;

[0014] Figure 7 An example method for encoding point clouds is shown in flowchart form;

[0015] Figure 8 A portion of an example octree is shown;

[0016] Figure 9 An example of a neighboring sub-volume is shown;

[0017] Figure 10 An example neighbor configuration is shown, illustrating the occupancy among neighboring nodes;

[0018] Figure 11 A schematic example of a process using point cloud entropy encoding with a parent pattern-dependent context is illustrated.

[0019] Figure 12 A schematic embodiment of a process for point cloud entropy encoding using a context that depends on neighbor configuration is shown;

[0020] Figure 13 An example method for decoding bitstreams of compressed point cloud data is shown in flowchart form;

[0021] Figure 14 A simplified block diagram of an encoder is shown;

[0022] Figure 15 A simplified block diagram of an example decoder is shown;

[0023] Figure 16 An example Cartesian coordinate system and example rotations and / or reflections about an axis are shown;

[0024] Figure 17 The class demonstrates the invariance of neighbor configuration under one or several iterations of rotation around the Z-axis;

[0025] Figure 18 The class is shown to be an invariant for the neighbor configuration used for vertical reflection;

[0026] Figure 19 The class demonstrates the invariance of both rotation and reflection;

[0027] Figure 20 The class is shown to be invariant under three rotations and reflections;

[0028] Figure 21 The equivalence between non-binary encoding and concatenated binary encoding for occupancy patterns is shown;

[0029] Figure 22 A flowchart illustrates an example method for encoding occupancy patterns in a tree-based point cloud encoder using binary encoding.

[0030] Figure 23 A simplified block diagram of a portion of the example encoder is shown;

[0031] Figure 24 An example context reduction operation based on neighbor masking is illustrated schematically;

[0032] Figure 25 Another example of context reduction operation based on neighbor masking is shown; and

[0033] Figure 26 An example of a method for binary encoding occupancy patterns using combined context reduction is shown in flowchart form.

[0034] Similar reference numerals may have been used in different accompanying drawings to denote similar components. Detailed Implementation

[0035] This application describes methods for encoding and decoding point clouds, as well as encoders and decoders for encoding and decoding point clouds. A bit sequence can be encoded using binary entropy encoding, which signals the occupancy pattern of sub-volumes used for the volume. Context can be based on neighbor configuration and a partial sequence of previously encoded bits of the bit sequence. It is determined whether a context reduction operation should be applied, and if so, the operation reduces the number of available contexts. Example context reduction operations include reducing neighbor configuration based on masking of sub-volumes associated with previously encoded bits, special handling of empty neighbor configurations, and statistically based context merging. Reduction can be applied before encoding, and it can be determined during encoding whether the reduced set of contexts is satisfied.

[0036] In one aspect, this application provides a method for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having multiple nodes with parent-child relationships and representing the geometry of a volume space recursively divided into sub-volumes and containing the points of the point cloud, wherein the occupancy of the sub-volumes of the volume is indicated using a bit sequence, each bit in the bit sequence indicating the occupancy of a corresponding sub-volume within the volume in scan order, and wherein the volume has multiple neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the method comprising: for the bits of the volume For at least one bit in the sequence, determine that a context reduction condition is met, and on this basis, select a reduced set of contexts, the reduced set of contexts containing contexts smaller than the product of the neighbor configuration count and the number of previously encoded bits in the sequence; select a context from the reduced set of contexts for encoding the at least one bit based on the occupancy status of at least some of the neighbor volumes and at least one previously encoded bit in the bit sequence; entropy encode the at least one bit using a binary entropy encoder based on the selected context to generate encoded data for the bit stream; and update the selected context.

[0037] On the other hand, this application provides a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure having multiple nodes with parent-child relationships and representing a geometric structure of a volume space recursively divided into sub-volumes and containing the points of the point cloud, wherein the occupancy of the sub-volumes of the volume is indicated using a bit sequence, each bit in the bit sequence indicating the occupancy of the corresponding sub-volume within the volume in scan order, and wherein the volume has multiple neighboring volumes whose occupancy pattern is a neighbor configuration. The decoding method includes: determining, for at least one bit in the bit sequence of the volume, that a context reduction condition is met, and on this basis, selecting a reduced set of contexts, the reduced set of contexts containing contexts smaller than the product of the neighbor configuration count and the number of previously encoded bits in the sequence; selecting a context from the reduced set of contexts for encoding the at least one bit based on the occupancy status of at least some neighbor volumes in the neighbor volumes and at least one previously encoded bit in the bit sequence; performing entropy decoding on the at least one bit using a binary entropy decoder based on the selected context to generate reconstructed bits from the bit stream; and updating the selected context.

[0038] In some implementations, the context reduction condition includes determining that one or more previously encoded occupied bits are associated with one or more corresponding sub-volumes, said one or more corresponding sub-volumes being located between the sub-volume associated with said at least one bit and said one or more neighboring volumes among the neighboring volumes. In some cases, this may include determining that four sub-volumes associated with the previously encoded bit share a face with a specific neighboring volume.

[0039] In some implementations, context reduction conditions may include determining that at least four bits of the bit sequence have been previously encoded.

[0040] In some implementations, determining that the context reduction condition is met includes determining that the occupancy pattern of the neighbor volumes indicates that the plurality of neighbor volumes are not occupied. In some of those cases, the selected set of reduced contexts may include a plurality of contexts corresponding to the number of previously encoded bits in the bit sequence, and optionally, selecting contexts may include selecting contexts based on the sum of previously encoded bits in the bit sequence.

[0041] In some implementations, the context reduction condition may include determining that at least a threshold number of bits in the bit sequence have been pre-encoded, and the reduced set of contexts may include a lookup table that maps each possible combination of neighbor configurations and patterns of previously encoded bits in the bit sequence to fewer contexts. In some examples, the lookup table may be generated based on: iteratively grouping the available contexts into multiple classes based on determining that the distance measurement between each pair of available contexts is less than a threshold value, and each of the multiple classes may include the corresponding contexts in a smaller set, and for each possible combination of neighbor configurations and patterns of previously encoded bits in the bit sequence, there may be available contexts.

[0042] In some implementations, at least some neighboring volumes are neighboring volumes that share at least one face with the volume.

[0043] On the other hand, this application describes an encoder and decoder configured to implement such encoding and decoding methods.

[0044] In another aspect, this application describes a non-transitory computer-readable medium storing computer-executable program instructions that, when executed, cause one or more processors to perform the described encoding and / or decoding methods.

[0045] On the other hand, this application describes a computer-readable signal containing program instructions that, when executed by a computer, cause the computer to perform the described encoding and / or decoding methods.

[0046] Other aspects and features of this application will be understood by those skilled in the art from the following description of the examples taken in conjunction with the accompanying drawings.

[0047] Any feature described with respect to one aspect or embodiment of the invention may also be used with respect to one or more other aspects / embodiments. These and other aspects of the invention will become apparent and illustrated by reference to the embodiments described herein.

[0048] In the following description, the terms "node," "volume," and "subvolume" are sometimes used interchangeably. It will be understood that a node is associated with a volume or subvolume. A node is a specific point on a tree, which can be an internal node or a leaf node. A volume or subvolume is a finite physical space represented by a node. In some cases, the term "volume" can be used to represent the largest bounded space defined to contain a point cloud. For the purpose of constructing a tree structure of interconnected nodes to encode point cloud data, a volume can be recursively divided into multiple subvolumes.

[0049] In this application, the term "and / or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any element, any sub-combination, or all elements listed separately, without necessarily excluding other elements.

[0050] In this application, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any single element, any sub-combination, or all of the listed elements, without excluding any other elements or requiring all elements.

[0051] A point cloud is a collection of points in a three-dimensional coordinate system. These points are typically intended to represent the outer surface of one or more objects. Each point has a location (location) in the three-dimensional coordinate system. This location can be represented by three coordinates (X, Y, Z), which can be a Cartesian coordinate system or any other coordinate system. These points may have other associated properties, such as color, and in some cases, three-component values, such as R, G, B or Y, Cb, Cr. Depending on the desired application of the point cloud data, other associated properties may include transparency, reflectivity, normal vector, etc.

[0052] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or terrain might be static point cloud data. LiDAR-based environmental scans for machine vision purposes might be dynamic because the point cloud (at least potentially) changes over time, such as during each successive scan volume. Therefore, a dynamic point cloud is a time-ordered sequence of point clouds.

[0053] Point cloud data can be used in a variety of applications, including preservation (scanning of historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous vehicles), and virtual reality systems, to give some examples. Dynamic point cloud data used for applications such as machine vision can be quite different from static point cloud data used for preservation purposes. For example, automotive vision typically involves relatively low-resolution, colorless, highly dynamic point clouds acquired through LiDAR (or similar) sensors with high capture frequencies. These point clouds are not intended for human consumption or viewing, but rather for machine object detection / classification in the decision-making process. A typical LiDAR framework, for example, contains tens of thousands of points, while high-quality virtual reality applications require millions. It can be expected that, with increasing computing speeds and the emergence of new applications, higher resolution data will be needed over time.

[0054] While point cloud data is useful, the lack of effective and efficient compression, i.e., encoding and decoding processes, can hinder adoption and deployment. A unique challenge in encoding point clouds, not present in other data compression scenarios such as audio or video, is encoding the geometry of the point cloud. Point clouds tend to be sparsely distributed, making efficient encoding of point locations even more challenging.

[0055] One of the more common mechanisms for encoding point cloud data is the use of a tree-based structure. In a tree-based structure, the bounded 3D volume of the point cloud is recursively divided into subvolumes. Nodes in the tree correspond to subvolumes. The decision to further subdivide subvolumes can be based on the tree's resolution and / or whether a subvolume contains arbitrary points. Leaf nodes can have an occupancy flag indicating whether their associated subvolume contains points. Segmentation flags can signal whether a node has child nodes (i.e., whether the current volume has been further subdivided into subvolumes). In some cases, these flags can be entropy-encoded, and in others, predictive coding can be used.

[0056] A commonly used tree structure is the octree. In this structure, the volume / subvolume is a cube, and each partition of a subvolume produces eight more subvolumes / subcubes. Another commonly used tree structure is the KD tree, where the volume (cube or rectangular prism) is recursively divided into two parts by a plane orthogonal to one of its axes. The octree is a special case of the KD tree where the volume is partitioned by three planes, each orthogonal to one of the three axes. Both examples involve cubes or rectangular prisms; however, this application is not limited to this tree structure, and in some applications, the volume and subvolumes can have other shapes. The partitioning of a volume does not necessarily have to be into two subvolumes (KD tree) or eight subvolumes (octree), but may involve other partitions, including partitioning into non-rectangular shapes or involving non-adjacent subvolumes.

[0057] For ease of explanation, reference may be made to an octree, as it is a popular candidate tree structure for automotive applications; however, it should be understood that the methods and devices described herein may be implemented using other tree structures.

[0058] Now for reference Figure 1 The diagram shows a simplified block diagram of a point cloud encoder 10 according to an aspect of this application. The point cloud encoder 10 includes a tree building module 12 for receiving point cloud data and generating a tree (an octree in this example) that represents the geometry of a volumetric space containing the point cloud and indicates the location or position of points from the point cloud within that geometry.

[0059] The basic process for creating an octree to encode a point cloud may include:

[0060] 1. Start with the boundary volume (cube) of the coordinate system containing the point cloud.

[0061] 2. Divide the volume into 8 sub-volumes (eight sub-cubes).

[0062] 3. For each subvolume, if the subvolume is empty, mark it with 0; if there is at least one point, mark it with 1.

[0063] 4. For all sub-volumes marked as 1, repeat step (2) to divide these sub-volumes until the maximum division depth is reached.

[0064] 5. For all leaf sub-volumes (sub-cubes) at the maximum depth, mark the leaf sub-volume with 1 if the leaf cube is not empty, otherwise mark it with 0.

[0065] The process described above can be described as an occupancy-segmentation process, where segmentation implies occupancy, constrained by the existence of a maximum depth or resolution beyond which no further segmentation will occur. In this case, a single flag signals whether a node has been segmented, and therefore whether it is occupied by at least one point, and vice versa. At the maximum depth, a flag signals occupancy, and no further segmentation is possible.

[0066] In some implementations, partitioning and occupancy are independent, allowing nodes to be occupied and partitioned or not partitioned. This implementation has two variations:

[0067] 1. Segmentation before Occupation. A signal flag indicates whether a node has been segmented. If segmented, the node must contain points – segmentation implies occupation. Otherwise, if the node is not segmented, another occupation flag signals whether the node contains at least one point. Therefore, when a node is not further segmented, i.e., it is a leaf node, the leaf node must have an associated occupation flag indicating whether it contains any points.

[0068] 2. Occupy first, then split. A single flag indicates whether the node is occupied. If it is not occupied, no split will occur. If it is occupied, a split flag is encoded to indicate whether the node should be further split.

[0069] Regardless of which of the above processes is used to construct the tree, it can be traversed in a predetermined order (breadth-first or depth-first, and according to the scan pattern / order within each partitioned sub-volume) to generate a series of bits (occupancy and / or partition flags) from the flags. This can be called tree serialization or binarization. Figure 1 As shown, in this example, the point cloud encoder 10 includes a binarizer 14 for binarizing the octree to produce a bitstream of binarized data representing the tree.

[0070] The bit sequence can then be encoded using an entropy encoder 16 to produce a compressed bitstream. The entropy encoder 16 can encode the bit sequence using a context model 18, which specifies the probabilities to be used for encoding bits based on context determinations made by the entropy encoder 16. The context model 18 can be adaptively updated after each bit or a defined set of bits has been encoded. In some cases, the entropy encoder 16 can be a binary arithmetic encoder. In some implementations, the binary arithmetic encoder can employ context-adaptive binary arithmetic coding (CABAC). In some implementations, encoders other than arithmetic encoders can be used.

[0071] In some cases, the entropy encoder 16 may not be a binary encoder, but can operate on non-binary data. The output octree data from the tree building module 12 may not be evaluated in binary form, but can be encoded as non-binary data. For example, in the case of an octree, the eight flags (e.g., occupancy flags) within a subvolume in its scan order can be considered as 2... 8 -1 bits (e.g., an integer value between 1 and 255, because the value 0 is impossible for partitioning a subvolume, i.e., it will not be partitioned if it is completely unoccupied). In some implementations, this number can be encoded by an entropy encoder using a multi-symbol arithmetic encoder. Within a subvolume, such as a cube, the sequence of flags defining this integer can be referred to as the "pattern".

[0072] Similar to video or image coding, point cloud coding can include predictive operations in which efforts are made to predict patterns in sub-volumes. Predictions can be spatial (depending on previously encoded sub-volumes in the same point cloud) or temporal (depending on previously encoded point clouds in a temporal sequence of the point cloud).

[0073] exist Figure 2The diagram shows a block diagram of an example point cloud decoder 50 corresponding to encoder 10. Point cloud decoder 50 includes an entropy decoder 52 that uses the same context model 54 used by encoder 10. Entropy decoder 52 receives an input bitstream of compressed data and performs entropy decoding on the data to produce an output sequence of decompressed bits. This sequence is then converted into reconstructed point cloud data by tree reconstructor 56. Tree reconstructor 56 reconstructs the tree structure based on the decompressed data and knowledge of the scan order in which the tree data was binarized. Therefore, tree reconstructor 56 is able to reconstruct the positions of points from the point cloud (limited by the resolution of the tree encoding).

[0074] Figure 3 An exemplary sub-volume 100 is shown. In this example, for ease of illustration, sub-volume 100 is shown in two dimensions, and its size is 16x16. It should be noted that this sub-volume has been divided into four 8x8 sub-squares, two of which are further subdivided into 4x4 sub-squares, three of which are further subdivided into 2x2 sub-squares, and one of the 2x2 sub-squares is then subdivided into a 1x1 square. The 1x1 square represents the maximum depth of the tree and signifies the highest resolution of the location point data. Points from the point cloud are shown as points in the figure.

[0075] The structure of tree 102 is shown to the right of sub-volume 100. To the right of tree 102 are sequences of segmentation flags 104 and corresponding occupancy flags 106 obtained in a predefined breadth-first scan order. It will be observed that in this illustrative example, for each unsegmented sub-volume (node), there exists an occupancy flag, i.e., it has an associated segmentation flag set to zero. These sequences can be entropy-encoded.

[0076] Figure 4 Another example using occupancy segmentation conditions is shown in the figure. Figure 4 The recursive partitioning and encoding of octree 150 is illustrated. Only a portion of octree 150 is shown in the figure. FIFO 152 is shown as processing nodes for partitioning to illustrate the breadth-first nature of this process. FIFO 152 outputs the occupied node 154 queued in FIFO 152 for further partitioning after processing its parent node 156. The tree builder partitions the subvolume associated with occupied node 154 into eight subvolumes (cubes) and determines their occupancy. Occupancy can be indicated by an occupancy flag for each subvolume. In the prescribed scan order, the flag can be referred to as the occupancy pattern of node 154. This pattern can be specified by an integer representing the sequence of occupancy flags associated with subvolumes in the pre-scan order. For octrees, the pattern is an integer in the range [1, 255].

[0077] The entropy encoder then encodes the pattern using a non-binary arithmetic encoder based on the probabilities specified by the context model. In this example, the probabilities can be a pattern distribution based on an initial distribution model and are updated adaptively. In one implementation, the pattern distribution is actually a counter (an integer from 1 to 255) of the number of times each pattern is encountered during encoding. The pattern distribution can be updated after each subvolume is encoded. The pattern distribution can be normalized as needed because the relative frequency of the pattern is closely related to the probability assessment, rather than the absolute count.

[0078] Based on this pattern, the child nodes that are occupied (e.g., those with flag = 1) are then pushed into FIFO 152 for further splitting in sequence (providing that these nodes are not the maximum depth of the tree).

[0079] Now for reference Figure 5 , Figure 5 An example cube 180 of an octree is shown. Cube 180 is subdivided into eight sub-cubes. The scan order of reading the flags will produce an octet string that can be read as an integer [1, 255] in binary form. Based on the scan order and the final bit position of the flag in each sub-cube within the string, these sub-cubes have Figure 5 The values ​​shown. Assuming both the encoder and decoder use the same scan order, the scan order can be any sequence of subcubes.

[0080] As an example, Figure 6 Cube 180 is shown as having four "front" child cubes occupied. The occupied child cubes are cubes 1+4+16+64, which corresponds to pattern 85. Integer pattern numbers specify the occupancy pattern within the child cubes.

[0081] Octree representation, or more generally, arbitrary tree representation, is efficient in representing points with spatial correlation because trees tend to decompose point coordinates into higher-order bits. For an octree, each depth level refines the coordinates of points within a subvolume into one bit at an overhead of eight bits per refinement for each component. Further compression can be achieved by entropy encoding the segmentation information (i.e., patterns) associated with each tree node. Further compression is possible because the pattern distribution is non-uniform—non-uniformity is another consequence of correlation.

[0082] A potential inefficiency in the current system is that the pattern distribution (e.g., a histogram of pattern numbers seen in previously encoded nodes of a tree) is developed during the encoding process of the point cloud. In some cases, the pattern distribution can be initialized as equiprobable, or it can be initialized as another predetermined distribution; however, using a pattern distribution means that the context model does not consider or utilize local geometric dependencies.

[0083] In European Patent Application No. 18305037.6, the applicant describes a method and apparatus for selecting from a distribution of available patterns for encoding an occupancy pattern of a particular node, based on some occupancy information from previously encoded nodes near that node. In one example implementation, occupancy information is obtained from occupancy patterns from a parent to the particular node. In another example implementation, occupancy information is obtained from one or more nodes adjacent to the particular node. European Patent Application No. 18305037.6 is incorporated herein by reference.

[0084] Now for reference Figure 7 The example method 200 for encoding point clouds is illustrated in flowchart form. In this example, method 200 involves recursively segmenting the occupied nodes (sub-volumes) and performing a breadth-first traversal of the tree for encoding.

[0085] In operation 202, the encoder determines the occupancy pattern of the current node. The current node is an occupied node that has been divided into eight child nodes, each corresponding to a sub-cube. The occupancy pattern of the current node specifies the occupancy of the eight child nodes in the scan order. As mentioned above, this occupancy pattern can be indicated using an integer between 1 and 255, such as an eight-bit binary string.

[0086] In operation 204, the encoder selects a probability distribution from a set of probability distributions. The selection of the probability distribution is based on some occupancy information from previously encoded nearby nodes, i.e., at least one node that is a neighbor of the current node. In some embodiments, two nodes are neighboring if they are associated with corresponding subvolumes sharing at least one face. In a more general definition, nodes are neighbors if they share at least one edge. In an even broader definition, two nodes are neighboring if they share at least one vertex. The current node is the parent pattern of its children, providing occupancy data for the current node and its seven sibling nodes. In some implementations, the occupancy information is the parent pattern. In some implementations, the occupancy information is the occupancy data of a set of neighboring nodes, which includes nodes at the same depth level in the tree as the current node but with different parent nodes. In some cases, combinations of these are possible. For example, the set of neighboring nodes may include some sibling nodes and some non-sibling nodes.

[0087] Once a probability distribution is selected, as shown in operation 206, the encoder then uses the selected probability distribution to entropy encode the occupancy pattern of the current node. Then, in operation 208, the selected probability distribution is updated based on the occupancy pattern; for example, it may increment the count corresponding to that occupancy pattern. In operation 210, the encoder evaluates whether there are any other nodes to encode; if so, it returns to operation 202 to encode the next node.

[0088] The probability distribution selection in operation 204 will be based on the occupancy data of nearby previously encoded nodes. This allows both the encoder and decoder to make the same selection independently. For a discussion of probability distribution selection below, refer to... Figure 8 The diagram schematically illustrates a portion of the octree 300, including the current node 302. The current node 302 is an occupied node and is being evaluated for encoding. The current node 302 is one of the eight children of its parent node 306, which in turn is a child of its grandparent node (not shown). The current node 302 is divided into eight child nodes 304. The occupancy pattern of the current node 302 is based on the occupancy of its child nodes 304. For example, as shown in the diagram, black dots represent the convention for occupant nodes, and the occupancy pattern can be 00110010, i.e., pattern 50.

[0089] The current node 302 has a sibling node 308, which shares the same parent node 306. The parent pattern is the occupancy pattern of the parent node 306, which, as shown in the figure, will be 00110000, i.e., pattern 48. The parent pattern can be used as the basis for selecting a suitable probability distribution for entropy coding of the current node's occupancy pattern.

[0090] Figure 9 The set of neighbors surrounding the current node is shown, where neighbors are defined as nodes sharing a face. In this example, the node / sub-volume is a cube, and the cube at the center of the image has six neighbors, one for each. In an octree, it will be apparent that the current node's neighbors will include three sibling nodes. It will also include three nodes that do not share a parent node. Therefore, occupancy data for some neighbor nodes will be available because they are siblings, but occupancy data for others may be available or unavailable depending on whether those nodes were previously encoded. Special handling may be necessary for handling missing neighbors. In some implementations, missing neighbors may be assumed to be occupied or may be assumed to be unoccupied. It will be understood that the neighbor definition can be extended to include adjacent nodes based on shared edges or shared vertices to include additional neighbor sub-volumes in the evaluation.

[0091] It will be understood that the aforementioned process focuses on the occupancy of nearby nodes in an attempt to determine the probability of occupancy of the current node 302, in order to select a more appropriate context(s) and entropy encode the occupancy data of the current node 302 with more accurate probabilities. It will be understood that, compared to the evaluation based on the occupancy state of sibling nodes, the occupancy state of neighboring nodes sharing a face with the current node 302 can be a more accurate assessment of whether the current node 302 is likely to be isolated, where three will share only one edge and one will share only one vertex (in the case of an octree). However, evaluating the occupancy state of siblings has the advantage of modularity because all relevant data used for evaluation is part of the parent node, meaning it has a smaller memory footprint available for implementation, whereas evaluating the occupancy state of neighbors involves buffering tree occupancy data in case it is needed when determining the neighbor occupancy state related to encoding future nearby nodes.

[0092] Neighbor occupancy can be read in the scanning order in which values ​​are effectively assigned to each neighbor, which is very similar to the occupancy patterns described above. As shown in the figure, neighbor nodes effectively take values ​​of 1, 2, 4, 8, 16, or 32, resulting in 64 (0 to 63) possible neighbor occupancy configurations. This value can be referred to as the "neighbor configuration" in this document. As an example, Figure 10 An example of neighbor configuration 15 is shown, where neighbors 1, 2, 4, and 8 are occupied while neighbors 16 and 32 are empty.

[0093] In some cases, both criteria (parent pattern and neighbor configuration) can be applied simultaneously, or a choice can be made between them. For example, if neighbors are available, probability distribution selection can be based on neighbor nodes; however, if one or more neighbors are unavailable because they come from nodes that have not yet been encoded, probability distribution selection can revert to analysis based on sibling nodes (parent pattern).

[0094] In yet another embodiment, the probability distribution selection may alternatively or additionally be based on the grandparent pattern. In other words, the probability distribution selection may be based on the occupancy status of the uncle node, which is the brother of parent node 306.

[0095] In another implementation, additional or alternative evaluations can be incorporated into the probability distribution selection. For example, the probability distribution selection can examine the occupancy status of the neighboring nodes of a parent node or the neighboring nodes of a grandparent node.

[0096] In some implementations, any two or more of the above criteria for assessing local occupancy status can be used in combination.

[0097] In the case of a non-binary entropy encoder, the occupancy data of the current node can be encoded by selecting a probability distribution. The probability distribution contains multiple probabilities corresponding to the number of possible occupancy patterns of the current node. For example, when encoding the occupancy patterns of an octree, there are 2... 8 -1 = 255 possible patterns, meaning each probability distribution contains 255 probabilities. In some embodiments, the number of probability distributions can be equal to the number of possible occupancy results in the selection criteria, i.e., using neighbor, sibling, and / or parent occupancy data. For example, if the parent pattern of an octree is used as the selection criterion for determining the probability distribution to use, there will be 255 probability distributions, each involving 255 probabilities. In the case of a neighbor configuration, if neighbors are defined as shared surfaces, there will be 64 probability distributions, each containing 255 probabilities.

[0098] It will be understood that, due to a lack of data, i.e., context dilution, too many distributions can lead to slow adaptation. Therefore, in some embodiments, similar patterns can be grouped so that the same probability distribution can be used. For example, a separate distribution can be used for patterns corresponding to fully occupied, vertically oriented, horizontally oriented, predominantly empty, and then all other cases. This can reduce the number of probability distributions to approximately five. It should be understood that different groupings of patterns can be formed to result in different numbers of probability distributions.

[0099] Now for reference Figure 11 The diagram illustrates a schematic embodiment of a point cloud entropy encoding process 400 that uses a parent-pattern-dependent context. In this example, the current node 402 has been divided into eight child nodes, and its occupancy pattern 404 will be encoded using a non-binary entropy encoder 406. The non-binary entropy encoder 406 uses a probability distribution selected from one of six possible probability distributions 408. The selection is based on the parent pattern—that is, based on occupancy information from the parent node to the current node 402. The parent pattern is identified by an integer between 1 and 255.

[0100] The choice of probability distribution can be a decision tree, evaluating whether the occupancy pattern corresponds to an entire node (e.g., pattern = 255), a horizontal structure (e.g., pattern = 170 or 85; assuming the Z-axis is vertical), a vertical structure (e.g., pattern = 3, 12, 48, 192), a sparsely filled distribution (e.g., pattern = 1, 2, 4, 8, 16, 32, 64, or 128; i.e., no sibling nodes are occupied), a semi-sparsely filled distribution (total number of occupied nodes between the current node and its sibling nodes is 3), and all other cases. The example patterns indicated for different categories are merely examples. For example, the "horizontal" category could include patterns involving two or three occupied cubes at the same horizontal level. The "vertical" category could include patterns involving three or four occupied cubes arranged in a wall-like configuration. It should also be understood that finer gradients can be used. For example, the "horizontal" category could be further subdivided into levels in the upper part of the cube and levels in the lower part of the cube, each with a different probability distribution. Other occupancy pattern groupings with some correlation can be performed and assigned to corresponding probability distributions. The following section further elaborates on the discussion of pattern grouping in the context of neighbor configurations and the invariance between neighbor configurations.

[0101] Figure 12 A schematic embodiment of a point cloud entropy encoding process 500 using a context-dependent neighbor configuration is shown. This example assumes the above combination Figure 9 The definition of the neighbors and neighbor configuration numbers used. This example also assumes that each neighbor configuration has a dedicated probability distribution, meaning there are 64 different probability distributions. The current node 502 has an occupancy pattern 504 to be encoded. The probability distribution is selected based on the nodes that are neighbors of the current node 502. That is, the neighbor configuration NC in [0,63] is found and used to select the associated probability distribution.

[0102] It should be understood that in some embodiments, neighbor configurations can be grouped such that more than one neighbor configuration uses the same probability distribution based on similarity in patterns. In some embodiments, the process can use different arrangements of neighbors for contextualization (selection) of the distribution. Additional neighbors can be added, such as eight neighbors diagonally adjacent on all three axes, or twelve neighbors diagonally adjacent on two axes. Embodiments that avoid specific neighbors can also be used, for example, to avoid using neighbors that introduce additional dependencies in depth-first scans, or to introduce dependencies only on specific axes to reduce the codec state of the large tree.

[0103] In this example, the case of NC=0 is handled in a specific way. If no neighbor is occupied, it can be indicated that the current node 502 is isolated. Therefore, process 500 further checks how many child nodes of the current node 502 are occupied. If only one child node is occupied, i.e., NumberOccupied(NO) equals 1, a flag is encoded to indicate that a single child node is occupied, and the index of that node is encoded using 3 bits. If multiple child nodes are occupied, process 500 uses the NC=0 probability distribution to encode the occupancy pattern.

[0104] Now for reference Figure 13 , Figure 13 An example method 600 for decoding a bitstream of encoded point cloud data is shown in flowchart form.

[0105] In operation 602, the decoder selects one of the probability distributions based on occupancy information from one or more nodes near the current node. As mentioned above, occupancy information can be the parent pattern from the parent node to the current node, i.e., the occupancy of the current node and its sibling nodes, or it can be the occupancy from neighboring nodes to the current node, which may include some sibling nodes. Other or additional occupancy information may be used in some implementations.

[0106] Once a probability distribution has been selected, in operation 604, the decoder entropy uses the selected probability distribution to decode a portion of the bitstream to reconstruct the occupancy pattern of the current node. The occupancy pattern is used by the decoder to reconstruct the tree, thereby reconstructing the encoded point cloud data. Once the point cloud data is decoded, it can be output from the decoder for use, such as for drawing views, segmentation / classification, or other applications.

[0107] In operation 606, the decoder updates the probability distribution based on the reconstructed occupancy pattern, and then, if there are other nodes to decode, it moves to the next node in the buffer and returns to operation 602.

[0108] The example implementations of the above methods have been demonstrated to provide compression improvements with negligible increases in encoding complexity. Although neighbor-based selection has higher computational complexity and memory usage, it exhibits better compression performance than parent-pattern-based selection. In some tests, the relative improvement in bits per point compared to the MPEG point cloud test model ranged from 4% to 20%. It has been noted that initializing the probability distribution based on a distribution derived from the test data improves performance compared to initializing using a uniform distribution.

[0109] Some of the examples above are based on a tree encoding process that uses a non-binary encoder to signal occupancy patterns. Further developments using binary entropy encoders are discussed below.

[0110] In a variation of the neighborhood-based probability distribution selection, the number of distributions can be reduced by leveraging the symmetry of neighbors. Structurally similar configurations with lines of symmetry can reuse the same distribution by permuting neighbors or permuting pattern distributions. In other words, neighbor configurations that can use the same pattern distribution can be grouped into a class. A class containing more than one neighbor configuration may be referred to as a "neighbor configuration" in this paper because one neighbor configuration effectively subsumes other neighbor configurations by reflecting or permuting those other configurations.

[0111] As an example, consider eight corner patterns NC∈[21,22,25,26,37,38,41,42], each representing the symmetry of a corner neighbor pattern. These values ​​of NC are likely associated with specific but distinct node patterns. These associated patterns may also follow the same symmetry as the neighbor patterns. For instance, a method could be implemented that reuses a single distribution to represent multiple cases of NC by permuting the probabilities of this distribution.

[0112] The encoder derives the pattern number of a node based on the occupancy of its child nodes. The encoder selects a distribution and permutation function based on the neighbor configuration. The encoder reorders the probabilities contained in the distribution according to the permutation function, and then uses the permuted distribution to perform arithmetic encoding on the pattern number. The arithmetic encoder updates the probabilities of the permuted distribution by mapping them back to the original distribution through the inverse permutation function.

[0113] The corresponding decoder first selects the same distribution and permutation function based on the neighbor configuration. The permutation distribution is generated in exactly the same way as the encoder, and this distribution is used by the arithmetic decoder to perform entropy decoding on the pattern number. Then, the bits including the pattern number are assigned to the corresponding children.

[0114] It should be noted that the same permutation can be achieved without reordering the data in the distribution itself, but by introducing an indirect level and using a permutation function to replace the lookup of a given index in the distribution.

[0115] Alternative implementations consider permutations of the patterns themselves rather than the distribution, allowing shuffling before or after entropy encoding / decoding. This approach may be better suited for efficient implementation via bit-by-bit shuffling. In this case, neither the encoder nor the decoder performs a reordering of the distribution; instead, the calculation of the encoded pattern number is modified to... Where c i Let be the occupancy state of the i-th child, and σ(i) be the permutation function. One example permutation function allows... The distribution with NC=22 is used for the distribution with NC=41. The decoder can use a permutation function to... The occupancy status of child nodes is derived from the encoded pattern number.

[0116] The method for deriving the desired permutation can be based on the rotational symmetry of the neighbor configuration or on reflection along a specific axis. Furthermore, the permutation does not necessarily need to permutate all positions based on, for example, symmetry; instead, partial permutations can be used. For example, when permuting NC=22 to NC=41, positions on the symmetry axis may not be permuted, resulting in a mapping. Positions 0, 2, 4, and 6 will not be swapped. In other embodiments, only positions 1 and 7 are swapped.

[0117] The following provides an example of an embodiment based on rotational symmetry and reflection for the specific case of an octree with six neighbors that share a common face with the current cube. Without loss of generality, as... Figure 16 As shown, the Z-axis extends perpendicularly to the direction in which the figure is viewed. The relative positions of neighbors, such as "above" (and "below"), should be understood as along the Z-axis in the increasing (and decreasing) Z-direction. The same annotation applies to left / right along the X-axis and front / back along the Y-axis.

[0118] Figure 16 Three rotations, 2102, 2104, and 2106, are shown along the Z, Y, and X axes, respectively. Each rotation is 90 degrees, meaning they represent a quarter-turn rotation along their respective axes.

[0119] Figure 17 The invariant classes of neighbor configurations are shown under one or more iterations of rotation 2102 along the Z-axis. This invariance represents the same statistical behavior of point cloud geometry along any direction belonging to the XY plane. This is especially true for the use case of a car driving on a locally approximated Earth surface in the XY plane. A horizontal configuration is a given occupancy of four neighbors (located to the left, right, front, and back of the current cube), independent of the occupancy of the upper neighbor (2202) and the lower neighbor (2204). Under rotation 2102, the four horizontal configurations 2206, 2208, 2210, and 2212 belong to the same class of invariances. Similarly, the two configurations 2214 and 2216 belong to the same class of invariances. Under rotation 2102, there are only six invariant classes (grouped under class set 2218).

[0120] The vertical configuration is the given occupancy of two neighbors, 2202 and 2204, and is independent of the occupancy of the four neighbors located to the left, right, front, and back of the current cube. For example... Figure 18 As shown, there are four possible vertical configurations. Therefore, if we consider the invariance relative to rotation 2102 along the Z-axis, there are 6 x 4 = 24 possible configurations.

[0121] Reflection along the Z-axis 2108 Figure 16 As shown. Figure 18 The vertical configurations 2302 and 2304 shown belong to the same invariant class under reflection 2108. There are three invariant classes under reflection 2108 (grouped under class 2306). Invariance under reflection 2108 means that, in terms of point cloud geometric statistics, the behavior in the upward and downward directions is essentially the same. This is an accurate assumption for a car moving on a road.

[0122] If we assume that the invariance is the same under both rotation 2102 and reflection 2108, then the product of the two sets 2218 and 2306 yields 18 classes of invariance. These 18 classes represent... Figure 19 middle.

[0123] Applying further invariances under two additional rotations 2104 and 2106, configurations 2401 and 2402 belong to the same class of invariances. Furthermore, configurations 2411 and 2412, configurations 2421 and 2422, configurations 2431, 2432 and 2433, configurations 2441 and 2442, configurations 2451 and 2452, and finally configurations 2461 and 2462 belong to the same category. Therefore, the invariance under the three rotations (2102, 2104, and 2106) and reflection 2108 result in 10 classes of invariances, such as... Figure 20 As shown.

[0124] From the examples provided above, assuming that the number of effective neighbor configurations (i.e., the class that can group 64 neighbor configurations) is 64, 24, 18, or 10, assuming that the number of rotations and reflections is invariant.

[0125] Before entropy encoding, the pattern undergoes the same transformations, namely rotation and reflection, because the neighbor configuration does indeed belong to one of the invariant classes. This preserves statistical consistency between the invariant neighbor configuration and the encoded pattern.

[0126] It will also be understood that during tree traversal, child nodes will have some neighboring nodes at the same tree depth that have been previously visited and can be used causally as dependencies. For these same-level neighbors, the parent's juxtaposed neighbors can be used instead of consulting them. Since same-level neighbors halve the size of the parent object, a configuration would consider a neighbor occupied if any one of the four directly adjacent neighboring children (i.e., the four children that share a face with the current node) is occupied.

[0127] Occupation pattern using binary-encoded entropy-coded trees

[0128] The aforementioned technique for encoding tree occupancy using neighbor occupancy information is described in detail in European Patent Application No. 18305037.6. The described embodiments focus on non-binary entropy encoding of occupancy patterns, where the pattern distribution is selected based on neighbor occupancy information. However, in some cases, the use of a binary encoder may be more efficient in terms of hardware implementation. Furthermore, instantaneous updates to numerous probabilities may require fast memory access and computation within the heart of the arithmetic encoder. Therefore, it may be advantageous to find methods and apparatus for entropy encoding occupancy patterns using a binary arithmetic encoder. Using a binary encoder would be advantageous if it could be performed without significantly reducing compression performance, while preventing an excessive number of contexts to be tracked.

[0129] The entropy formula reflects the use of a binary encoder instead of a non-binary encoder:

[0130] H(X1,X2|Y)=H(X1|Y)H(X2|Y,X1)

[0131] Where X = (X1, X2) is the non-binary information to be encoded, and Y is the context used for encoding, i.e., the neighbor configuration or the distribution of the selected pattern. To convert the non-binary encoding of X to binary encoding, the information (X1, X2) is split into information X1 and X2, which can be encoded separately without increasing entropy. For this, one must encode one of the two based on the other; here, X2 depends on X1. This can be extended to n bits of information in X. For example, for n = 3:

[0132] H(X1,X2,X3|Y)=H(X1|Y)H(X2|Y,X1)H(X3|Y,X1,X2)

[0133] It will be understood that as the occupancy pattern, i.e., the bit sequence X, becomes longer, more conditions exist for encoding the later bits in that sequence. For binary encoders (such as CABAC), this means a significant increase in the number of contexts to track and manage. Using an octree as an example, where the occupancy pattern is an eight-bit sequence b = b0…b7, this bit sequence can be divided into eight binary information bits b0…b7. Encoding can use the neighbor configuration N (or NC) to determine the context. Assuming, as described above, by grouping the neighbor configurations into invariant classes, the number of valid neighbor configurations can be reduced to 10, where N is an integer belonging to {0, 1, 2, …, 9}. For brevity, “classes of invariant neighbor configurations” may sometimes be simply referred to as “neighbor configurations” in this document, although it should be understood that this reduction in the number of neighbor configurations can be achieved based on class-based grouping of invariant neighbor configurations.

[0134] Figure 21This illustrates dividing an eight-bit pattern or sequence into eight individual bits for binary entropy encoding. Note that the first bit of the sequence is encoded based on neighbor configurations, thus there are ten available contexts. The next bit of the sequence is encoded based on the neighbor configurations and any previously encoded bit (i.e., bit b0). This involves a total of 20 available contexts: 10 from N and 2 from b0. The final bit b7 is entropy encoded using contexts selected from 1280 available contexts: obtained as 10 from N and 128 from the partial patterns given by the previously encoded bits b0,…,b6. That is, for each bit, the number of contexts (i.e., possible combinations of conditions / dependencies) is the product of the number of defined neighbor configurations (in this example, based on grouping 64 neighbor configuration pairs into 10) and the number of possible partial patterns from an ordered sequence of n-1 previously encoded bits (given by 2n-1).

[0135] As a result, a total of 2550 contexts need to be associated with the binary encoding of the occupancy pattern. This is a very large number of contexts to track, and the relative scarcity can lead to poor performance due to context dilution, especially for subsequent bits in the sequence.

[0136] Therefore, in one aspect, this application discloses an encoder and decoder that determine whether the set of contexts can be reduced, and if so, apply a context reduction operation to achieve a smaller set of available contexts for entropy encoding of at least a portion of the occupancy patterns using a binary encoder. In another aspect, this application also discloses an encoder and decoder that use the same context reduction operation to apply one or more rounds of state reduction to perform efficient context selection from a fixed number of contexts. In some implementations, context reduction is applied before generating a lookup table of context and / or algorithmic conditions, which is then used by the encoder or decoder when selecting a suitable context. The reduction is based on testable conditions, which the encoder and decoder evaluate to determine which lookup table to select from or how to index / select from that lookup table to obtain the selected context.

[0137] Now for reference Figure 22 The diagram illustrates an example method 3000, which uses binary encoding to encode occupancy patterns in a tree-based point cloud encoder. Method 3000 can be implemented using either an encoder or a decoder. In the case of an encoder, the encoding operation is encoding, while in the case of a decoder, the encoding operation is decoding. Encoding and decoding are context-based entropy encoding and decoding.

[0138] Example method 3000 is used for entropy encoding of the occupancy pattern, i.e., the bit sequence, of a specific node / volume. The occupancy pattern signals the occupancy status of the node / volume's child nodes (sub-volumes). In the case of an octree, there are eight child nodes / sub-volumes. In operation 3002, the neighbor configuration is determined. The neighbor configuration is the occupancy status of one or more volumes adjacent to the volume for which the occupancy pattern is to be encoded. As mentioned above, various possible implementations exist for determining the neighbor configuration. In some examples, there are 10 neighbor configurations, and the neighbor configuration of the current volume is identified based on the occupancy of the six volumes sharing a face with the current volume.

[0139] In operation 3004, the index i of the child node to the current volume is set to 0. Then, in operation 3006, an evaluation is performed regarding the possibility of context reduction. Different possible context reduction operations will be discussed in detail below. For example, the possibility of context reduction can be evaluated based on which bit (e.g., index value) in the bit sequence is encoded. In some cases, context reduction may be possible for the later bits in the sequence, not the earlier bits. The evaluation of the possibility of context reduction can be based, for example, on the neighbor configuration, as some neighbor configurations can allow for simplification. In some implementations, other factors can be used to evaluate the possibility of context reduction. For example, an upper limit Bo can be provided as the maximum number of contexts that the binary encoder can use to encode bits, and if the initial number of contexts to be encoded is greater than Bo, then context reduction is applied (otherwise no), such that the reduced number of contexts is at most Bo. Such a boundary Bo can be defined in the encoder and / or decoder specifications to ensure that software or hardware implementations capable of handling Bo contexts will always be able to encode and / or decode the point cloud without overflow in terms of the number of contexts. Knowing the boundaries in advance allows for anticipating the complexity and memory footprint caused by the binary entropy encoder, thus aiding in hardware design. Typical values ​​for Bo range from tens to hundreds.

[0140] If it is determined that context reduction is available, then in operation 3008, a context reduction operation is applied. The context reduction operation reduces the number of available contexts in the available context set to a smaller set containing fewer total contexts. Recall that the number of available contexts can depend in part on the bit position in the sequence, i.e., the index, because a context can depend on partial patterns of previously encoded bits from the bit sequence. In some implementations, the number of available contexts in the set before reduction can be based on the number of neighbor configurations multiplied by the number of partial patterns that may have previously encoded bits. For the bit at index i, where i ranges from 0 to n, the number of partial patterns can be increased by 2i. i Provided.

[0141] As described above, in some implementations, a context reduction operation is performed before encoding, and the resulting reduced context set is the set of contexts available to the encoder and decoder during the encoding operation. The use and / or selection of the reduced context set during encoding can be based on the evaluation of one or more conditions before determining the number of contexts that can be reduced using those reduced sets corresponding to the conditions evaluated in operation 3006. For example, in the case of a specific neighbor configuration that allows the use of a reduced context set, the encoder and / or decoder can first determine whether the neighbor configuration condition is met, and then, if so, use the corresponding reduced context set.

[0142] In operation 3010, bit b is determined based on the neighbor configuration and partial patterns of previously encoded bits in the bit sequence. i The context is selected from the set of available contexts (or a reduced set, if any). Then, in operation 3012, the current bit is entropy encoded by the binary encoder using the selected context.

[0143] If in operation 3014, index i indicates that the currently encoded bit is the last bit in the sequence, i equals i max If the condition is met, the encoding process proceeds to the next node. Otherwise, in operation 3016, index i is incremented, and the process returns to operation 3006.

[0144] It will be recognized that in some implementations, context selection may not depend on the neighbor configuration. In some cases, it may depend only on a partial pattern of previously encoded bits in the sequence, if any.

[0145] Figure 23 A simplified block diagram of a portion of the example encoder 3100 is shown in the figure. In this illustration, it should be understood that an occupancy pattern 3102 is obtained by dividing the corresponding volume into child nodes and looping through a FIFO buffer 3104 that maintains the geometry of the point cloud. Encoding of the occupancy pattern 3102 is illustrated as involving cascaded binary encoders 3106, each binary encoder for each bit of the pattern. Between at least some of the binary encoders 3106 is a context reduction operation 3108, which operates to reduce the available context to a smaller set of available contexts.

[0146] although Figure 23A series of binary encoders 3106 are shown, but in some implementations, only one binary encoder is used. When more than one encoder is used, the encoding can be (partially) parallelized. Given the contextual dependencies of preceding bits in a bit-to-bit sequence, the encoding of a pattern does not necessarily have to be fully parallelized, but pipelined processes can be improved by using cascaded binary encoders to achieve some degree of parallelization and speed improvement.

[0147] Context reduction operation

[0148] The example above suggests that the encoding process includes a context reduction operation with respect to at least one bit of the occupancy pattern, in order to reduce the set of available contexts to a smaller set of available contexts. In this sense, the "context reduction operation" can be understood as identifying and merging contexts in a specific bit b. i In cases where context is considered duplicate or redundant, it is treated as a duplicate or redundant context. As mentioned above, the reduced context set can be determined before encoding and can be provided to the encoder and decoder, who then determine whether to use the reduced context set based on the same conditions described below for reducing the context set.

[0149] Reduce by configuring neighbors with screening / shielding

[0150] The first example of context reduction involves reducing the number of neighbor configurations based on masking / masking. In principle, based on the fact that neighbor volumes help indicate whether the current volume or sub-volume is occupied, neighbor configurations incorporate the occupancy status of neighbor volumes as a factor into the context selection process. Bits associated with sub-volumes in the current volume are also used as factors in context selection when they are decoded; however, information from neighboring sub-volumes may be more important and informative than occupancy information from neighboring volumes located on the other side of the current sub-volume. In this sense, previously decoded bits are associated with sub-volumes that are "masked" or "masked" by neighboring volumes. This could mean that in this case, the occupancy of neighboring volumes can be ignored because the relevance of their occupancy status is encompassed by the occupancy status of sub-volumes between the current sub-volume and neighboring volumes, thus allowing for a reduction in the number of neighbor configurations.

[0151] Now for reference Figure 24This diagram illustrates an example context reduction operation based on neighbor masking. This example involves encoding an occupancy pattern for volume 3200. The occupancy pattern signals the occupancy status of eight sub-volumes within volume 3200. In this example, the four sub-volumes in the upper half of volume 3200 have been encoded. Therefore, their occupancy status is known. The bits of the encoded occupancy pattern are associated with a fifth sub-volume 3204, located in the lower half of volume 3200, below the four previously encoded sub-volumes.

[0152] The encoding in this example involves determining the context based on neighbor configurations. Ten neighbor configurations 3202 are shown. The volume 3200 containing the fifth sub-volume 3204 to be encoded is shown in light gray and indicated by reference numeral 3200. The neighbor configurations 3202 are based on the occupancy status of volumes adjacent to and sharing a face with volume 3200. Neighboring volumes include the top neighbor volume 3206.

[0153] In this example, the number of neighbor configurations can be reduced from 10 to 7 by ignoring the top neighbor volume 3206 in at least some configurations. Figure 24 As shown, three of the four configurations of the top neighbor volume 3206 can be included in the equivalent configuration of the top neighbor volume 3206 without being counted as a factor, thus reducing the number of neighbor configurations to a total of seven. Maintaining the configuration showing all six neighbor volumes remains advantageous because there are no existing five-volume neighbor configurations that can be consolidated into a six-volume configuration (eliminating five elements). This means that even if the top neighbor volume is removed, a new five-element neighbor configuration and a reduction in the non-global context will occur.

[0154] In this example, the top neighbor volume 3206 can be removed from the neighbor configuration because the context determination for encoding the occupancy bits associated with the fifth sub-volume 3204 will already take into account the occupancy status of the four previously encoded sub-volumes directly above it, and is a better indicator of the probability and directionality of the fifth sub-volume's occupancy compared to the occupancy status of the more distant top neighbor volume 3206.

[0155] The example above is merely one example. In this example, when the occupancy bits corresponding to the fifth sub-volume 3204 are encoded, the top neighboring volume 3206 is masked or occluded by the previously encoded sub-volume. Depending on the encoding order within volume 3200, many other possible masking / occlusion cases can be implemented and utilized to reduce the available neighbor configurations.

[0156] Now for reference Figure 25 , Figure 25A second example of masking / masking is shown. In this example, the occupancy pattern of volume 3200 is almost fully encoded. The sub-volume to be encoded is the eighth sub-volume, which is hidden (not visible) in the lower left corner of the figure. In this case, the occupancy states of all other seven sub-volumes have been encoded. In particular, the sub-volumes along the top (thus reducing the neighbor configuration to a total of seven) and the sub-volumes along the right and front. Thus, in addition to masking the top neighbor volume, the sub-volume with the previously encoded occupancy bits also masks the front neighbor volume 3210 and the right neighbor volume 3212. This allows the neighbor configuration to be reduced from a total of seven to a total of five, as shown in the figure.

[0157] It should be understood that the two examples of masking described above are illustrative, and in some cases, different configurations can be combined to consider different masking scenarios. The context reduction operation of masking / masking based on previously encoded subvolumes is general and not limited to these two examples, although it will be understood that it cannot be applied to the case of the first subvolume to be encoded, as it requires at least one pre-encoded occupant bit associated with the nearby subvolume in order to perform any masking / masking.

[0158] It should also be understood that the degree of masking / masking performed to demonstrate a reduction in neighbor configuration can vary across different implementations. In the two examples above, all four sub-volumes sharing a face with a neighboring volume are pre-encoded before that neighboring volume is considered masked / masked and thus removed from the neighbor configuration. In other examples, partial masking / masking may be sufficient, for example, from one to three previously encoded sub-volumes sharing a face.

[0159] Context reduction through special case handling

[0160] In some cases, context reduction occurs without losing useful information. In the example context determination process described above, the context used to encode the occupancy bits is based on a neighbor configuration, i.e., the occupancy patterns of volumes adjacent to the current volume, and a partial pattern of occupancy based on sub-volumes within the current volume that can be attributed to previously encoded data. The latter case results in the need to track 2... 7 = 128 contexts, tracked relative to the eighth bit in the occupancy pattern bit sequence. Even if the neighbor configuration is reduced to a total of five, this means tracking 640 contexts.

[0161] Based on the fact that previously encoded bits in a bit sequence have an order, the number of contexts is large, and this order is related to the evaluation context. However, in some cases, the order may not contain useful information. For example, when the neighbor configuration is empty, i.e., N 10=0, which can be assumed that any point within the volume is sparsely filled, meaning they do not have sufficiently strong directionality to justify tracking individual contexts for different occupancy patterns in sibling sub-volumes. In the case of empty neighbor relationships, the point cloud has no local orientation or topology, meaning that 2-1 of previously encoded bits based on the bit sequence can be used to determine the orientation of the point cloud. j The number of conditions is reduced to j+1. That is, the context used to encode one of the bits in the bit sequence is based on the previously encoded bits, not on their ordered pattern, but only on their sum. In other words, the entropy expression in this special case can be expressed as:

[0162] H(b|n)≈H(b0|0)H(b1|0,b0)H(b2|0,b0+b1)...H(b7|0,b0+b1+...+b6)

[0163] In some implementations, similar observations can be made regarding the full neighbor configuration. In some examples, the full neighbor configuration lacks directionality, meaning that the order of previously encoded bits does not need to be considered when determining the context. In some examples, this context reduction operation can be applied only to some bits in a bit sequence, such as some later bits in that sequence. In some cases, applying this context reduction operation to later bits can be conditional upon determining that all previous bits associated with the previously encoded subvolume are also fully occupied.

[0164] Statistical context reduction

[0165] Statistical analysis can be used to reduce context by identifying which contexts lead to substantially the same statistical behavior and then combining them. This analysis can be performed a priori using test data to develop a reduced set of contexts, which is then provided to both the encoder and decoder. In some cases, analysis can be performed on the current point cloud using two-pass encoding to develop a custom reduced set of contexts for specific point cloud data. In some such cases, the decoder can be signaled with a mapping from the non-reduced set of contexts to the custom reduced set by using a specialized syntax encoded into the bitstream.

[0166] Two contexts can be compared using the concept of "distance". The first context c has a probability p that bit b is equal to zero, and the second context c' has a probability p' that bit b' is equal to zero. The distance between c and c' is:

[0167] d(c,c')=|p log2p-p'log2p'|+|(1-p)log2(1-p)-(1-p')log2(1-p')|

[0168] Using this similarity (distance) measurement, context can then be grouped during the process, such as:

[0169] 1. Start from the M1 context and fix the threshold level ε

[0170] 2. For a given context, regroup all contexts whose distance from the given context is less than a threshold level ε into one class.

[0171] 3. Repeat step 2 for all unreorganized contexts until all of them are placed into a single class.

[0172] 4. Label class M2 as from 1 to M2: This results in a brute force reduction function that maps {1,2,…,M1]→[1,2,…,M2], where M1≥M2.

[0173] The brute-force reduction function used to map a set of contexts to a smaller set of contexts can be stored in memory for application by the encoder / decoder as a context reduction operation during encoding. The mapping can be stored as a lookup table or other data structure. For example, the brute-force reduction function can be applied only to the later bits in a bit sequence (pattern).

[0174] Combinations and subcombinations of context-reduced operations

[0175] The above describes three example context reduction operations. In some implementations, each of them can be applied individually and independently. In some implementations, any two or more of them can be combined. Additional context reduction operations can be implemented individually or in combination with any one or more of the context reduction operations described above.

[0176] Figure 26 An example of a method 3300 involving occupancy pattern binary encoding for combined context reduction is illustrated in flowchart form. Given a neighbor configuration N with 10 elements in {0, 1, 2, ..., 9}. 10 Method 3300 encodes an 8-bit binary pattern b0, b1, ..., b7. The first condition for evaluation is whether the neighbor configuration is empty, i.e., N. 10 =0. If so, these bits are encoded without regard to their order, as shown in Figure 3302. Otherwise, these bits are encoded in the conventional manner until bit b4, at which point the encoder and decoder begin to reduce the number of contexts using the strong context reduction function BRi by mapping the set of contexts defined by the partial patterns of neighbor configurations and previously encoded bits to a smaller set of contexts with substantially similar statistical results.

[0177] In this example, the last two bits b6 and b7 are encoded using a reduced neighbor configuration based on masking / masking.

[0178] All functions can be implemented as lookup tables (LUTs) to reduce the size of the context set. In a practical implementation, all reductions come down to reduction functions, i.e., simple LUTs, that take the context as input and provide a reduced context as output. In this example embodiment, the total number of contexts has been reduced from 2550 to 576, and the output sizes of each reduction function BRi are 70, 106, 110, and 119, respectively.

[0179] Context selection in a system with a fixed number of contexts

[0180] Each of the previously described context reduction operations can be further used in a compression system with a static (fixed) minimum number of contexts. In such a design, for a given symbol in an 8-bit binary pattern, one or more reduction operations are applied to determine the context probability model used to encode or decode the symbol.

[0181] Impact on compressibility

[0182] Using a 10-neighbor configuration and non-binary encoding can provide compression gain for point cloud encoding relative to the current implementation of the MPEG test model. However, the concatenated binary encoding using a 10-neighbor configuration and 2550 contexts, as suggested above, results in even better compression efficiency. Even when reducing the total number of contexts to 576 using context reduction, as detailed above, binary encoding compression still slightly outperforms the implementation using non-binary encoding and is significantly better than the test model. This observation has been shown to be consistent across different test point cloud datasets.

[0183] Now for reference Figure 14 , Figure 14A simplified block diagram of an example embodiment of encoder 1100 is shown. Encoder 1100 includes processor 1102, memory 1104, and encoding application 1106. Encoding application 1106 may include a computer program or application stored in memory 1104 and containing instructions that, when executed, cause processor 1102 to perform operations as described herein. For example, encoding application 1106 may encode a bitstream according to the procedures described herein and output the encoded bitstream. It will be understood that encoding application 1106 may be stored on a non-transitory computer-readable medium, such as an optical disk, flash memory device, random access memory, hard disk drive, etc. When the instructions are executed, processor 1102 performs the operations and functions specified in the instructions to serve as a dedicated processor for implementing the above-described processes(s). In some examples, such a processor may be referred to as a "processor circuit" or a "processor circuit system".

[0184] Still referencing Figure 15 , Figure 15 A simplified block diagram of an example embodiment of decoder 1200 is shown. Decoder 1200 includes processor 1202, memory 1204, and decoding application 1206. Decoding application 1206 may include a computer program or application stored in memory 1204 and containing instructions that, when executed, cause processor 1202 to perform operations such as those described herein. It will be understood that decoding application 1206 may be stored on a computer-readable medium, such as an optical disc, flash memory device, random access memory, hard disk drive, etc. When the instructions are executed, processor 1202 performs the operations and functions specified in the instructions to act as a dedicated processor for implementing the above-described processing(s). In some examples, such a processor may be referred to as a "processor circuit" or a "processor circuit system".

[0185] It will be understood that the decoder and / or encoder according to this application can be implemented in many computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer-readable storage medium, including CDs, RAM, ROM, flash memory, etc.

[0186] It will be understood that the decoders and / or encoders described herein, as well as the modules, routines, procedures, threads, or other software components implementing the described methods / processes for configuring the encoder or decoder, can be implemented using standard computer programming techniques and languages. This application is not limited to specific processors, computer languages, computer programming conventions, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated circuit (ASIC), etc.

[0187] This application also provides a computer-readable signal that encodes data generated by applying the encoding process according to this application.

[0188] Certain modifications and alterations can be made to the described embodiments. Therefore, the embodiments discussed above are considered illustrative rather than limiting.

Claims

1. A method for encoding occupancy data of a point cloud to generate a bitstream of compressed point cloud data, wherein the occupancy of sub-volumes of a volume is indicated using a bit sequence, wherein each bit in the bit sequence indicates the occupancy of a corresponding sub-volume within the volume in scan order, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the method comprising: For at least one bit in the bit sequence of the volume, The context reduction condition is determined to be met, and on this basis, a reduced set of contexts is selected, which contains fewer contexts than the product of the neighbor configuration count and the number of previously encoded bits in the bit sequence; Based on the occupancy status of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence, a context for encoding the at least one bit is selected from the reduced set of contexts; as well as The at least one bit is entropy encoded using a binary entropy encoder based on the selected context to produce encoded data for the bit stream.

2. The method of claim 1, wherein the context reduction condition includes: One or more previously encoded occupied bits are identified as associated with one or more corresponding sub-volumes, the one or more corresponding sub-volumes being located between the sub-volume associated with the at least one bit and the one or more neighboring volumes in the neighboring volumes.

3. The method of claim 2, wherein the context reduction condition includes: The four sub-volumes associated with the previously encoded bits are identified as sharing a surface with specific neighboring volumes.

4. The method of claim 2, wherein the context reduction condition includes: It is determined that at least four bits of the bit sequence have been previously encoded.

5. The method of claim 1, wherein determining that the context reduction condition is satisfied includes determining that the occupancy pattern of the neighboring volumes indicates that the plurality of neighboring volumes are not occupied.

6. The method of claim 5, wherein, based on the determination that the plurality of neighboring volumes are unoccupied, the selected reduced set of contexts includes a plurality of contexts corresponding to the number of previously encoded bits in the bit sequence.

7. The method of claim 6, wherein selecting the context based on the determination that the plurality of neighboring volumes are unoccupied includes selecting the context based on the sum of previously encoded bits in the bit sequence.

8. The method of claim 1, wherein the context reduction condition includes: It is determined that at least a threshold number of bits in the bit sequence have been previously encoded, and the reduced set of contexts includes a lookup table that maps each possible combination of neighbor configurations and patterns of previously encoded bits in the bit sequence to the reduced set of contexts.

9. The method of claim 8, wherein the lookup table is generated based on: iteratively grouping the available contexts into multiple classes based on determining that the distance measurement between each pair of available contexts is less than a threshold, and wherein each of the multiple classes includes a corresponding context in the reduced set of contexts, and wherein an available context exists for each of the possible combinations of neighbor configuration and patterns of previously encoded bits in the bit sequence.

10. The method of claim 1, wherein the at least some of the neighboring volumes are neighboring volumes that share at least one face with the volume.

11. A method for decoding a bitstream of compressed occupancy data of a point cloud to produce a reconstructed point cloud, wherein the occupancy of sub-volumes of a volume is indicated using a bit sequence, wherein each bit in the bit sequence indicates the occupancy of a corresponding sub-volume within the volume in scan order, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the method comprising: For at least one bit in the bit sequence of the volume, The context reduction condition is determined to be met, and on this basis, a reduced set of contexts is selected, which contains fewer contexts than the product of the neighbor configuration count and the number of previously encoded bits in the bit sequence; Based on the occupancy status of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence, a context for encoding the at least one bit is selected from the reduced set of contexts; as well as The at least one bit is entropy decoded using a binary entropy decoder based on the selected context to generate reconstructed bits from the bit stream.

12. The method of claim 11, wherein the context reduction condition includes: One or more previously encoded occupied bits are identified as associated with one or more corresponding sub-volumes, the one or more corresponding sub-volumes being located between the sub-volume associated with the at least one bit and the one or more neighboring volumes in the neighboring volumes.

13. The method of claim 12, wherein the context reduction condition includes: The four sub-volumes associated with the previously encoded bits are identified as sharing a surface with specific neighboring volumes.

14. The method of claim 12, wherein the context reduction condition includes: It is determined that at least four bits of the bit sequence have been previously encoded.

15. The method of claim 11, wherein determining that the context reduction condition is satisfied includes determining that the occupancy pattern of the neighboring volumes indicates that the plurality of neighboring volumes are not occupied.

16. The method of claim 15, wherein, based on the determination that the plurality of neighboring volumes are unoccupied, the selected reduced set of contexts includes a plurality of contexts corresponding to the number of previously encoded bits in the bit sequence.

17. The method of claim 16, wherein selecting the context based on the determination that the plurality of neighboring volumes are unoccupied includes selecting the context based on the sum of previously encoded bits in the bit sequence.

18. The method of claim 11, wherein the context reduction condition includes: It is determined that at least a threshold number of bits in the bit sequence have been previously encoded, and the reduced set of contexts includes a lookup table that maps each possible combination of neighbor configurations and patterns of previously encoded bits in the bit sequence to the reduced set of contexts.

19. The method of claim 18, wherein the lookup table is generated based on: iteratively grouping available contexts into multiple classes based on determining that the distance measurement between each pair of available contexts is less than a threshold, and wherein each of the multiple classes includes a corresponding context in the reduced set of contexts, and wherein an available context exists for each of the possible combinations of neighbor configuration and patterns of previously encoded bits in the bit sequence.

20. The method of claim 11, wherein the at least some of the neighboring volumes are neighboring volumes that share at least one face with the volume.

21. An encoder for occupancy data of a point cloud to generate a bitstream of compressed point cloud data, wherein occupancy of sub-volumes of a volume is indicated using a bit sequence, each bit in the bit sequence indicating occupancy of a corresponding sub-volume within the volume in scan order, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the encoder comprising: processor; Memory; as well as An encoding application, the encoding application comprising instructions executable by the processor, the instructions, when executed, causing the processor to: For at least one bit in the bit sequence of the volume, Determine that the context reduction condition is met, and on this basis, select a reduced set of contexts, the reduced set of contexts containing contexts smaller than the product of the neighbor configuration count and the number of previously encoded bits in the bit sequence; Based on the occupancy status of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence, a context for encoding the at least one bit is selected from the reduced set of contexts; as well as The at least one bit is entropy encoded using a binary entropy encoder based on the selected context to produce encoded data for the bit stream.

22. A decoder for decoding a bitstream of compressed occupancy data of a point cloud to produce a reconstructed point cloud, wherein the occupancy of sub-volumes of a volume is indicated using a bit sequence, each bit in the bit sequence indicating the occupancy of a corresponding sub-volume within the volume in scan order, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the decoder comprising: processor; Memory; as well as A decoding application, the decoding application comprising instructions executable by the processor, the instructions, when executed, causing the processor to: For at least one bit in the bit sequence of the volume, Determine that the context reduction condition is met, and on this basis, select a reduced set of contexts, the reduced set of contexts containing contexts smaller than the product of the neighbor configuration count and the number of previously encoded bits in the bit sequence; Based on the occupancy status of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence, a context for encoding the at least one bit is selected from the reduced set of contexts; as well as A binary entropy decoder is used to entropy decode at least one bit based on the selected context to produce reconstructed bits from the bit stream.

Images