Method and device for binary entropy coding of point cloud

Abstract

A method and apparatus for binary entropy coding of point clouds. Binary entropy coding is used to encode a bit sequence that signals an occupancy pattern for a sub-volume of the volume. The context may be based on the neighbor configuration and a partial sequence of previously encoded bits of the sequence of bits. A determination is made as to whether a context reduction operation is applied, and if so, the operation reduces the number of available contexts. Exemplary context reduction operations include reducing neighbor configurations based on masking by sub-volumes associated with previously encoded bits, special processing of null neighbor configurations, and statistics-based context merging.

Description

[0001] Related Applications Citations

[0002] This application is a divisional application of the invention patent application with international application number PCT / CA2019 / 050399, international application date April 3, 2019, date of entry into the Chinese national phase December 8, 2020, Chinese national application number 201980039083.6, and invention name “Method and device for binary entropy coding of point clouds”. Technical Field

[0003] The present application relates generally to point cloud compression, and more particularly to methods and apparatus for binary entropy encoding of point clouds. Background Art

[0004] Data compression is used in communications and computer networking to efficiently store, transmit, and reproduce information. There is a growing interest in the representation of three-dimensional objects or spaces, as it may involve large data sets for which efficient and effective compression would be very useful and valuable. In some cases, a three-dimensional object or space can be represented using a point cloud, which is a collection of points, each with three coordinate positions (X, Y, Z) and, in some cases, other properties such as color data (e.g., brightness and chromaticity), transparency, reflectivity, normal vector, etc. Point clouds can be static (a fixed object or a snapshot of an environment / object at a single point in time) or dynamic (a time series of point clouds).

[0005] Example applications of point clouds include terrain and mapping applications. Autonomous vehicles and other machine vision applications may rely on point cloud sensor data in the form of 3D scans of the environment, such as from LiDAR scanners. Virtual reality simulations may rely on point clouds.

[0006] It will be appreciated that point clouds may involve large amounts of data, and that it is important to compress (encode and decode) this data quickly and accurately. It would therefore be advantageous to provide methods and apparatus for more efficiently and / or effectively compressing point cloud data. Furthermore, it would be advantageous to find methods and apparatus for encoding point clouds that can be implemented using context-adaptive binary entropy coding without having to manage an excessive amount of context. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Reference will now be made, by way of example, to the accompanying drawings which show example embodiments of the present application, in which:

[0008] Figure 1 A simplified block diagram of an example point cloud encoder is shown;

[0009] Figure 2 A simplified block diagram of an example point cloud decoder is shown;

[0010] Figure 3 An example partial subvolume and associated tree structure for encoding is shown;

[0011] Figure 4 shows the recursive partitioning and encoding of the octree;

[0012] Figure 5 An example scan pattern within an example cube from an octree is shown;

[0013] Figure 6 An example occupancy pattern within an example cube is shown;

[0014] Figure 7 An example method for encoding a point cloud is shown in flowchart form;

[0015] Figure 8 A portion of an example octree is shown;

[0016] Fig. 9 An example of a neighbor subvolume is shown;

[0017] Fig.10 An example neighbor configuration is shown, which illustrates occupancy between neighbor nodes;

[0018] Fig.11 An exemplary embodiment of a process for point cloud entropy encoding using parent-pattern dependent context is schematically illustrated;

[0019] Fig.12 An illustrative embodiment of a process for point cloud entropy encoding using a context dependent neighbor configuration is shown;

[0020] Fig.13 An example method for decoding a bitstream of compressed point cloud data is shown in flowchart form;

[0021] Fig.14 An example simplified block diagram of an encoder is shown;

[0022] Fig.15 An example simplified block diagram of a decoder is shown;

[0023] Fig.16 An example Cartesian coordinate system and example rotations and / or reflections about an axis are shown;

[0024] Fig.17 A class showing the invariance of the neighbor configuration under one or several iterations of rotation about the Z axis;

[0025] Fig.18 shows the class of invariance for neighbor configurations for vertical reflections;

[0026] Fig.19 A class showing invariance to both rotation and reflection;

[0027] Fig. 20 The class of invariance under three rotations and reflection is shown;

[0028] Fig.21 The equivalent relationship between non-binary coding and concatenated binary coding for occupancy patterns is shown;

[0029] Fig. 22 An example method for encoding occupancy patterns in a tree-based point cloud encoder using binary encoding is shown in flowchart form;

[0030] Fig.23 A simplified block diagram of a portion of an example encoder is shown;

[0031] Fig.24 An example context reduction operation based on neighbor masking is schematically illustrated;

[0032] Fig.25 Another example context reduction operation based on neighbor masking is shown; and

[0033] Fig.26 An example of a method for binary encoding occupancy patterns using combined context reduction is shown in flowchart form.

[0034] Similar reference numerals may have been used in different drawings to denote similar components. DETAILED DESCRIPTION

[0035] The present application describes methods for encoding and decoding point clouds, as well as encoders and decoders for encoding and decoding point clouds. A bit sequence can be encoded using binary entropy coding, which signals an occupancy pattern for a subvolume of a volume. The context can be based on a neighbor configuration and a partial sequence of previously encoded bits of the bit sequence. It is determined whether a context reduction operation is applied, and if so, the operation reduces the number of available contexts. Example context reduction operations include reducing neighbor configurations based on masking of subvolumes associated with previously encoded bits, special handling of empty neighbor configurations, and statistics-based context merging. The reduction can be applied before encoding, and it can be determined during encoding whether a situation for using a reduced set of contexts is met.

[0036] In one aspect, the present application provides a method for encoding a point cloud to generate a bit stream of compressed point cloud data, wherein the point cloud is defined in a tree structure, the tree structure has a plurality of nodes, the plurality of nodes have a parent-child relationship and represent a geometric structure of a volume space, the volume space is recursively divided into sub-volumes, and contains the points of the point cloud, wherein the occupancy of the sub-volumes of the volume is indicated using a bit sequence, each bit in the bit sequence indicates the occupancy of a corresponding sub-volume in a scanning order within the volume, and wherein the volume has a plurality of neighbor volumes, the occupancy pattern of the neighbor volumes is a neighbor configuration, the method comprising: for the bit sequence of the volume at least one bit in a sequence, determining that a context reduction condition is satisfied, and on this basis, selecting a reduced context set, the reduced context set comprising contexts that are smaller than the product of a count of neighbor configurations and the number of previously encoded bits in the sequence; selecting a context for encoding the at least one bit from the reduced context set based on the occupancy state of at least some of the neighbor volumes and at least one previously encoded bit of the bit sequence; entropy encoding the at least one bit based on the selected context using a binary entropy encoder to generate encoded data for the bit stream; and updating the selected context.

[0037] On the other hand, the present application provides a method for decoding a bit stream of compressed point cloud data to produce a reconstructed point cloud, wherein the point cloud is defined in a tree structure, the tree structure has multiple nodes, the multiple nodes have a parent-child relationship, and represent the geometric structure of a volume space, the volume space is recursively divided into sub-volumes, and contains the points of the point cloud, wherein the occupancy of the sub-volumes of the volume is indicated using a bit sequence, each bit in the bit sequence indicates the occupancy of the corresponding sub-volume in the volume in a scanning order, and wherein the volume has multiple neighbor volumes, and the occupancy pattern of the neighbor volumes is a neighbor configuration. The decoding method comprises: determining, for at least one bit in the bit sequence of the volume, that a context reduction condition is satisfied, and on this basis, selecting a reduced context set, wherein the reduced context set contains contexts that are smaller than the product of a count of neighbor configurations and the number of previously encoded bits in the sequence; selecting a context from the reduced context set for encoding the at least one bit based on the occupancy status of at least some of the neighbor volumes and at least one previously encoded bit of the bit sequence; entropy decoding the at least one bit based on the selected context using a binary entropy decoder to generate a reconstructed bit from the bit stream; and updating the selected context.

[0038] In some implementations, the context reduction condition includes determining that one or more previously encoded occupied bits are associated with one or more corresponding subvolumes, the one or more corresponding subvolumes being located between the subvolume associated with the at least one bit and the one or more neighbor volumes in the neighbor volumes. In some cases, this may include determining that four subvolumes associated with the previously encoded bits share a face with a particular neighbor volume.

[0039] In some implementations, the context reduction condition may include determining that at least four bits of the bit sequence have been previously encoded.

[0040] In some implementations, determining that the context reduction condition is satisfied comprises determining that the occupancy pattern of the neighbor volumes indicates that the plurality of neighbor volumes are unoccupied. In some of those cases, the selected reduced set of contexts may comprise a plurality of contexts corresponding to a number of previously encoded bits in a bit sequence, and optionally, selecting a context may comprise selecting a context based on a sum of previously encoded bits in a bit sequence.

[0041] In some implementations, the context reduction condition may include determining that at least a threshold number of bits in the bit sequence have been pre-encoded, and the reduced set of contexts may include a lookup table that maps each possible combination of a neighbor configuration and a pattern of previously encoded bits in the bit sequence to fewer contexts. In some examples, the lookup table may be generated based on the following: based on determining that a distance measure between each pair of available contexts is less than a threshold value, iteratively grouping the available contexts into a plurality of classes, and each of the plurality of classes may include a corresponding context in a smaller set, and for each possible combination of a neighbor configuration and a pattern of previously encoded bits in the bit sequence, there may be an available context.

[0042] In some implementations, at least some of the neighbor volumes are neighbor volumes that share at least one face with the volume.

[0043] In another aspect, the present application describes encoders and decoders configured to implement such encoding and decoding methods.

[0044] In another aspect, the present application describes a non-transitory computer-readable medium storing computer-executable program instructions that, when executed, cause one or more processors to perform the described encoding and / or decoding methods.

[0045] In another aspect, the present application describes a computer-readable signal containing program instructions which, when executed by a computer, cause the computer to perform the described encoding and / or decoding method.

[0046] Those skilled in the art will understand other aspects and features of the present application from a review of the following description of the examples in conjunction with the accompanying drawings.

[0047] Any features described in relation to one aspect or embodiment of the invention may also be used in relation to one or more other aspects / embodiments.These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described herein.

[0048] Sometimes in the following description, the terms "node", "volume", and "subvolume" may be used interchangeably. It will be appreciated that a node is associated with a volume or subvolume. A node is a specific point on a tree, which may be an internal node or a leaf node. A volume or subvolume is a finite physical space represented by a node. In some cases, the term "volume" may be used to represent the largest bounded space defined to contain a point cloud. For the purpose of establishing a tree structure of interconnected nodes to encode point cloud data, a volume may be recursively divided into a plurality of subvolumes.

[0049] In the present application, the term “and / or” is intended to cover all possible combinations and sub-combinations of the listed elements, including any one element, any sub-combination or all of the elements listed individually, without necessarily excluding other elements.

[0050] In the present application, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any one of the listed elements alone, any subcombination or all of the elements, without necessarily excluding any other elements and without necessarily requiring all elements.

[0051] A point cloud is a collection of points in a three-dimensional coordinate system. These points are usually intended to represent the outer surface of one or more objects. Each point has a position (location) in a three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be Cartesian coordinates or any other coordinate system. These points may have other associated attributes, such as color, and in some cases may also be three-component values, such as R, G, B or Y, Cb, Cr. Depending on the desired application of the point cloud data, other related attributes may include transparency, reflectivity, normal vector, etc.

[0052] Point clouds can be static or dynamic. For example, a detailed scan or map of an object or terrain might be static point cloud data. LiDAR-based environmental scans for machine vision purposes might be dynamic, in that the point cloud (at least potentially) changes over time, such as with each successive scan of a volume. A dynamic point cloud is therefore a time-ordered sequence of point clouds.

[0053] Point cloud data can be used in multiple applications, including conservation (scanning of historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous cars), and virtual reality systems, to give some examples. Dynamic point cloud data for applications such as machine vision may be quite different from static point cloud data used for conservation purposes. For example, automotive vision typically involves relatively small resolution, colorless, highly dynamic point clouds that are obtained by LiDAR (or similar) sensors with a high capture frequency. Such point clouds are not intended for human consumption or viewing, but for machine object detection / classification in the decision-making process. For example, a typical LiDAR frame contains tens of thousands of points, while high-quality virtual reality applications require millions of points. It can be expected that higher resolution data will be required over time as computing speeds increase and new applications emerge.

[0054] Although point cloud data is useful, the lack of effective and efficient compression, i.e. encoding and decoding processes, can hinder adoption and deployment. When encoding point clouds, a special challenge that does not arise in other data compression situations such as audio or video is encoding the geometry of the point cloud. Point clouds tend to be sparsely distributed, which makes efficiently encoding the locations of the points more challenging.

[0055] One of the more common mechanisms for encoding point cloud data is by using a tree-based structure. In a tree-based structure, a bounded three-dimensional volume of a point cloud is recursively partitioned into sub-volumes. The nodes of the tree correspond to sub-volumes. The decision whether to further partition a sub-volume can be based on the resolution of the tree and / or whether any points are contained in the sub-volume. A leaf node can have an occupancy flag that indicates whether its associated sub-volume contains a point. A segmentation flag can signal whether a node has child nodes (i.e., whether the current volume has been further partitioned into sub-volumes). In some cases, these flags can be entropy encoded, and in some cases, predictive coding can be used.

[0056] A commonly used tree structure is an octree. In this structure, the volume / subvolume is a cube, and each division of the subvolume produces another eight subvolumes / subcubes. Another commonly used tree structure is the KD tree, in which the volume (cube or rectangular cuboid) is recursively divided into two parts by a plane orthogonal to one of its axes. The octree is a special case of the KD tree, in which the volume is divided by three planes, each of which is orthogonal to one of the three axes. Both examples involve cubes or rectangular cuboids; however, the present application is not limited to this tree structure, and in some applications, the volume and subvolumes may have other shapes. The partitioning of the volume does not necessarily have to be divided into two subvolumes (KD tree) or eight subvolumes (octree), but may involve other partitions, including division into non-rectangular shapes or involving non-adjacent subvolumes.

[0057] For ease of explanation, the present application may refer to an octree because an octree is a popular candidate tree structure for automotive applications, but it should be understood that the methods and apparatus described herein may be implemented using other tree structures.

[0058] Reference now Figure 1 , which shows a simplified block diagram of a point cloud encoder 10 according to aspects of the present application. The point cloud encoder 10 includes a tree construction module 12 for receiving point cloud data and generating a tree (an octree in this example) that represents the geometry of the volume space containing the point cloud and indicates the positions or locations of points from the point cloud in the geometry.

[0059] The basic process for creating an octree to encode a point cloud can include:

[0060] 1. Start with a bounding volume (cube) that contains the point cloud in a coordinate system

[0061] 2. Divide the volume into 8 sub-volumes (eight sub-cubes)

[0062] 3. For each subvolume, mark the subvolume with 0 if it is empty and with 1 if it has at least one point.

[0063] 4. For all subvolumes marked as 1, repeat (2) to split these subvolumes until the maximum segmentation depth is reached

[0064] 5. For all leaf subvolumes (subcubes) of the maximum depth, mark the leaf subvolume with 1 if the leaf cube is not empty, otherwise with 0.

[0065] The process above can be described as an occupancy equals segmentation process, where segmentation implies occupancy, with the constraint that there is a maximum depth or resolution beyond which no further segmentation will occur. In this case, a single flag signals whether a node is segmented and therefore occupied by at least one point, and vice versa. At the maximum depth, the flag signals occupancy and no further segmentation can occur.

[0066] In some implementations, segmentation and occupation are independent, such that a node may be occupied and may or may not be segmented. There are two variants of this implementation:

[0067] 1. Split first, then occupy. A signal flag indicates whether a node is split. If split, the node must contain points - split means occupied. Otherwise, if the node is not split, another occupied flag signals whether the node contains at least one point. Therefore, when a node is not further split, that is, it is a leaf node, the leaf node must have an associated occupied flag to indicate whether it contains any points.

[0068] 2. Occupy first and then split. A single flag indicates whether the node is occupied. If it is not occupied, no split will occur. If it is occupied, a split flag is encoded to indicate whether the node is further split.

[0069] Regardless of which of the above processes is used to build the tree, it can be traversed in a predetermined order (breadth-first or depth-first, and according to the scanning mode / order within each divided sub-volume) to produce a series of bits (occupancy and / or segmentation flags) from the flags. This can be called serialization or binarization of the tree. Figure 1 As shown, in this example, the point cloud encoder 10 includes a binarizer 14 for binarizing the octree to generate a bit stream representing binarized data of the tree.

[0070] Then, the bit sequence can be encoded using an entropy encoder 16 to produce a compressed bit stream. The entropy encoder 16 can encode the bit sequence using a context model 18, which specifies the probability for encoding the bit based on the context determination of the entropy encoder 16. The context model 18 can be adaptively updated after encoding each bit or a defined bit set. In some cases, the entropy encoder 16 can be a binary arithmetic encoder. In some implementations, the binary arithmetic encoder can adopt context adaptive binary arithmetic coding (CABAC). In some implementations, an encoder other than an arithmetic encoder can be used.

[0071] In some cases, the entropy encoder 16 may not be a binary encoder, but may operate on non-binary data. The output octree data from the tree construction module 12 may not be evaluated in binary form, but may be encoded as non-binary data. For example, in the case of an octree, eight flags (e.g., occupancy flags) within a subvolume in their scan order may be considered as 2 8 -1 bit number (e.g., an integer value between 1 and 255, since the value 0 is impossible for a subvolume to be split, i.e., if it is completely unoccupied, it will not be split). In some implementations, this number can be encoded by an entropy encoder using a multi-symbol arithmetic encoder. Within a subvolume, such as a cube, the sequence of flags that define this integer can be called a "mode".

[0072] As with video or image encoding, point cloud encoding can include predictive operations in which an effort is made to predict the pattern of a subvolume. The prediction can be spatial (depending on previously encoded subvolumes in the same point cloud) or temporal (depending on previously encoded point clouds in a time-ordered sequence of point clouds).

[0073] exist Figure 2A block diagram of an example point cloud decoder 50 corresponding to encoder 10 is shown in . Point cloud decoder 50 includes an entropy decoder 52 that uses the same context model 54 used by encoder 10. Entropy decoder 52 receives an input bit stream of compressed data and entropy decodes the data to produce an output sequence of decompressed bits. This sequence is then converted to reconstructed point cloud data by a tree reconstructor 56. Tree reconstructor 56 reconstructs the tree structure based on the decompressed data and knowledge of the scan order in which the tree data was binarized. Thus, tree reconstructor 56 is able to reconstruct the positions of points from the point cloud (subject to the resolution of the tree encoding).

[0074] Figure 3 An exemplary partial subvolume 100 is shown in FIG. In this example, for ease of illustration, the subvolume 100 is shown in two dimensions and has a size of 16x16. It should be noted that the subvolume has been divided into four 8x8 subsquares, two of which have been further subdivided into 4x4 subsquares, three of which have been further divided into 2x2 subsquares, and one of the 2x2 subsquares has then been divided into 1x1 squares. The 1x1 square is the maximum depth of the tree and represents the highest resolution of the location point data. Points from the point cloud are shown as points in the figure.

[0075] The structure of the tree 102 is shown to the right of the subvolume 100. A sequence of segmentation flags 104 and a corresponding sequence of occupancy flags 106 obtained in a predefined breadth-first scan order are shown to the right of the tree 102. It will be observed that in this illustrative example, for each subvolume (node) that is not segmented there is an occupancy flag, i.e. it has an associated segmentation flag set to zero. These sequences can be entropy coded.

[0076] Figure 4 Another example using the occupied segmentation condition is shown in FIG. Figure 4 The recursive partitioning and encoding of the octree 150 is shown. Only a portion of the octree 150 is shown in the figure. The FIFO 152 is shown processing nodes for partitioning to illustrate the breadth-first nature of the process. The FIFO 152 outputs an occupied node 154 queued in the FIFO 152 for further partitioning after its parent node 156 is processed. The tree builder partitions the subvolume associated with the occupied node 154 into eight subvolumes (cubes) and determines their occupancy. The occupancy can be indicated by an occupancy flag for each subvolume. In a prescribed scan order, the flags can be referred to as the occupancy mode of the node 154. The mode can be specified by an integer representing a sequence of occupancy flags associated with the subvolumes in the pre-scan order. For the octree, the mode is an integer in the range [1,255].

[0077] The entropy encoder then encodes the pattern using a non-binary arithmetic encoder based on the probability specified by the context model. In this example, the probability can be a pattern distribution based on the initial distribution model and is updated adaptively. In one implementation, the pattern distribution is actually a counter of the number of times each pattern is encountered during encoding (an integer from 1 to 255). After encoding each subvolume, the pattern distribution can be updated. The pattern distribution can be normalized as needed, because the relative frequency of the pattern is closely related to the probability evaluation and not the absolute count.

[0078] Based on the pattern, then those child nodes that are occupied (eg, have flag = 1) are pushed into the FIFO 152 to be further split in turn (provided these nodes are not the maximum depth of the tree).

[0079] Reference now Figure 5 , Figure 5 An example cube 180 of an octree is shown. Cube 180 is subdivided into eight subcubes. The scan order of reading the flags will produce an eight-bit string that can be read in binary as an integer [1,255]. Based on the scan order and the final bit position of each subcube flag in the string, these subcubes have Figure 5 Assuming that both the encoder and decoder use the same scan order, the scan order can be any sequence of sub-cubes.

[0080] As an example, Figure 6 Cube 180 is shown with the four "front" sub-cubes occupied. The sub-cube based on occupancy is cube 1+4+16+64, which would correspond to pattern 85. The integer pattern number specifies the occupancy pattern in the sub-cube.

[0081] Octree representations, or more generally any tree representation, are efficient at representing points with spatial correlation because the tree tends to decompose the high-order bits of the point coordinates. For an octree, each depth level refines the coordinates of points within a subvolume by one bit per component at an overhead of eight bits per refinement. Further compression can be obtained by entropy encoding the segmentation information (i.e., the pattern) associated with each tree node. Further compression is possible because the patterns are not uniformly distributed - non-uniformity is another consequence of correlation.

[0082] One potential inefficiency in the current system is that a pattern distribution (e.g., a histogram of pattern numbers seen in previously encoded nodes of the tree) is developed in the process of encoding the point cloud. In some cases, the pattern distribution can be initialized to be equiprobable, or it can be initialized to some other predetermined distribution; but using one pattern distribution means that the context model does not consider or exploit local geometric correlations.

[0083] In European Patent Application No. 18305037.6, the applicant describes a method and apparatus for selecting from a distribution of available patterns to be used to encode an occupancy pattern of a particular node based on some occupancy information from previously encoded nodes in the vicinity of the particular node. In one example implementation, the occupancy information is obtained from an occupancy pattern parented to the particular node. In another example implementation, the occupancy information is obtained from one or more nodes adjacent to the particular node. European Patent Application No. 18305037.6 is incorporated herein by reference.

[0084] Reference now Figure 7 , which shows in flowchart form an example method 200 for encoding a point cloud. The method 200 in this example involves recursive partitioning of occupied nodes (subvolumes) and breadth-first traversal of the tree for encoding.

[0085] In operation 202, the encoder determines an occupancy pattern of a current node. The current node is an occupied node that has been divided into eight child nodes, each child node corresponding to a respective child cube. The occupancy pattern of the current node specifies the occupancy of the eight child nodes in a scanning order. As described above, an integer between 1 and 255 may be used to indicate such an occupancy pattern, such as an eight-bit binary string.

[0086] In operation 204, the encoder selects a probability distribution from a set of probability distributions. The selection of the probability distribution is based on some occupancy information from nearby previously encoded nodes, i.e., at least one node that is a neighbor of the current node. In some embodiments, two nodes are adjacent if they are associated with corresponding subvolumes that share at least one face. In a broader definition, if the nodes share at least one edge, they are neighbors. In a broader definition, if the two nodes share at least one vertex, they are adjacent. The parent mode of the current node being a child node provides occupancy data of the current node and seven sibling nodes to the current node. In some implementations, the occupancy information is a parent mode. In some implementations, the occupancy information is occupancy data of a set of neighbor nodes, the set of neighbor nodes including nodes that are at the same depth level of the tree as the current node but have different parent nodes. In some cases, combinations of these are possible. For example, a set of neighbor nodes may include some sibling nodes and some non-sibling nodes.

[0087] Once the probability distribution is selected, the encoder then entropy encodes the occupancy pattern of the current node using the selected probability distribution, as shown in operation 206. Then, in operation 208, the selected probability distribution is updated based on the occupancy pattern, for example, it can increase the count corresponding to the occupancy pattern. In operation 210, the encoder evaluates whether there are more nodes to encode, and if so, returns to operation 202 to encode the next node.

[0088] The probability distribution selection in operation 204 will be based on the occupancy data of nearby previously encoded nodes. This allows both the encoder and decoder to make the same selection independently. For the discussion of probability distribution selection below, reference will be made to Figure 8 , which schematically shows a partial octree 300 including a current node 302. The current node 302 is an occupied node and is being evaluated for encoding. The current node 302 is one of eight children of a parent node 306, which in turn is a child of a grandparent node (not shown). The current node 302 is divided into eight child nodes 304. The occupancy pattern of the current node 302 is based on the occupancy of the child nodes 304. For example, as shown in the figure, using the convention that black dots are occupied nodes, the occupancy pattern can be 00110010, i.e., pattern 50.

[0089] The current node 302 has a sibling node 308 that has the same parent node 306. The parent pattern is the occupancy pattern of the parent node 306, which as shown would be 00110000, i.e., pattern 48. The parent pattern can be used as a basis for selecting a suitable probability distribution for entropy encoding the occupancy pattern of the current node.

[0090] Fig. 9 The set of neighbors around the current node is shown, where neighbors are defined as nodes that share faces. In this example, the nodes / subvolumes are cubes, and the cube in the center of the image has six neighbors, one for each node. In an octree, it will be appreciated that the neighbors of the current node will include three sibling nodes. It will also include three nodes that do not have the same parent node. Therefore, occupancy data for some neighbor nodes will be available because they are siblings, but occupancy data for some neighbor nodes may or may not be available, depending on whether these nodes have been previously encoded. Special processing may apply to handle missing neighbors. In some implementations, missing neighbors may be assumed to be occupied or may be assumed to be unoccupied. It will be understood that the neighbor definition can be extended to include neighboring nodes based on shared edges or based on shared vertices to include additional neighbor subvolumes in the evaluation.

[0091] It will be appreciated that the foregoing process looks at the occupancy of nearby nodes in an attempt to determine the likelihood of occupancy of the current node 302 in order to select a more appropriate context(s) and entropy encode the occupancy data of the current node 302 using a more accurate probability. It will be appreciated that the occupancy state of neighbor nodes that share a face with the current node 302 may be a more accurate assessment of whether the current node 302 is likely to be isolated than an assessment based on the occupancy state of sibling nodes, three of which will only share one edge and one of which will only share one vertex (in the case of an octree). However, the assessment of the occupancy state of siblings has the advantage of being modular, as all relevant data for the assessment is part of the parent node, meaning that it has a smaller memory footprint available for implementation, whereas the assessment of neighbor occupancy state involves buffering tree occupancy data in case the tree occupancy data is needed when determining neighbor occupancy states related to encoding future nearby nodes.

[0092] The occupancy of neighbors can be read in a scan order that effectively assigns a value to each neighbor, much like described above with respect to occupancy patterns. As shown, neighbor nodes effectively take values ​​of 1, 2, 4, 8, 16, or 32, so there are 64 (0 to 63) possible neighbor occupancy configurations. This value may be referred to herein as a "neighborhood configuration." As an example, Fig.10 An example of a neighbor configuration 15 is shown, where neighbors 1, 2, 4, and 8 are occupied and neighbors 16 and 32 are empty.

[0093] In some cases, both criteria (parent mode and neighbor configuration) can be applied simultaneously or a choice can be made between the two. For example, if neighbors are available, a probability distribution selection can be made based on neighbor nodes; however, if one or more neighbors are not available because they are from nodes that have not yet been encoded, the probability distribution selection can revert to an analysis based on sibling nodes (parent mode).

[0094] In yet another embodiment, the probability distribution selection may be alternatively or additionally based on the grandparent mode. In other words, the probability distribution selection may be based on the occupancy state of the uncle node that is the brother of the parent node 306.

[0095] In yet another implementation, additional or alternative evaluations may be factored into the probability distribution selection. For example, the probability distribution selection may look at the occupancy status of the neighbor nodes of the parent node or the neighbor nodes of the grandparent node.

[0096] In some implementations, any two or more of the above criteria for evaluating local occupancy status may be used in combination.

[0097] In the case of a non-binary entropy encoder, the occupancy data of the current node can be encoded by selecting a probability distribution. The probability distribution contains a number of probabilities corresponding to the number of possible occupancy patterns of the current node. For example, in the case of encoding the occupancy pattern of an octree, there are 2 8 -1 = 255 possible modes, which means that each probability distribution contains 255 probabilities. In some embodiments, the number of probability distributions can be equal to the number of possible occupancy results in the selection criteria, i.e. using neighbor, sibling and / or parent occupancy data. For example, in the case of using the parent mode of the octree as the selection criterion for determining the probability distribution to use, there will be 255 probability distributions, each involving 255 probabilities. In the case of a neighbor configuration, if neighbors are defined as shared faces, there will be 64 probability distributions, each containing 255 probabilities.

[0098] It will be appreciated that due to the lack of data, i.e. context dilution, too many distributions may lead to slow adaptation. Therefore, in some embodiments, similar patterns may be grouped so as to use the same probability distribution. For example, a separate distribution may be used for patterns corresponding to fully occupied, vertically oriented, horizontally oriented, mainly empty, and then all other situations. This may reduce the number of probability distributions to approximately five. It should be appreciated that different groupings of patterns may be formed to result in different numbers of probability distributions.

[0099] Reference now Fig.11 , which diagrammatically illustrates an illustrative embodiment of a process 400 for point cloud entropy encoding using context that depends on a parent pattern. In this example, the current node 402 has been divided into eight child nodes, and its occupancy pattern 404 will be encoded using a non-binary entropy encoder 406. The non-binary entropy encoder 406 uses a probability distribution selected from one of six possible probability distributions 408. The selection is based on the parent pattern - that is, the selection is based on the occupancy information from the parent node to the current node 402. The parent pattern is identified by an integer between 1 and 255.

[0100] The choice of probability distribution can be a decision tree that evaluates whether the pattern corresponds to an entire node (e.g., pattern = 255), a horizontal structure (e.g., pattern = 170 or 85; assuming the Z axis is vertical), a vertical structure (e.g., pattern = 3, 12, 48, 192), a sparsely populated distribution (e.g., pattern = 1, 2, 4, 8, 16, 32, 64, or 128; i.e., no sibling nodes are occupied), a semi-sparsely populated distribution (a total of 3 occupied nodes between the current node and the sibling nodes), and all other cases. The example patterns indicated for the different categories are examples only. For example, the "horizontal" category may include patterns involving two or three occupied cubes on the same horizontal level. The "vertical" category may include patterns involving three or four occupied cubes arranged in a wall-like manner. It should also be understood that finer gradients may be used. For example, the "horizontal" category may be further subdivided into levels in the upper part of the cube and levels in the lower part of the cube, each with a different probability distribution. Other groupings of occupancy patterns with some relevance may be made and assigned to corresponding probability distributions. Further discussion on pattern grouping in the context of neighbor configurations and invariance between neighbor configurations is elaborated further below.

[0101] Fig.12 An exemplary embodiment of a process 500 for entropy encoding of a point cloud using a context dependent neighbor configuration is shown. This example assumes that the above combined Fig. 9 Definition of the neighbor and neighbor configuration numbers used. This example also assumes that each neighbor configuration has a dedicated probability distribution, which means there are 64 different probability distributions. The current node 502 has an occupancy pattern 504 to be encoded. The probability distribution is selected based on the nodes that are neighbors to the current node 502. That is, a neighbor configuration NC in [0,63] is found and used to select the associated probability distribution.

[0102] It should be understood that in some embodiments, neighbor configurations can be grouped so that more than one neighbor configuration uses the same probability distribution based on similarities in the pattern. In some embodiments, the process can use different arrangements of neighbors for contextualization (selection) of the distribution. Other neighbors can be added, such as eight neighbors diagonally adjacent on all three axes, or twelve neighbors diagonally adjacent on two axes. Embodiments that avoid specific neighbors can also be used, such as to avoid using neighbors that introduce additional dependencies in a depth-first scan, or to introduce dependencies only on specific axes in order to reduce the codec state of large trees.

[0103] In this example, the case of NC=0 is handled in a special way. If no neighbors are occupied, it can be indicated that the current node 502 is isolated. Therefore, the process 500 further checks how many children of the current node 502 are occupied. If only one child node is occupied, that is, NumberOccupied(NO) is equal to 1, then the flag is encoded to indicate that a single child node is occupied, and the index of the node is encoded using 3 bits. If multiple child nodes are occupied, the process 500 uses the NC=0 probability distribution to encode the occupancy pattern.

[0104] Reference now Fig.13 , Fig.13 An example method 600 for decoding a bitstream of encoded point cloud data is shown in flowchart form.

[0105] In operation 602, the decoder selects one of the probability distributions based on occupancy information from one or more nodes near the current node. As described above, the occupancy information can be a parent pattern from a parent node to the current node, i.e., the occupancy of the current node and its sibling nodes, or it can be the occupancy of a neighbor node to the current node, which may include some sibling nodes. Other or additional occupancy information may be used in some implementations.

[0106] Once the probability distribution has been selected, the decoder entropy decodes a portion of the bitstream using the selected probability distribution to reconstruct the occupancy pattern of the current node in operation 604. The occupancy pattern is used by the decoder to reconstruct the tree in order to reconstruct the encoded point cloud data. Once the point cloud data is decoded, it can be output from the decoder for use, such as for rendering views, segmentation / classification, or other applications.

[0107] In operation 606 , the decoder updates the probability distribution based on the reconstructed occupancy pattern, and then moves to the next node in the buffer and returns to operation 602 if there are more nodes to decode.

[0108] It has been demonstrated that example implementations of the above methods provide compression improvements with negligible increase in coding complexity. Neighbor-based selection has been shown to perform better than parent-based selection, despite its higher computational complexity and memory usage. In some tests, relative improvements in bits per point ranged from 4% to 20% relative to an MPEG point cloud test model. It has been noted that initializing the probability distribution based on a distribution derived from test data can improve performance compared to initialization with a uniform distribution.

[0109] Some of the above examples are based on a tree coding process that uses a non-binary encoder to signal the occupancy pattern. New developments that employ a binary entropy encoder are described further below.

[0110] In a variation of the neighbor-based probability distribution selection, the number of distributions can be reduced by exploiting the symmetry of the neighbors. By permuting the neighbors or permuting the pattern distribution, structurally similar configurations with lines of symmetry can reuse the same distribution. In other words, neighbor configurations that can use the same pattern distribution can be grouped into a class. A class containing more than one neighbor configuration can be referred to as a "neighbor configuration" in this article because one of the neighbor configurations effectively subsumes the other neighbor configurations by reflecting or permuting those other configurations.

[0111] As an example, consider eight corner patterns NC∈[21,22,25,26,37,38,41,42], each representing a symmetry of corner neighbor patterns. These values ​​of NC are likely to be associated with specific but different node patterns. These associated patterns are also likely to follow the same symmetry as the neighbor patterns. For example, a method can be implemented that reuses a single distribution to represent multiple cases of NC by permuting the probabilities of this distribution.

[0112] The encoder derives the mode number of the node based on the occupancy of the child nodes. The encoder selects the distribution and permutation function based on the neighborhood configuration. The encoder reorders the probabilities contained in the distribution according to the permutation function and then arithmetically encodes the mode number using the permuted distribution. The arithmetic encoder's updates to the probabilities of the permuted distribution are mapped back to the original distribution using the inverse permutation function.

[0113] The corresponding decoder first selects the same distribution and permutation function based on the neighbor configuration. The permutation distribution is generated in exactly the same way as the encoder, which is used by the arithmetic decoder to entropy decode the mode number. Then, the bits including the mode number are allocated to the corresponding children respectively.

[0114] It should be noted that the same permutation can be achieved without reordering the data of the distribution itself, but instead introducing a level of indirection and using a permutation function to permute the lookup to a given index in the distribution.

[0115] An alternative embodiment considers permutation of the modes themselves rather than the distribution, allowing shuffling before or after entropy encoding / decoding, respectively. This approach may be more suitable for efficient implementation via a bitwise shuffling operation. In this case, neither the encoder nor the decoder performs a reordering of the distribution, but rather modifies the calculation of the coding mode number to where c i is the occupancy state of the i-th child, and σ(i) is a permutation function. One such example permutation function allows The distribution of NC=22 is used for the distribution of NC=41. The decoder can use a permutation function to use to derive the occupancy state of the child nodes from the encoded mode number.

[0116] The method of deriving the required permutation can be based on the rotational symmetry of the neighbor configuration, or can be based on reflection along a specific axis. Furthermore, the permutation does not have to permute all positions according to, for example, symmetry; rather, partial permutations can be used. For example, when permuting NC=22 to NC=41, positions on the symmetry axis may not be permuted, resulting in a mapping, Positions 0, 2, 4, and 6 are not permuted. In other embodiments, only positions 1 and 7 are swapped.

[0117] An example of an embodiment based on rotational symmetry and reflection is provided below for the specific case of an octree with six neighbors that share common faces with the current cube. Without loss of generality, Fig.16 As shown, the Z axis extends vertically relative to the direction in which the figure is viewed. The relative position of neighbors, such as "above" (respectively "below"), should be understood as in increasing (respectively decreasing) Z direction along the Z axis. The same comments apply to left / right along the X axis and front / back along the Y axis.

[0118] Fig.16 Three rotations 2102, 2104 and 2106 along the Z, Y and X axes are shown respectively. The angles of these three rotations are 90 degrees, i.e. they rotate a quarter of a turn along their respective axes.

[0119] Fig.17 The invariance classes of neighbor configurations under one or more iterations of rotation 2102 along the Z axis are shown. This invariance represents the same statistical behavior of the point cloud geometry along any direction belonging to the XY plane. This is particularly true for the use case of a car driving on the surface of the earth locally approximated by the XY plane. A horizontal configuration is a given occupancy of four neighbors (located to the left, right, front, and back of the current cube), independent of the occupancy of the upper neighbor (2202) and the lower neighbor (2204). Under rotation 2102, the four horizontal configurations 2206, 2208, 2210, and 2212 belong to the same class of invariance. Similarly, the two configurations 2214 and 2216 belong to the same class of invariance. There are only six invariance classes under rotation 2102 (grouped under class set 2218).

[0120] The vertical configuration is a given occupancy of two neighbors 2202 and 2204, regardless of the occupancy of the four neighbors located to the left, right, front, and back of the current cube. Fig.18 As shown, there are four possible vertical configurations. Therefore, if the invariance with respect to rotation 2102 along the Z axis is considered, there are 6x4=24 possible configurations.

[0121] The reflection along the Z axis is as follows: Fig.16 shown. Fig.18 The vertical configurations 2302 and 2304 shown belong to the same invariance class under reflection 2108. There are three invariance classes under reflection 2108 (grouped under class 2306). Invariance under reflection 2108 means that the upward and downward directions behave essentially the same in terms of point cloud geometry. This is an accurate assumption for a car moving on a road.

[0122] If invariance under both rotation 2102 and reflection 2108 is assumed, then the product of the two sets 2218 and 2306 yields 18 invariance classes. These 18 classes are represented in Fig.19 middle.

[0123] Applying further invariance under two more rotations 2104 and 2106, the two configurations 2401 and 2402 belong to the same class of invariance. Furthermore, two configurations 2411, 2412, two configurations 2421, 2422, three configurations 2431, 2432, 2433, two configurations 2441, 2442, two configurations 2451, 2452, and finally two configurations 2461, 2462 belong to the same class. Thus, the invariance under three rotations (2102, 2104, and 2106) and reflection 2108 result in 10 classes of invariance, as Fig. 20 shown.

[0124] From the examples provided above, assuming that is invariant under three rotations and reflections, the number of valid neighbor configurations (ie, the classes into which the 64 neighbor configurations can be grouped) is 64, 24, 18, or 10.

[0125] Before entropy coding, the patterns undergo the same transformations, i.e. rotation and reflection, since the neighbor configurations do belong to one of the invariance classes. This preserves the statistical consistency between the invariant neighbor configurations and the encoded patterns.

[0126] It will also be appreciated that during a traversal of the tree, a child node will have certain neighbor nodes at the same tree depth that have been previously visited and can be used causally as dependencies. For these same-level neighbors, the parent's collocated neighbors may not be consulted, and the same-level neighbors may be used instead. Since same-level neighbors halve the size of the parent, one configuration considers a neighbor to be occupied if any of the four directly adjacent neighboring child nodes (i.e., the four child nodes that share a face with the current node) are occupied.

[0127] Entropy coded tree occupancy pattern using binary encoding

[0128] The above-mentioned technique of encoding tree occupancy using neighbor occupancy information is described in detail in European Patent Application No. 18305037.6. The described embodiments focus on non-binary entropy coding using occupancy patterns, where the pattern distribution is selected based on neighbor occupancy information. However, in some cases, the use of a binary encoder may be more efficient in terms of hardware implementation. Moreover, the instant update of many probabilities may require fast access to memory and calculations in the heart of the arithmetic encoder. Therefore, it may be advantageous to find methods and devices for entropy encoding occupancy patterns using a binary arithmetic encoder. If a binary encoder can be completed without significantly reducing compression performance while preventing an excessive number of contexts to be tracked, it would be advantageous to use a binary encoder.

[0129] The use of a binary encoder instead of a non-binary encoder is reflected in the entropy formula:

[0130] H(X1,X2|Y)=H(X1|Y)H(X2|Y,X1)

[0131] Where X = (X1, X2) is the non-binary information to be encoded and Y is the context for encoding, i.e. the neighbor configuration or the selected mode distribution. In order to convert the non-binary encoding of X into a binary encoding, the information (X1, X2) is split into information X1 and X2, which can be encoded separately without increasing entropy. To do this, one must encode one of the two according to the other, where X2 depends on X1. This can be extended to n bits of information in X. For example, for n = 3:

[0132] H(X1,X2,X3|Y)=H(X1|Y)H(X2|Y,X1)H(X3|Y,X1,X2)

[0133] It will be appreciated that as the occupancy pattern, i.e., the bit sequence X, becomes longer and longer, there are more conditions for encoding the following bits in the sequence. For a binary encoder (e.g., CABAC), this means that the number of contexts to be tracked and managed is greatly increased. Using an octree as an example, where the occupancy pattern is an eight-bit sequence b=b0...b7, the bit sequence can be divided into eight binary information bits b0...b7. The encoding can use a neighbor configuration N (or NC) to determine the context. Assuming that, as described above, by grouping neighbor configurations into invariance classes, the neighbor configuration can be reduced to 10 valid neighbor configurations, then N is an integer belonging to {0,1,2,...,9}. For the sake of abbreviation, the "class of invariant neighbor configurations" may sometimes be referred to herein as a "neighbor configuration", although it should be understood that this reduced number of neighbor configurations can be achieved based on the following: class-based grouping of neighbor configurations based on invariance.

[0134] Fig.21The eight-bit pattern or sequence is shown divided into eight separate bits for binary entropy coding. It will be noted that the first bit of the sequence is encoded based on the neighbor configuration, so there are ten available contexts. The next bit of the sequence is encoded based on the neighbor configuration and any previously encoded bit (i.e., bit b0). This involves a total of 20 available contexts: the product of 10 from N and 2 from b0. The final bit b7 is entropy encoded using a context selected from the 1280 available contexts: obtained as the product of 10 from N and 128 from the partial pattern given by the previously encoded bits b0,...,b6. That is, for each bit, the number of contexts (i.e., the possible combinations of conditions / dependencies) is the product of the number of defined neighbor configurations (in this example, based on grouping the 64 neighbor configuration pairs into 10) and the number of partial patterns that may exist from the ordered sequence of n-1 previously encoded bits (given by 2n-1).

[0135] As a result, there are a total of 2550 contexts to keep track of with the binary encoding of the occupancy pattern. This is a very large number of contexts to keep track of, and the relative scarcity may lead to poor performance due to context dilution, especially for later bits in the sequence.

[0136] Thus, in one aspect, the present application discloses encoders and decoders that determine whether a context set can be reduced and, if so, apply a context reduction operation to achieve a smaller set of available contexts for entropy encoding at least a portion of an occupancy pattern using a binary encoder. In another aspect, the present application also discloses encoders and decoders that apply one or more rounds of state reduction using the same context reduction operation to perform efficient context selection from a fixed number of contexts. In some implementations, context reduction is applied prior to generating a lookup table of contexts and / or algorithmic conditions, which is then used by the encoder or decoder in selecting an appropriate context. The reduction is based on a testable condition that the encoder and decoder evaluate to determine which lookup table to select from or how to index / select from the lookup table to obtain the selected context.

[0137] Reference now Fig. 22 , which shows in flowchart form an example method 3000 for encoding occupancy patterns in a tree-based point cloud encoder using binary encoding. The method 3000 can be implemented by an encoder or a decoder. In the case of an encoder, the encoding operation is encoding, while in the case of a decoder, the encoding operation is decoding. The encoding and decoding are context-based entropy encoding and decoding.

[0138] Example method 3000 is used to entropy encode an occupancy pattern, i.e., a bit sequence, of a particular node / volume. The occupancy pattern signals the occupancy status of the child nodes (subvolumes) of the node / volume. In the case of an octree, there are eight child nodes / subvolumes. In operation 3002, a neighbor configuration is determined. A neighbor configuration is the occupancy status of one or more volumes adjacent to the volume for which the occupancy pattern is to be encoded. As described above, there are various possible implementations for determining a neighbor configuration. In some examples, there are 10 neighbor configurations, and the neighbor configuration of the current volume is identified based on the occupancy of six volumes that share a face with the current volume.

[0139] In operation 3004, the index i of the child node to the current volume is set to 0. Then in operation 3006, an evaluation is performed on whether context reduction is possible. Different possible context reduction operations will be discussed in detail below. For example, it can be evaluated whether context reduction is possible based on which bit (e.g., index value) in the bit sequence is encoded. In some cases, context reduction may be possible for the last few bits in the sequence, not for the first few bits. The evaluation of whether context reduction is possible can be based on neighbor configurations, for example, because some neighbor configurations can allow simplification. In some implementations, other factors can be used to evaluate whether context reduction is possible. For example, an upper limit Bo can be provided as the maximum number of contexts that a binary encoder can use to encode bits, and if the initial number of contexts to be encoded bits is greater than Bo, context reduction is applied (otherwise no) so that the number of contexts after reduction is at most Bo. Such a boundary Bo can be defined in the encoder and / or decoder specification to ensure that software or hardware implementations that can handle Bo contexts will always be able to encode and / or decode point clouds without overflowing in terms of the number of contexts. Knowing the bounds of Bo in advance also allows anticipating the complexity and memory footprint caused by the binary entropy encoder, thus facilitating the design of the hardware. Typical values ​​of Bo are from tens to several hundred.

[0140] If it is determined that context reduction is available, then in operation 3008, a context reduction operation is applied. The context reduction operation reduces the number of available contexts in the set of available contexts to a smaller set containing less total context. It will be recalled that the number of available contexts may depend in part on the bit position, i.e., index, in the sequence, because the context may depend on a partial pattern of previously encoded bits from the bit sequence. In some implementations, before reduction, the number of contexts available in the set may be based on the number of neighbor configurations multiplied by the number of partial patterns that may have previously encoded bits. For a bit at index i, where i ranges from 0 to n, the number of partial patterns may be multiplied by 2. i Given.

[0141] As described above, in some implementations, a context reduction operation is performed prior to encoding, and the resulting reduced context set is the context set available to the encoder and decoder for use during the encoding operation. The use and / or selection of the reduced context set during encoding can be based on the evaluation of one or more conditions, which determine before using those reduced sets corresponding to the conditions evaluated in operation 3006 that the number of contexts can be reduced. For example, in the case of a particular neighbor configuration that allows the use of a reduced context set, the encoder and / or decoder can first determine whether the neighbor configuration condition is met, and then, if met, use the corresponding reduced context set.

[0142] In operation 3010, based on the neighbor configuration and a partial pattern of previously encoded bits in the bit sequence, a bit b is determined. i The context of , that is, selected from the set of available contexts (or reduced set, if any). Then in operation 3012, the current bit is entropy encoded by the binary encoder using the selected context.

[0143] If in operation 3014 the index i indicates that the currently encoded bit is the last bit in the sequence, i.e., i is equal to i max , the encoding process proceeds to the next node. Otherwise, the index i is increased in operation 3016, and the process returns to operation 3006.

[0144] It will be appreciated that in some implementations, context selection may not depend on the neighborhood configuration. In some cases, it may depend only on a partial pattern of previously encoded bits in the sequence, if any.

[0145] Fig.23 A simplified block diagram of a portion of an example encoder 3100 is shown in FIG. In this illustration, it should be understood that an occupancy pattern 3102 is obtained when the corresponding volume is divided into sub-nodes and looped through a FIFO buffer 3104 that holds the geometry of the point cloud. The encoding of the occupancy pattern 3102 is illustrated as involving a cascade of binary encoders 3106, one for each bit of the pattern. Between at least some of the binary encoders 3106 is a context reduction operation 3108 that operates to reduce the available context to a smaller set of available contexts.

[0146] although Fig.23A series of binary encoders 3106 are shown, but in some implementations only one binary encoder is used. In the case where more than one encoder is used, the encoding can be (partially) parallelized. Given a context dependency of a bit on the previous bits in the bit sequence, the encoding of the pattern does not necessarily have to be fully parallelized, but pipelining can be improved by using cascaded binary encoders for the pattern to achieve some degree of parallelization and speed improvement.

[0147] Context reduction operation

[0148] The above example proposes that the encoding process includes a context reduction operation on at least one bit of the occupancy pattern in order to reduce the set of available contexts to a smaller set of available contexts. In this sense, a "context reduction operation" can be understood as identifying and merging at least one bit of the occupancy pattern on a particular bit. i As described above, the reduced context set can be determined before encoding and can be provided to the encoder and decoder, and the encoder and decoder determine whether to use the reduced context set based on the same conditions for reducing the context set described below.

[0149] Reduced by neighbor configuration with screening / shielding

[0150] A first example context reduction operation involves reducing the number of neighbor configurations based on shielding / occlusion. In principle, the neighbor configuration factors in the occupancy state of neighboring volumes into the context selection process, based on the neighboring volumes helping to indicate whether the current volume or subvolume is occupied. When bits related to subvolumes in the current volume are decoded, they will also be factored into the context selection; however, information from neighboring subvolumes may be more important and more informative than occupancy information of neighboring volumes located on the other side of the subvolume from the current subvolume. In this sense, the previously decoded bits are associated with subvolumes that "shield" or "occlude" the neighboring volume. This may mean that in this case, the occupancy of the neighbor volume can be ignored because the relevance of its occupancy state is subsumed by the occupancy state of the subvolumes between the current subvolume and the neighbor volume, thereby allowing a reduction in the number of neighbor configurations.

[0151] Reference now Fig.24, which diagrammatically illustrates an example context reduction operation based on neighbor masking. The example involves encoding an occupancy pattern of volume 3200. The occupancy pattern signals the occupancy status of eight subvolumes within volume 3200. In this example, four subvolumes in the upper half of volume 3200 have been encoded. Their occupancy status is therefore known. The bits of the encoded occupancy pattern are associated with a fifth subvolume 3204, which is located in the lower half of volume 3200, below the four previously encoded subvolumes.

[0152] The encoding in this example includes determining context based on neighbor configurations. Ten neighbor configurations 3202 are shown. Volume 3200 containing a fifth subvolume 3204 to be encoded is shown in light grey and indicated by reference numeral 3200. Neighbor configurations 3202 are based on the occupancy state of volumes adjacent to volume 3200 and sharing faces with it. Neighbor volumes include a top neighbor volume 3206.

[0153] In this example, by ignoring the top neighboring volume 3206 in at least some configurations, the number of neighbor configurations can be reduced from 10 to 7. Fig.24 3206 can be relegated to an equivalent configuration that does not factor into the top neighbor volume 3206, thereby reducing the number of neighbor configurations to a total of 7. Keeping the configuration showing all six neighbor volumes is still advantageous because there is no existing 5-volume neighbor configuration (eliminating 5 elements) that can be consolidated into the 6-volume configuration, which means that even if the top neighbor volume is removed, a new 5-element neighbor configuration and a non-holistic reduction in context will occur.

[0154] In this example, the top neighboring volume 3206 can be removed from the neighbor configuration because the context determination for the encoding of the occupancy bits associated with the fifth subvolume 3204 will have taken into account the occupancy states of the four previously encoded subvolumes directly above it, which is a better indicator of the likelihood and directionality of occupancy of the fifth subvolume than the occupancy state of the more distant top neighboring volume 3206.

[0155] The above example is just one example, in which the top neighboring volume 3206 is shielded or obscured by the previously encoded subvolume when encoding the occupancy bits corresponding to the fifth subvolume 3204. Depending on the encoding order within the volume 3200, many other possible shielding / obstruction situations can be realized and utilized to reduce the available neighbor configurations.

[0156] Reference now Fig.25 , Fig.25A second example of shielding / masking is shown. In this example, the occupancy pattern of volume 3200 is almost completely encoded. The subvolume to be encoded is the eighth subvolume and is hidden (not visible) in the lower left corner of the figure. In this case, the occupancy status of all seven other subvolumes has been encoded. In particular, the subvolume along the top (thus reducing the neighbor configuration to a total of seven) and the subvolumes along the right and front sides. Therefore, in addition to shielding the top neighbor volume, the subvolume with the previously encoded occupancy bit also shields the front neighbor volume 3210 and the right neighbor volume 3212. This can allow the neighbor configuration to be reduced from a total of seven to a total of five, as shown in the figure.

[0157] It will be appreciated that the two preceding examples of masking are illustrative and that in some cases different configurations may be incorporated to account for different masking situations. The context reduction operation based on masking / occlusion of previously encoded subvolumes is general and not limited to these two examples, although it will be appreciated that it cannot be applied to the case of the first subvolume to be encoded, as it requires at least one pre-encoded occupied bit associated with the nearby subvolumes in order for any masking / occlusion to occur.

[0158] It should also be understood that the degree of shielding / masking performed to demonstrate a reduction in neighbor configuration may vary in different implementations. In the two examples above, all four subvolumes that share a face with a neighbor volume have been pre-encoded before the neighbor volume is considered shielded / masked and thus removed from the neighbor configuration. In other examples, partial shielding / masking may be sufficient, such as from one to three previously encoded subvolumes that share a face.

[0159] Context reduction through special case handling

[0160] In some cases, context reduction occurs without losing useful information. In the example context determination process described above, the context used to encode the occupancy bits is based on the neighbor configuration, i.e., the occupancy pattern of volumes adjacent to the current volume, and on the partial pattern of occupancy attributable to subvolumes in the current volume that have been previously encoded. The latter case results in tracking 2 7 = 128 contexts to track relative to the eighth bit in the occupancy pattern bit sequence. Even if the neighbor configuration is reduced to a total of five, this means 640 contexts to track.

[0161] The number of contexts is large, based on the fact that the previously encoded bits of the bit sequence have an order, and this order is relevant to the evaluation context. However, in some cases, the order may not contain useful information. For example, in the case where the neighbor configuration is empty, that is, N 10= 0, any points within the volume can be assumed to be sparsely populated, which means that they do not have strong enough directionality to justify tracking separate contexts for different occupancy patterns in sibling subvolumes. In the case of empty neighbor relations, the point cloud has no local orientation or topology, which means that the 2 previously encoded bits based on the bit sequence can be j conditions is reduced to j+1 conditions. That is, the context used to encode one of the bits of the bit sequence is based on the previously encoded bits, not on their ordered pattern, but only on their sum. In other words, the entropy expression in this special case can be expressed as:

[0162] H(b|n)≈H(b0|0)H(b1|0,b0)H(b2|0,b0+b1)...H(b7|0,b0+b1+...+b6)

[0163] In some implementations, similar observations can be made with respect to full neighbor configurations. In some examples, full neighbor configurations lack directionality, meaning that the order of previously encoded bits need not be considered when determining context. In some examples, this context reduction operation can be applied to only some of the bits in a bit sequence, such as some later bits in the sequence. In some cases, applying this context reduction operation to later bits can be conditioned on determining that previous bits associated with previously encoded subvolumes are also fully occupied.

[0164] Statistics-based context reduction

[0165] Statistical analysis can be used to reduce contexts by determining which contexts result in approximately the same statistical behavior and then combining them. This analysis can be performed a priori using test data to develop a reduced set of contexts, which are then provided to both the encoder and decoder. In some cases, the analysis can be performed on the current point cloud using two-pass encoding to develop a custom reduced set of contexts for specific point cloud data. In some such cases, the mapping from a non-reduced set of contexts to a custom reduced set of contexts can be signaled to the decoder using a dedicated syntax encoded into the bitstream.

[0166] Two contexts can be compared by the concept of "distance". The first context c has a probability p that bit b is equal to zero, and the second context c' has a probability p' that bit b' is equal to zero. The distance between c and c' is:

[0167] d(c,c')=|p log2p-p'log2p'|+|(1-p)log2(1-p)-(1-p')log2(1-p')|

[0168] Using this measure of similarity (distance), contexts can then be grouped in processes such as:

[0169] 1. Start with the M1 context and fix the threshold level ε

[0170] 2. For a given context, regroup all contexts whose distance to the given context is less than a threshold level ε into one category.

[0171] 3. Repeat 2 for all unreorganized contexts until all of them are put into one class

[0172] 4. Labeling class M2 from 1 to M2: This leads to a brute force reduction function that maps {1,2,…,M1]→[1,2,…,M2], where M1≥M2.

[0173] A brute force reduction function for mapping a context set to a smaller context set may be stored in a memory to be applied by an encoder / decoder as a context reduction operation during encoding. The mapping may be stored as a lookup table or other data structure. For example, a brute force reduction function may be applied only to the following bits in a bit sequence (pattern).

[0174] Combination and subcombination of contextual reduction operations

[0175] Three example context reduction operations are described above. In some implementations, each of them can be applied separately and independently. In some implementations, any two or more of them can be combined. Additional context reduction operations can be implemented separately or in combination with any one or more of the above context reduction operations.

[0176] Fig.26 An example of a method 3300 involving combined context-reduced occupancy pattern binary encoding is shown in flowchart form. Given a neighborhood configuration N with 10 elements in {0, 1, 2, ..., 9} 10 , method 3300 encodes an 8-bit binary pattern b0, b1, ..., b7. The first condition evaluated is whether the neighbor configuration is empty, that is, N 10 = 0. If so, the bits are encoded without reference to their order, as indicated by reference numeral 3302. Otherwise, the bits are encoded in a conventional manner until bit b4, at which point the encoder and decoder begin to use a brute force context reduction function BRi to reduce the number of contexts by mapping the set of contexts defined by the neighborhood configuration and partial patterns of previously encoded bits to a smaller set of contexts with substantially similar statistics.

[0177] In this example, the last two bits, b6 and b7, are encoded using a reduced neighbor configuration based on masking / obscuration.

[0178] All functions can be implemented as lookup tables (LUTs) for reducing the size of context sets. In a practical implementation, all reductions boil down to reduction functions, i.e., simple LUTs, which take contexts as input and provide reduced contexts as output. In this example embodiment, the total number of contexts has been reduced from 2550 to 576, with the output sizes of each reduction function BRi being 70, 106, 110, and 119, respectively.

[0179] Context selection in a system with a fixed number of contexts

[0180] Each previously described context reduction operation can be further used in a compression system with a static (fixed) minimum number of contexts. In such a design, for a given symbol in an 8-bit binary pattern, one or more reduction operations are applied to determine a context probability model for encoding or decoding the symbol.

[0181] Impact on compression performance

[0182] Using a 10 neighbor configuration and non-binary encoding provides compression gains for point cloud coding relative to current implementations of the MPEG test model. However, the above proposed concatenated binary encoding using a 10 neighbor configuration and using 2550 contexts leads to a much better improvement in compression efficiency. Even when reducing the contexts to 576 total using context reduction such as the three techniques detailed above, the binary encoding compression still slightly outperforms the implementation using non-binary encoding and is much better than the test model. This observation has been shown to be consistent between different test point cloud data.

[0183] Reference now Fig.14 , Fig.14A simplified block diagram of an example embodiment of an encoder 1100 is shown. The encoder 1100 includes a processor 1102, a memory 1104, and an encoding application 1106. The encoding application 1106 may include a computer program or application that is stored in the memory 1104 and contains instructions that, when executed, cause the processor 1102 to perform operations such as those described herein. For example, the encoding application 1106 may encode a bitstream according to the process described herein and output the encoded bitstream. It will be understood that the encoding application 1106 may be stored on a non-transitory computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions so as to be used as a dedicated processor for implementing the above-mentioned (multiple) processes. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system".

[0184] Now also refer to Fig.15 , Fig.15 A simplified block diagram of an example embodiment of a decoder 1200 is shown. The decoder 1200 includes a processor 1202, a memory 1204, and a decoding application 1206. The decoding application 1206 may include a computer program or application that is stored in the memory 1204 and contains instructions that, when executed, cause the processor 1202 to perform operations such as those described herein. It will be understood that the decoding application 1206 may be stored on a computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions so as to function as a dedicated processor to implement the above-mentioned (multiple) processes. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system".

[0185] It will be appreciated that decoders and / or encoders according to the present application can be implemented in many computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. Decoders or encoders can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. Software instructions can be stored on any suitable non-transient computer-readable memory, including CD, RAM, ROM, flash memory, etc.

[0186] It will be understood that the decoder and / or encoder described herein and the modules, routines, processes, threads or other software components that implement the described methods / processes for configuring the encoder or decoder can be implemented using standard computer programming techniques and languages. The application is not limited to specific processors, computer languages, computer programming conventions, data structures, other such implementation details. Those skilled in the art will recognize that the described process can be implemented as a part of a computer executable code stored in a volatile or non-volatile memory, as a part of an application specific integrated chip (ASIC), etc.

[0187] The present application also provides a computer readable signal encoding data produced by application of an encoding process according to the present application.

[0188] Certain modifications and adaptations may be made to the described embodiments.Therefore, the embodiments discussed above are considered to be illustrative rather than restrictive.

Claims

1. A method of encoding occupancy data of a point cloud to generate a bit stream of compressed point cloud data, wherein occupancy of sub-volumes of a volume is indicated using a bit sequence, wherein each bit in the bit sequence indicates the occupancy of a corresponding sub-volume in a scan order within the volume, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the method comprising: at least one bit in said bit sequence for said volume, determining that a context reduction condition is satisfied and, based thereon, selecting a reduced set of contexts containing contexts that are smaller than a product of a count of neighbor configurations and a number of previously encoded bits in the sequence; selecting a context for encoding the at least one bit from the reduced set of contexts based on occupancy states of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence; as well as The at least one bit is entropy encoded based on the selected context using a binary entropy encoder to produce encoded data for the bitstream.

2. The method according to claim 1, wherein the context reduction condition comprises: It is determined that one or more previously encoded occupied bits are associated with one or more corresponding subvolumes located between the subvolume associated with the at least one bit and the one or more neighboring volumes of the neighboring volumes.

3. The method according to claim 2, wherein the context reduction condition comprises: It is determined that four subvolumes associated with previously encoded bits share faces with a particular neighboring volume.

4. The method according to claim 2, wherein the context reduction condition comprises: It is determined that at least four bits of the bit sequence have been previously encoded. 5 . The method of claim 1 , wherein determining that the context reduction condition is satisfied comprises determining that the occupancy pattern of the neighboring volumes indicates that the plurality of neighboring volumes are unoccupied. 6 . The method of claim 5 , wherein based on a determination that the plurality of neighboring volumes are unoccupied, the selected reduced set of contexts comprises a number of contexts corresponding to a number of previously encoded bits in the bit sequence. 7 . The method of claim 6 , wherein based on the determination that the plurality of neighboring volumes are unoccupied, selecting the context comprises selecting the context based on a sum of previously encoded bits in the bit sequence.

8. The method of claim 1, wherein the context reduction condition comprises: It is determined that at least a threshold number of bits in the bit sequence have been previously encoded, and wherein the reduced set of contexts includes a lookup table that maps each possible combination of a neighbor configuration and a pattern of previously encoded bits in the bit sequence to fewer contexts.

9. A method according to claim 8, wherein the lookup table is generated based on the following items: iteratively grouping the available contexts into multiple classes based on determining that the distance measure between each pair of available contexts is less than a threshold, and wherein each of the multiple classes includes a corresponding context in the smaller set, and wherein there is an available context for each possible combination of neighbor configuration and pattern of previously encoded bits in the bit sequence.

10. The method of claim 1, wherein the at least some of the neighboring volumes are neighboring volumes that share at least one face with the volume.

11. A method of decoding a bit stream of compressed occupancy data of a point cloud to produce a reconstructed point cloud, wherein occupancy of sub-volumes of a volume is indicated using a bit sequence, wherein each bit in the bit sequence indicates the occupancy of a corresponding sub-volume in a scan order within the volume, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the method comprising: at least one bit in said bit sequence for said volume, determining that a context reduction condition is satisfied and, based thereon, selecting a reduced set of contexts containing contexts that are smaller than a product of a count of neighbor configurations and a number of previously encoded bits in the sequence; selecting a context for encoding the at least one bit from the reduced set of contexts based on occupancy states of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence; as well as The at least one bit is entropy decoded based on the selected context using a binary entropy decoder to produce reconstructed bits from the bitstream.

12. The method of claim 11, wherein the context reduction condition comprises: It is determined that one or more previously encoded occupied bits are associated with one or more corresponding subvolumes located between the subvolume associated with the at least one bit and the one or more neighboring volumes of the neighboring volumes.

13. The method of claim 12, wherein the context reduction condition comprises: It is determined that four subvolumes associated with previously encoded bits share faces with a particular neighboring volume.

14. The method of claim 12, wherein the context reduction condition comprises: It is determined that at least four bits of the bit sequence have been previously encoded. 15 . The method of claim 11 , wherein determining that the context reduction condition is satisfied comprises determining that the occupancy pattern of the neighboring volumes indicates that the plurality of neighboring volumes are unoccupied.

16. The method of claim 15, wherein based on the determination that the plurality of neighboring volumes are unoccupied, the selected reduced set of contexts comprises a number of contexts corresponding to a number of previously encoded bits in the bit sequence.

17. The method of claim 16, wherein based on the determination that the plurality of neighboring volumes are unoccupied, selecting the context comprises selecting the context based on a sum of previously encoded bits in the bit sequence.

18. The method of claim 11, wherein the context reduction condition comprises: It is determined that at least a threshold number of bits in the bit sequence have been previously encoded, and wherein the reduced set of contexts includes a lookup table that maps each possible combination of a neighbor configuration and a pattern of previously encoded bits in the bit sequence to fewer contexts.

19. A method according to claim 18, wherein the lookup table is generated based on the following items: iteratively grouping the available contexts into multiple classes based on determining that the distance measure between each pair of available contexts is less than a threshold, and wherein each of the multiple classes includes a corresponding context in the smaller set, and wherein there is an available context for each possible combination of neighbor configuration and pattern of previously encoded bits in the bit sequence.

20. The method of claim 11, wherein the at least some of the neighboring volumes are neighboring volumes that share at least one face with the volume.

21. An encoder for occupancy data of a point cloud to generate a bit stream of compressed point cloud data, wherein occupancy of sub-volumes of a volume is indicated using a bit sequence, each bit in the bit sequence indicating the occupancy of a corresponding sub-volume in a scan order within the volume, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the encoder comprising: processor; Memory; as well as a coded application comprising instructions executable by the processor, the instructions, when executed, causing the processor to: at least one bit in said bit sequence for said volume, determining that a context reduction condition is satisfied and, based thereon, selecting a reduced set of contexts that contains contexts that are smaller than a product of a count of neighbor configurations and a number of previously coded bits in the sequence; selecting a context for encoding the at least one bit from the reduced set of contexts based on occupancy states of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence; as well as The at least one bit is entropy encoded based on the selected context using a binary entropy encoder to produce encoded data for the bitstream.

22. A decoder for decoding a bit stream of compressed occupancy data of a point cloud to produce a reconstructed point cloud, wherein occupancy of sub-volumes of a volume is indicated using a bit sequence, each bit in the bit sequence indicating the occupancy of a corresponding sub-volume in a scan order within the volume, and wherein the volume has a plurality of neighboring volumes, the occupancy pattern of the neighboring volumes being a neighbor configuration, the decoder comprising: processor; Memory; as well as a decoding application comprising instructions executable by the processor, the instructions, when executed, causing the processor to: at least one bit in said bit sequence for said volume, determining that a context reduction condition is satisfied and, based thereon, selecting a reduced set of contexts that contains contexts that are smaller than a product of a count of neighbor configurations and a number of previously coded bits in the sequence; selecting a context for encoding the at least one bit from the reduced set of contexts based on occupancy states of at least some of the neighboring volumes and at least one previously encoded bit of the bit sequence; as well as At least one bit is entropy decoded based on the selected context using a binary entropy decoder to produce reconstructed bits from the bitstream.

Images