Method and device for binary-entropy coding point cloud

Abstract

To provide a method and device for binary-entropy coding a point cloud.SOLUTION: A method, for an encoder, includes: determining a pattern of occupation concerning a present node as an occupant node divided into eight child nodes each corresponding to respective sub-cubes (202); selecting a certain probability distribution based on certain occupation information from at least one node neighboring the present node from among a set of probability distributions (204); entropy encoding the pattern of occupation concerning the present node by using a selected probability distribution (206); updating the selected probability distribution based on the pattern of occupation and evaluating whether or not there is a further node for which coding should be done (208); and returning to 202, if it is present, in order to making coding of a next code.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] This application relates generally to point cloud compression, and more particularly to a method and device for binary entropy coding of point clouds. [Background technology]

[0002] Data compression is used in communications and computer networking to efficiently store, transmit, and reproduce information. There is growing interest in representing three-dimensional objects or spaces, which can involve large data sets, for which efficient and effective compression would be highly useful and beneficial. In some cases, a three-dimensional object or space can be represented using a point cloud, which is a set of points each having three coordinate locations (X, Y, Z) (and possibly other attributes such as color data (e.g., luminance and chrominance), transparency, reflectance, normal vectors, etc.). A point cloud can be static (a stationary object or a snapshot of an environment / object at a single point in time) or dynamic (a time-ordered sequence of point clouds).

[0003] Exemplary applications for point clouds include topography and mapping applications. Autonomous vehicles and other machine vision applications may rely on point cloud sensor data in the form of 3D scans of an environment, such as from LiDAR scanning devices. Virtual reality simulations may rely on point clouds.

[0004] It should be appreciated that point clouds may involve large amounts of data, and compressing (encoding and decoding) that data quickly and accurately is of great interest. Therefore, it would be advantageous to provide methods and devices for more efficiently and / or effectively compressing data related to point clouds. Furthermore, it would be advantageous to find methods and devices for coding point clouds that can be implemented using context-adaptive binary entropy coding without requiring management of an excessive number of contexts. Summary of the Invention [Means for solving the problem]

[0005] This application describes methods for encoding and decoding point clouds, and encoders and decoders for encoding and decoding point clouds. Bit sequences indicating occupancy patterns for subvolumes of a volume may be coded using entropy coding (e.g., binary entropy coding). Probabilities associated with each entropy coder for use in entropy coding the occupancy patterns may be selected based on occupancy data for neighboring subvolumes of the current subvolume, and further based on occupancy data for at least one child subvolume of the neighboring subvolumes.

[0006] In an example useful for understanding the present application, the context may be based on a neighborhood configuration and a partial sequence of previously coded bits of the bit sequence. A decision may be made as to whether to apply a context reduction operation, and if so, the operation reduces the number of available contexts. Exemplary context reduction operations include reducing neighborhood configurations based on shielding by sub-volumes associated with previously coded bits, special handling for empty neighborhood configurations, and statistically-based context merging. The reduction may be applied before coding, and a decision may be made during coding as to whether the conditions for using a reduced context set are met.

[0007] In one aspect, the present application provides a method for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure, the tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing geometric shapes in a volume space recursively divided into subvolumes containing points of the point cloud. The method includes determining an occupancy pattern for a current node associated with a subvolume divided into further subvolumes, each subvolume corresponding to a child node of the current node, based on the occupancy status of the child node. The method further includes selecting one or more probabilities associated with respective entropy coders for entropy encoding the occupancy pattern, the selecting being based on occupancy data for a plurality of neighboring nodes of the current node and occupancy data for at least one child node of the plurality of neighboring nodes. The method further includes entropy encoding the occupancy pattern using the associated one or more entropy coders based on the selected one or more probabilities to generate encoded data for a bitstream.

[0008] In another aspect, the present application provides a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure, the tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing geometric shapes in a volume space recursively divided into sub-volumes containing points of the point cloud. The method includes selecting, for a current node associated with a sub-volume divided into further sub-volumes, each further sub-volume corresponding to a child node of the current node, one or more probabilities associated with respective entropy coders for entropy decoding an occupancy pattern, the selecting being based on occupancy data for a plurality of neighboring nodes of the current node and occupancy data of at least one child node of the plurality of neighboring nodes. The method further includes entropy decoding the bitstream using the one or more associated entropy coders based on the selected one or more probabilities to generate a reconstructed occupancy pattern for the current node, the reconstructed occupancy pattern indicating the occupancy of the child node.

[0009] In some implementations, selecting one or more probabilities may be based on a neighborhood configuration, which is determined based on the occupancy status of each of the neighboring nodes of the current node.

[0010] In some implementations, a neighboring node of a current node may be considered occupied for purposes of determining neighborhood configuration if its occupancy data indicates that it is occupied and if the occupancy data for its child nodes indicates that at least one of its occupied child nodes is located in the neighborhood of the current node.

[0011] In some implementations, a neighboring node of a current node may be considered unoccupied for purposes of determining neighborhood configuration if its occupancy data indicates that it is occupied and the occupancy data for its child nodes indicates that none of its occupied child nodes are located in the neighborhood of the current node, which may correspond to (intentionally / artificially) setting the occupied bit for that neighboring node to zero in determining neighborhood configuration.

[0012] In some implementations, a neighboring node of a current node may be considered occupied for purposes of determining neighborhood configuration if its occupancy data indicates that it is occupied and if it has not yet been coded. When a neighboring node has not yet been coded, the decoder does not yet have information about the occupancy of its child nodes, and therefore such information cannot be used in considering whether a neighboring node is occupied for purposes of determining neighborhood configuration.

[0013] In some implementations, a neighboring node of a current node may be considered unoccupied for purposes of determining neighborhood configuration if its occupancy data indicates that it is unoccupied.

[0014] In some implementations, neighboring nodes of the current node may be those nodes that are at the same depth in the tree structure as the current node and whose associated sub-volumes intersect with the sub-volume of the current node.

[0015] In some implementations, child nodes located nearby the current node may be those nodes that are one level deeper in the tree structure than the current node and whose associated subvolumes intersect with the subvolume of the current node.

[0016] In some implementations, the occupancy data for the plurality of neighboring nodes may include an occupancy status for each of the plurality of neighboring nodes.

[0017] In some implementations, the tree structure may represent an octree.

[0018] In some implementations, the encoding method may further include encoding a flag, the flag indicating that one or more probabilities associated with each entropy coder for entropy encoding the occupancy pattern are selected based on occupancy data for a plurality of neighboring nodes of the current node and occupancy data of at least one child node of the plurality of neighboring nodes.

[0019] In some implementations, the decoding method may further include decoding a flag, the flag indicating that one or more probabilities associated with each entropy coder for entropy decoding the occupancy pattern should be selected based on occupancy data for a plurality of neighboring nodes of the current node and occupancy data of at least one child node of the plurality of neighboring nodes.

[0020] In another aspect, this application provides a method for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure, the tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of a volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes, and a pattern of occupancy of the neighboring volumes being a neighborhood configuration. The method includes determining that a context reduction condition is met for at least one bit in a bit sequence of a volume, and based thereon, selecting a reduced context set including fewer contexts than the product of a count of neighboring configurations and the number of previously coded bits in the sequence; selecting a context from the reduced context set for coding the at least one bit based on the occupancy status of at least some of the neighboring volumes and at least one previously coded bit of the bit sequence; entropy encoding the at least one bit based on the selected context using a binary entropy encoder to generate encoded data for the bitstream; and updating the selected context.

[0021] In another aspect, this application provides a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure, the tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of a volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes, and a pattern of occupancy of the neighboring volumes being a neighborhood configuration. The decoding method includes determining that a context reduction condition is met for at least one bit in a bit sequence of a volume, and based thereon, selecting a reduced context set including fewer contexts than the product of the count of neighboring configurations and the number of previously coded bits in the sequence; selecting a context from the reduced context set for coding the at least one bit based on the occupancy status of at least some of the neighboring volumes and at least one previously coded bit of the bit sequence; entropy decoding the at least one bit based on the selected context using a binary entropy decoder to generate a reconstructed bit from the bitstream; and updating the selected context.

[0022] In some implementations, the context reduction condition may include determining that one or more previously coded occupied bits are associated with one or more respective sub-volumes that are located between the sub-volume associated with at least one bit and one or more of the neighboring volumes. In some cases, this may include determining that four sub-volumes associated with the previously encoded bits share a surface with a particular neighboring volume.

[0023] In some implementations, the context reduction condition may include determining that at least four bits of the bit sequence have been previously coded.

[0024] In some implementations, determining that a context reduction condition is met may include determining that a pattern of occupancy of neighboring volumes indicates that multiple neighboring volumes are unoccupied. In some of those cases, the selected reduced context set may include a number of contexts corresponding to a number of previously coded bits in the bit sequence, and optionally, selecting a context may include selecting a context based on a sum of previously coded bits in the bit sequence.

[0025] In some implementations, the context reduction condition may include determining that at least a certain threshold number of bits in the bit sequence have been previously coded, and the reduced context set may include a lookup table that maps each possible combination of neighborhood and previously coded bit patterns in the bit sequence to fewer contexts. In some examples, the lookup table may be generated based on iterative grouping of available contexts into multiple classes based on determining that a distance measure between each pair of available contexts is less than a certain threshold, and each class in the multiple classes may include a respective context in a smaller set, and there may be an available context for each possible combination of neighborhood and previously coded bit patterns in the bit sequence.

[0026] In some implementations, at least some of the neighboring volumes are neighboring volumes that share at least one face with the volume.

[0027] In a further aspect, the present application describes encoders and decoders configured to implement such methods of encoding and decoding.

[0028] In yet a further aspect, the present application describes a non-transitory computer-readable medium storing computer-executable program instructions that, when executed, cause one or more processors to perform the described methods of encoding and / or decoding.

[0029] In yet another aspect, this application describes computer-readable signals containing program instructions that, when executed by a computer, cause the computer to perform the described methods of encoding and / or decoding.

[0030] This application further describes computer-implemented applications, including topography applications, mapping applications, automotive industry applications, autonomous driving applications, virtual reality applications, cultural heritage applications, etc. These computer-implemented applications include processes for receiving a data stream or data file, decompressing the data stream or data file to obtain a bitstream of compressed point cloud data, and decoding the bitstream as described in the above aspects and implementations thereof, thereby utilizing point cloud compression techniques in accordance with aspects and implementations thereof described throughout this application.

[0031] The present application further describes methods for encoding and decoding point clouds, and encoders and decoders for encoding and decoding point clouds. In some implementations, a receiving unit receives multiplexed data, where the multiplexed data is obtained by multiplexing the coded point cloud data with other coded data types, such as metadata, images, video, audio, and / or graphics. The receiving unit includes a demultiplexing unit for separating the multiplexed data into the coded point data and the other coded data, and at least one decoding unit (or decoder) for decoding the coded point cloud data. In some other implementations, an emitting unit emits the multiplexed data, where the multiplexed data is obtained by multiplexing the coded point cloud data with other coded data types, such as metadata, images, video, audio, and / or graphics. The emitting unit includes at least one encoding unit (or encoder) for encoding the point cloud data and a multiplexing unit for combining the coded point cloud data and the other coded data into the multiplexed data. The present invention provides, for example, the following. (Item 1) 1. A method of encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing the geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud; The method comprises: With respect to a current node associated with a sub-volume, the sub-volume being divided into further sub-volumes, each further sub-volume corresponding to a child node of the current node: determining an occupancy pattern for the current node based on the occupancy status of the child nodes; selecting one or more probabilities associated with each entropy coder for entropy encoding the occupancy pattern, the selecting being based on occupancy data for a plurality of neighboring nodes of the current node and occupancy data for at least one child node of the plurality of neighboring nodes; entropy encoding the occupancy pattern based on the selected one or more probabilities using the associated one or more entropy coders to generate encoded data for the bitstream; A method comprising: (Item 2) 1. A method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud; The method comprises: With respect to a current node associated with a sub-volume, the sub-volume being divided into further sub-volumes, each further sub-volume corresponding to a child node of the current node: selecting one or more probabilities associated with each entropy coder for entropy decoding the occupancy pattern, the selecting being based on occupancy data for a plurality of neighboring nodes of the current node and occupancy data of at least one child node of the plurality of neighboring nodes; entropy decoding the bitstream based on the selected one or more probabilities using the one or more associated entropy coders to generate a reconstructed occupancy pattern for the current node; Including, The reconstructed occupancy pattern indicates the occupancy of the child nodes. (Item 3) 3. The method of claim 1, wherein selecting the one or more probabilities is based on a neighborhood configuration determined based on an occupancy status of each of the neighboring nodes of the current node. (Item 4) Item 4. The method of item 3, wherein a neighboring node of the current node is considered to be occupied for purposes of determining the neighborhood configuration if occupancy data for the neighboring node indicates that the neighboring node is occupied and the occupancy data for the child nodes of the neighboring node indicates that at least one of the occupied child nodes of the neighboring node is located in the neighborhood of the current node. (Item 5) 5. The method of claim 3 or 4, wherein a neighboring node of the current node is considered unoccupied for purposes of determining the neighborhood configuration if occupancy data for the neighboring node indicates that the neighboring node is occupied and the occupancy data for the neighboring node's child nodes indicates that none of the neighboring node's occupied child nodes are located in the neighborhood of the current node. (Item 6) 6. The method of any one of items 3-5, wherein a neighboring node of the current node is considered occupied for purposes of determining the neighborhood configuration if the neighboring node has not yet been coded. (Item 7) 7. The method of any one of items 1-6, wherein the neighboring nodes of the current node are those nodes that are at the same depth in the tree structure as the current node and whose associated sub-volumes intersect with the sub-volume of the current node. (Item 8) The method of any one of items 5-7 when dependent on item 4 or item 4, wherein child nodes located in the vicinity of the current node are those nodes that are one level deeper in the tree structure than the current node and whose associated sub-volumes intersect with the sub-volume of the current node. (Item 9) Item 9. The method of any one of items 1-8, wherein the occupancy data for the plurality of neighboring nodes comprises an occupancy status for each of the plurality of neighboring nodes. (Item 10) 10. The method according to any one of items 1-9, wherein the tree structure represents an octree. (Item 11) Item 2 or the method of any one of items 3-10 when dependent on item 2, further comprising decoding a flag from the bitstream, the flag indicating that the one or more probabilities associated with each entropy coder for entropy decoding the occupancy pattern should be selected based on the occupancy data for the plurality of neighboring nodes of the current node and the occupancy data of at least one child node of the plurality of neighboring nodes. (Item 12) 1. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud; The encoder comprises: a processor; Memory and an encoding application including instructions executable by said processor; Equipped with An encoder, wherein the instructions, when executed, cause the processor to perform the method of item 1 or any one of items 3-10 when dependent on item 1. (Item 13) 1. A decoder for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud; The decoder a processor; Memory and a decoding application including instructions executable by said processor; Equipped with The instructions, when executed, cause the processor to perform the method of item 2 or any one of items 3-11 when dependent on item 2. (Item 14) 12. A non-transitory processor-readable medium storing processor-executable instructions that, when executed by a processor, cause the processor to perform the method of any one of items 1-11. (Item 15) A computer-readable signal comprising program instructions that, when executed by a computer, cause the computer to perform the method described in any one of items 1-11.

[0032] Reference will now be made, by way of example, to the accompanying drawings which illustrate exemplary embodiments of the present application. [Brief explanation of the drawings]

[0033] [Figure 1] FIG. 1 shows a simplified block diagram of an exemplary point cloud encoder.

[0034] [Figure 2] FIG. 2 shows a simplified block diagram of an exemplary point cloud decoder.

[0035] [Figure 3] FIG. 3 shows an exemplary partial sub-volume and associated tree structure for coding.

[0036] [Figure 4] FIG. 4 illustrates the recursive partitioning and coding of an octree.

[0037] [Figure 5]FIG. 5 shows an example scanning pattern within an example cube from an octree.

[0038] [Figure 6] FIG. 6 shows an example occupation pattern within an example cube.

[0039] [Figure 7] FIG. 7 illustrates in flow chart form an exemplary method for encoding a point cloud.

[0040] [Figure 8] FIG. 8 illustrates a portion of an exemplary octree.

[0041] [Figure 9] FIG. 9 shows an example of a neighborhood sub-volume.

[0042] [Figure 10] FIG. 10 illustrates an exemplary neighborhood configuration showing occupancy among neighboring nodes.

[0043] [Figure 11] FIG. 11 graphically illustrates one illustrative embodiment of the process of point cloud entropy encoding using parent pattern dependent context.

[0044] [Figure 12] FIG. 12 shows an illustrative embodiment of a process for point cloud entropy encoding using neighborhood-dependent context.

[0045] [Figure 13] FIG. 13 illustrates in flow chart form an exemplary method for decoding a bitstream of compressed point cloud data.

[0046] [Figure 14] FIG. 14 shows an exemplary simplified block diagram of an encoder.

[0047] [Figure 15] FIG. 15 shows an exemplary simplified block diagram of a decoder.

[0048] [Figure 16] FIG. 16 illustrates an example Cartesian coordinate system and example rotations and / or reflections about axes.

[0049] [Figure 17] FIG. 17 shows the class of invariance of neighborhood configurations under one or several iterations of rotation around the Z axis.

[0050] [Figure 18] FIG. 18 shows the class of neighborhood configuration invariance with respect to vertical reflections.

[0051] [Figure 19] FIG. 19 shows the classes that are invariant with respect to both rotation and reflection.

[0052] [Figure 20] Figure 20 shows the classes of invariance under three rotations and reflections.

[0053] [Figure 21] FIG. 21 illustrates the equivalence between non-binary coding and cascaded binary coding with respect to occupancy patterns.

[0054] [Figure 22] FIG. 22 illustrates in flowchart form an exemplary method for coding occupancy patterns in a tree-based point cloud coder using binary coding.

[0055] [Figure 23] FIG. 23 shows a simplified block diagram of a portion of an exemplary encoder.

[0056] [Figure 24] FIG. 24 illustrates diagrammatically an exemplary context reduction operation based on neighborhood screening.

[0057] [Figure 25] FIG. 25 illustrates another exemplary context reduction operation based on neighborhood screening.

[0058] [Figure 26] FIG. 26 illustrates in flow chart form an example of a method for binary coding occupancy patterns using combined context reduction.

[0059] [Figure 27] FIG. 27 shows an example of neighboring subvolumes, some of which have already been coded.

[0060] [Figure 28] FIG. 28 shows an example of neighboring sub-volumes and their already coded sub-volumes.

[0061] [Figure 29] FIG. 29 shows an example of subvolume occupancy within neighboring subvolumes.

[0062] [Figure 30] FIG. 30 illustrates in flow chart form a method for encoding an occupancy pattern of a current node based at least in part on occupancy data for at least one child node of a plurality of neighboring nodes.

[0063] [Figure 31] FIG. 31 illustrates in flow chart form a method for decoding an occupancy pattern of a current node based at least in part on occupancy data for at least one child node of a plurality of neighboring nodes.

[0064] [Figure 32] FIG. 32 illustrates in flow chart form a method for determining neighborhood configurations as a function of sub-volumes of neighboring sub-volumes.

[0065] [Figure 33] FIG. 33 shows another example of subvolume occupancy within neighboring subvolumes.

[0066] [Figure 34] FIG. 34 shows another example of subvolume occupancy within neighboring subvolumes. DETAILED DESCRIPTION OF THE INVENTION

[0067] Similar reference numbers may be used in different figures to represent similar components.

[0068] Other aspects and features of the present application will be understood by those skilled in the art from a review of the following description of examples in conjunction with the accompanying figures.

[0069] Any feature described in connection with one aspect or embodiment of the invention may also be used in connection with one or more other aspects / embodiments. These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described herein.

[0070] At times in the following description, the terms "node," "volume," and "subvolume" may be used interchangeably. It should be understood that a node is associated with a volume or subvolume. A node is a particular point on a tree, which may be an internal node or a leaf node. A volume or subvolume is a bounded physical space that a node represents. The term "volume" may be used in some cases to refer to the largest bounded space defined to contain a point cloud. A volume may be recursively subdivided into subvolumes for the purpose of constructing a tree structure of interconnected nodes for coding point cloud data.

[0071] As used herein, the term "and / or" is intended to include any one of the listed elements alone, any subcombination, or all of the elements, and to cover all possible combinations and subcombinations of the listed elements without necessarily excluding additional elements.

[0072] As used herein, the phrase "or at least one of" is intended to cover any one or more of the listed elements, including any one of the listed elements alone, any subcombination, or all of the elements, without necessarily excluding any additional elements, and without necessarily requiring all of the elements.

[0073] A point cloud is a set of points in a three-dimensional coordinate system. The points are often intended to represent the exterior surfaces of one or more objects. Each point has a location (position) in the three-dimensional coordinate system. The position may be represented by three coordinates (X, Y, Z), which may be Cartesian or any other coordinate system. Points may have other associated attributes, such as color, which in some cases may also be three-component values ​​such as R, G, B or Y, Cb, Cr. Other associated attributes may include transparency, reflectance, normal vectors, etc., depending on the desired use for the point cloud data.

[0074] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or topography can be static point cloud data. LiDAR-based scanning of an environment for machine vision purposes can be dynamic in that the point cloud (at least potentially) changes over time, e.g., with each successive scan of a volume. A dynamic point cloud is thus a time-ordered sequence of point clouds.

[0075] Point cloud data can be used in several applications, including preservation (scanning historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous automobiles), and virtual reality systems, to name a few. Dynamic point cloud data for applications such as machine vision can be quite different from static point cloud data such as that for preservation purposes. For example, automotive vision typically involves relatively low-resolution, colorless, and highly dynamic point clouds acquired through LiDAR (or similar) sensors with high acquisition frequencies. The purpose of such point clouds is not for human consumption or viewing, but rather for machine object detection / classification in decision processes. As an example, a typical LiDAR frame contains about several thousand points, while high-quality virtual reality applications require millions of points. It can be expected that there will be a demand for higher resolution data over time as computational speeds improve and new applications are discovered.

[0076] Although point cloud data is useful, the lack of effective and efficient compression, i.e., encoding and decoding processes, can hinder adoption and deployment. A particular challenge in coding point clouds that does not arise with other data compression, such as audio or video, is coding the geometry of the point cloud. Point clouds tend to be populated sparsely, which makes efficiently coding the location of the points much more difficult.

[0077] One of the more common mechanisms for coding point cloud data is through the use of a tree-based structure. In a tree-based structure, a bounded three-dimensional volume for the point cloud is recursively divided into subvolumes. Nodes of the tree correspond to the subvolumes. The decision to further divide a subvolume may be based on the resolution of the tree and / or whether there are any points contained within the subvolume. A leaf node may have an occupancy flag that indicates whether its associated subvolume contains points. A split flag may indicate whether the node has child nodes (i.e., whether the current volume has been further divided into subvolumes). These flags may be entropy coded in some cases, and predictive coding may be used in some cases.

[0078] A commonly used tree structure is the octree. In this structure, all volumes / subvolumes are cubes, and each division of a subvolume results in eight additional subvolumes / subcubes. Another commonly used tree structure is the KD tree, in which a volume (a cube or a rectangular cuboid) is recursively divided into two by a plane orthogonal to one of the axes. The octree is a special case of the KD tree, in which the volume is divided by three planes, each orthogonal to one of the three axes. Both of these examples relate to a cube or a rectangular cuboid; however, the present application is not limited to such tree structures, and volumes and subvolumes may have other shapes in some applications. The partitioning of a volume is not necessarily into two subvolumes (KD tree) or eight subvolumes (octree), but may involve other partitioning, including division into non-rectangular shapes, or non-adjacent subvolumes.

[0079] Although this application may refer to octrees for ease of explanation and because they are popular candidate tree structures for automotive applications, it should be understood that the methods and devices described herein may be implemented using other tree structures.

[0080] Reference is now made to Figure 1, which illustrates a simplified block diagram of a point cloud encoder 10, in accordance with an aspect of the present application. The point cloud encoder 10 includes a tree construction module 12 that receives point cloud data and generates a tree (in this example, an octree) that represents a geometry in volume space that contains the point cloud and indicates the locations or positions of points from the point cloud within that geometry.

[0081] The basic process for creating an octree for coding a point cloud may include: 1. Start with a bounded volume (a cube) containing the points in a coordinate system. 2. Divide the volume into 8 subvolumes (8 subcubes). 3. For each subvolume, mark the subvolume with 0 if it is empty, and with 1 if there is at least one point in it. For all subvolumes marked in 4.1, repeat (2) to split those subvolumes until the maximum depth of splitting is reached. 5. For every leaf subvolume (subcube) at maximum depth, if it is not empty, mark the leaf cube with 1, otherwise mark it with 0.

[0082] The above process can be described as an occupancy = split process, where splitting implies occupancy and there is a constraint that there is a maximum depth or resolution beyond which no further splitting will occur. In this case, a single flag indicates whether a node is split, and therefore whether it is occupied by at least one point, and vice versa. At the maximum depth, the flag indicates occupancy and no further splitting is possible.

[0083] In some implementations, partitioning and occupancy are independent, whereby a node can be occupied and a node can be partitioned or not. Two variations of this implementation exist: 1. Occupancy after split. A signal flag indicates whether a node is split. If it is split, the node must contain a point, i.e., splitting implies occupancy. Otherwise, if the node should not be split, a further occupancy flag indicates whether the node contains at least one point. Thus, when a node is not further split, i.e., it is a leaf node, the leaf node must have an associated occupancy flag to indicate whether it contains any points. 2. Occupy then split. A single flag indicates whether the node is occupied. If it is not occupied, no splitting occurs. If it is occupied, a split flag is coded to indicate whether the node will be further split.

[0084] Regardless of the above-described process used to construct the tree, it may be traversed in a predetermined order (breadth-first or depth-first, and according to a scanning pattern / order within each divided subvolume) to generate a sequence of bits from the flags (occupancy and / or split flags). This may be referred to as serializing or binarizing the tree. As shown in Figure 1, in this example, the point cloud encoder 10 includes a binarizer 14 for binarizing the octree and generating a bitstream of binarized data representing the tree.

[0085] This sequence of bits may then be encoded using an entropy encoder 16 to generate a compressed bitstream. The entropy encoder 16 may encode the sequence of bits using a context model 18, which defines probabilities for coding bits based on context decisions by the entropy encoder 16. The context model 18 may be adaptively updated after coding each bit or a defined set of bits. The entropy encoder 16 may, in some cases, be a binary arithmetic encoder. The binary arithmetic encoder may, in some implementations, employ context-adaptive binary arithmetic coding (CABAC). In some implementations, coders other than arithmetic coders may also be used.

[0086] In some cases, the entropy encoder 16 may not be a binary coder and may instead operate on non-binary data. The output octree data from the tree construction module 12 may not be evaluated in binary form and may instead be encoded as non-binary data. For example, in the case of an octree, the eight flags (e.g., occupancy flags) within the subvolumes in their traversal order may be coded as follows: 8 It may be considered a 1-bit number (e.g., an integer having a value between 1 and 255, since the value 0 is not possible for a partitioned subvolume (i.e., it is not partitioned if it is not occupied at all). This number may, in some implementations, be encoded by an entropy encoder using a multi-symbol arithmetic coder. The sequence of flags that defines this integer within a subvolume (e.g., a cube) may be called a "pattern."

[0087] As in video or image coding, point cloud coding can involve prediction operations, in which an attempt is made to predict patterns for sub-volumes. Prediction can be spatial (relying on previously coded sub-volumes within the same point cloud) or temporal (relying on previously coded points in a time-ordered sequence of point clouds).

[0088] A block diagram of an exemplary point cloud decoder 50 corresponding to encoder 10 is shown in FIG. 2. Point cloud decoder 50 includes an entropy decoder 52 that uses the same context model 54 used by encoder 10. Entropy decoder 52 receives an input bitstream of compressed data and entropy decodes the data, producing an output sequence of decompressed bits. The sequence is then converted into reconstructed point cloud data by tree reconstructor 56. Tree reconstructor 56 reconstructs a tree structure from the decompressed data and knowledge of the scan order in which the tree data was binarized. Tree reconstructor 56 is therefore able to reconstruct point locations from the point cloud (given the resolution of the tree coding).

[0089] An exemplary partial subvolume 100 is shown in FIG. 3. In this example, the subvolume 100 is shown in two dimensions for ease of illustration, and the size of the subvolume 100 is 16×16. Note that the subvolume is divided into four 8×8 subsquares, two of which are further subdivided into 4×4 subsquares, three of which are further subdivided into 2×2 subsquares, and one of the 2×2 subsquares is then divided into a 1×1 square. The 1×1 square is the maximum depth of the tree and represents the finest resolution for the positional point data. Points from the point cloud are shown as dots in the figure.

[0090] The structure of the tree 102 is shown to the right of the subvolume 100. The sequence of split flags 104 and the corresponding sequence of occupancy flags 106 obtained in a given breadth-first traversal order are shown to the right of the tree 102. It will be observed that in this illustrative example, there is an occupancy flag for each subvolume (node) that is not split (i.e., has an associated split flag set to zero). These sequences may be entropy encoded.

[0091] Another example employing the occupancy = split condition is shown in FIG. 4. FIG. 4 illustrates the recursive splitting and coding of an octree 150. Only a portion of the octree 150 is shown in the figure. A FIFO 152 is shown processing a node for splitting to illustrate the breadth-first nature of this process. The FIFO 152 outputs an occupied node 154 that has been queued in the FIFO 152 for further splitting after processing its parent node 156. The tree builder splits the subvolume associated with the occupied node 154 into eight subvolumes (cubes) and determines their occupancy. The occupancy may be indicated by an occupancy flag for each subvolume. In a prescribed traversal order, the flags may be referred to as an occupancy pattern for the node 154. The pattern may be defined by an integer representing the sequence of occupancy flags associated with the subvolumes in the prescribed traversal order. For an octree, the pattern is an integer in the range [1, 255].

[0092] The entropy encoder then encodes the patterns using a non-binary arithmetic encoder based on the probabilities specified by the context model. In this example, the probabilities are pattern distributions based on an initial distribution model and can be adaptively updated. In one implementation, the pattern distribution is effectively a counter of the number of times each pattern (an integer between 1 and 255) is encountered during coding. The pattern distribution can be updated after each subvolume is coded. The pattern distribution can be normalized, if necessary, since the relative frequency of patterns is closely related to the probability assessment, not the absolute count.

[0093] Based on the pattern, those child nodes that are occupied (e.g., have flag=1) are then pushed into FIFO 152 for further splitting in order (provided the node is not at the maximum depth of the tree).

[0094] Reference is now made to Figure 5, which shows an example cube 180 from an octree. The cube 180 is subdivided into eight subcubes. The scan order for reading the flags results in an 8-bit string, which can be read as an integer [1,255] in binary. Based on the scan order and the resulting bit position of each subcube's flag within the string, the subcube has the value shown in Figure 5. The scan order can be any sequence of subcubes, provided that both the encoder and decoder use the same scan order.

[0095] As an example, Figure 6 shows cube 180 with four "front" subcubes occupied. This would correspond to pattern 85, based on the occupied subcubes being cubes 1 + 4 + 16 + 64. The integer pattern number defines the pattern of occupation within the subcubes.

[0096] Octree representations, or more generally, any tree representation, are efficient at representing spatially correlated points because trees tend to factor out the higher-order bits of point coordinates. With an octree, each level of depth refines the coordinates of a point within a subvolume by one bit for each component, at the expense of eight bits per refinement. Further compression is obtained by entropy coding the partitioning information (i.e., pattern) associated with each tree node. This further compression is possible because pattern distributions are not uniform; i.e., non-uniformity is another consequence of correlation.

[0097] One potential inefficiency in current systems is that a pattern distribution (e.g., a histogram of pattern numbers found in previously coded nodes of the tree) is developed over the course of coding the point cloud. In some cases, the pattern distribution can be initialized as equiprobable or to some other predetermined distribution, but the use of one pattern distribution means that the context model does not consider or exploit local geometric correlations.

[0098] In European Patent Application No. 18305037.6, the applicant described a method and device for selecting from available pattern distributions to be used in coding the pattern of occupancy of a particular node based on certain occupancy information from previously coded nodes in the vicinity of the particular node. In one exemplary implementation, the occupancy information is obtained from the pattern of occupancy of parents to the particular node. In another exemplary implementation, the occupancy information is obtained from one or more nodes located in the vicinity of the particular node. The contents of European Patent Application No. 18305037.6 are incorporated herein by reference.

[0099] Reference is now made to Figure 7, which illustrates in flowchart form an exemplary method 200 for encoding a point cloud. The method 200 in this example involves recursive partitioning of occupied nodes (subvolumes) and breadth-first traversal of the tree for coding.

[0100] In operation 202, the encoder determines a pattern of occupancy for the current node. The current node is an occupied node divided into eight child nodes, each corresponding to a respective subcube. The pattern of occupancy for the current node defines the occupancy of the eight child nodes in the traversal order. As explained above, this pattern of occupancy may be represented using integers between 1 and 255, e.g., an 8-bit binary string.

[0101] In operation 204, the encoder selects a probability distribution from a set of probability distributions. The selection of a probability distribution is based on occupancy information from nearby, previously coded nodes, i.e., at least one node that is a neighbor of the current node. In some embodiments, two nodes are neighbors if they are associated with respective subvolumes that share at least one face. In a broader definition, nodes are neighbors if they share at least one edge. In an even broader definition, two nodes are neighbors if they share at least one vertex. A parent pattern (in which the current node is a child node) provides occupancy data for the current node and its seven sibling nodes. In some implementations, the occupancy information is a parent pattern. In some implementations, the occupancy information is occupancy data for a set of neighboring nodes, which include nodes that have different parent nodes but are at the same tree depth level as the current node. In some cases, combinations of these are possible. For example, the set of neighboring nodes may include some sibling nodes and some non-sibling nodes.

[0102] Once a probability distribution is selected, the encoder entropy encodes the occupancy pattern for the current node using the selected probability distribution, as indicated by operation 206. It then updates the selected probability distribution based on the occupancy pattern (e.g., it may increment a count corresponding to the occupancy pattern) in operation 208. In operation 210, the encoder evaluates whether there are more nodes to code, and if so, returns to operation 202 to code the next node.

[0103] The probability distribution selection in operation 204 should be based on occupancy data for nearby, previously coded nodes. This allows both the encoder and decoder to make the same selection independently. For the following discussion of probability distribution selection, reference will be made to FIG. 8, which diagrammatically illustrates a partial octree 300 including a current node 302. The current node 302 is an occupied node and is being evaluated for coding. The current node 302 is one of eight children of a parent node 306, which is in turn a child of a grandparent node (not shown). The current node 302 is differentiated into eight child nodes 304. The occupancy pattern for the current node 302 is based on the occupancy of the child nodes 304. For example, as shown, using the notation where black dots are occupied nodes, the occupancy pattern could be 00110010, i.e., pattern 50.

[0104] The current node 302 has a sibling node 308 that has the same parent node 306. The parent pattern is the occupancy pattern for the parent node 306, which, as shown, would be 00110000, i.e., pattern 48. The parent pattern can serve as the basis for selecting a suitable probability distribution for entropy encoding the occupancy pattern for the current node.

[0105] FIG. 9 illustrates the set of neighbors surrounding a current node, where neighbors are defined as nodes that share a face. In this example, the node / subvolume is a cube, and the cube at the center of the image has six neighbors, one on each face. It should be understood that in an octree, the neighborhood of the current node would include three sibling nodes. It would also include three nodes that do not have the same parent node. Thus, occupancy data for some of the neighboring nodes will be available because they are siblings, but occupancy data for some neighboring nodes may or may not be available depending on whether those nodes have been previously coded. Special handling may be applied to deal with missing neighbors. In some implementations, missing neighbors may be presumed occupied or unoccupied. It should be understood that the neighborhood definition may be expanded to include neighboring nodes based on shared edges or shared vertices to include additional adjacent subvolumes in the assessment.

[0106] It should be appreciated that the foregoing process examines the occupancy of neighboring nodes in an attempt to determine the likelihood of occupancy of the current node 302 to select a more favorable context and use more accurate probabilities for entropy coding the occupancy data of the current node 302. It should be appreciated that the occupancy status of neighboring nodes that share faces with the current node 302 may be a more accurate assessment of whether the current node 302 is likely isolated than an assessment based on the occupancy status of sibling nodes, three of which will share only edges and one of which will share only vertices (in the case of an octree). However, assessing sibling occupancy status has the advantage of being modular in that all relevant data for the assessment is part of the parent node, which means it has a smaller memory footprint for implementation, but assessing neighbor occupancy status involves buffering tree occupancy data if it is needed when determining neighbor occupancy status in connection with coding future neighboring nodes.

[0107] Neighbor occupancy may be read in a scan order that effectively assigns a value to each neighbor, similar to that described above with respect to occupancy patterns. As shown, neighboring nodes effectively take on values ​​of 1, 2, 4, 8, 16, or 32, and therefore there are 64 (0-63) possible neighbor occupancy configurations. This value may be referred to herein as a "neighbor configuration." As an example, FIG. 10 illustrates an example neighborhood configuration 15 in which neighbors 1, 2, 4, and 8 are occupied and neighbors 16 and 32 are empty.

[0108] In some cases, both of the above criteria (parent pattern and neighborhood configuration) may be applied, or a choice may be made between them. For example, if neighborhoods are available, the probability distribution selection may be based on neighboring nodes; however, if one or more of the neighbors are not available because they are from a node that has not yet been coded, the probability distribution selection may fall back to an analysis based on sibling nodes (parent patterns).

[0109] In yet another embodiment, the probability distribution selection may alternatively or additionally be based on a grandparent pattern. In other words, the probability distribution selection may be based on the occupancy status of uncle nodes that are siblings of the parent node 306.

[0110] In still further implementations, additional or alternative assessments may be incorporated into the probability distribution selection. For example, the probability distribution selection may examine the occupancy status of the parent node's neighbors or the grandparent node's neighbors.

[0111] Any two or more of the above criteria for assessing local occupancy status may be used in combination in some implementations.

[0112] For non-binary entropy coders, the occupancy data for the current node can be coded by selecting a probability distribution. The probability distribution contains a number of probabilities corresponding to the number of possible occupancy patterns for the current node. For example, when coding the occupancy patterns of an octree, 2 8 -1 = 255 possible patterns, meaning that each probability distribution contains 255 probabilities. In some embodiments, the number of probability distributions may be equal to the number of possible occupancy outcomes in the selection criteria (i.e., using neighbor, sibling, and / or parent occupancy data). For example, if the parent pattern for the octree is used as the selection criteria to determine the probability distribution to use, there will be 255 probability distributions, each with 255 probabilities. For neighborhood configurations, if neighborhoods are defined as sharing a face, there will be 64 probability distributions, each containing 255 probabilities.

[0113] It should be understood that too many distributions can result in slow adaptation due to a lack of data (i.e., context dilution). Thus, in some embodiments, similar patterns can be grouped to use the same probability distribution. For example, separate distributions can be used for patterns corresponding to fully occupied cases, vertically oriented cases, horizontally oriented cases, mostly empty cases, and all other cases. This can reduce the number of probability distributions to approximately five. It should be understood that different groupings of patterns can be formed to result in different numbers of probability distributions.

[0114] Reference is now made to Figure 11, which diagrammatically illustrates one exemplary embodiment of a process 400 for point cloud entropy encoding using parent pattern dependent context. In this example, a current node 402 is split into eight child nodes, whose occupancy patterns 404 are to be encoded using a non-binary entropy encoder 406. The non-binary entropy encoder 406 uses a probability distribution selected from one of six possible probability distributions 408. The selection is based on the parent pattern, i.e., the selection is based on occupancy information from parent nodes for the current node 402. The parent patterns are identified by integers between 1 and 255.

[0115] The selection of the probability distribution may be a decision tree that assesses whether the pattern corresponds to a full node (e.g., pattern = 255), a horizontal structure (e.g., pattern = 170 or 85, assuming the Z axis is vertical), a vertical structure (e.g., pattern = 3, 12, 48, 192), a sparsely populated distribution (e.g., pattern = 1, 2, 4, 8, 16, 32, 64, or 128, i.e., none of the sibling nodes are occupied), a semi-sparsely populated distribution (total number of occupied nodes between the current node and sibling nodes ≦ 3), and all other cases. The exemplary patterns shown for different categories are merely examples. For example, the "horizontal" category may include patterns with two or three occupied cubes on the same horizontal level. The "vertical" category may include patterns with three or four occupied cubes in a wall-like arrangement. It should be understood that finer gradations may also be used. For example, the "horizontal" category may be further subdivided horizontally in the top portion of the cube and horizontally in the bottom portion of the cube, with different probability distributions for each. Other groupings of occupancy patterns with certain correlations may also be made and assigned corresponding probability distributions. Further discussion of grouping patterns in the context of neighborhoods and invariance between neighborhoods is provided further below.

[0116] Figure 12 shows an illustrative embodiment of a process 500 for point cloud entropy encoding using neighborhood-dependent context. This example assumes the definitions of neighborhood and neighborhood numbering used above in connection with Figure 9. This example also assumes that each neighborhood has a dedicated probability distribution, meaning that there are 64 different probability distributions. The current node 502 has an occupancy pattern 504 to be encoded. The probability distribution is selected based on the neighbors of the current node 502. That is, the neighborhood NC in [0, 63] is found and used to select the associated probability distribution.

[0117] It should be understood that in some embodiments, neighborhoods may be grouped such that two or more neighborhoods use the same probability distribution based on similarities in patterns. In some embodiments, the process may use different arrangements of neighborhoods for contextualization (selection) of distributions. Additional neighborhoods may be added, such as eight diagonally adjacent neighbors on all three axes or twelve diagonally adjacent neighbors on two axes. Embodiments that avoid certain neighborhoods may also be used, for example, to avoid using neighborhoods that introduce additional dependencies in depth-first traversals, or to only introduce dependencies on certain axes to reduce codec states for large trees.

[0118] In this example, the case of NC=0 is handled in a specific manner. If there are no occupied neighbors, it may indicate that the current node 502 is isolated. Therefore, the process 500 further checks how many of the child nodes for the current node 502 are occupied. If only one child node is occupied (i.e., the occupation number (NO) is equal to 1), a flag indicating that a single child node is occupied is encoded, and the index for the node is coded using 3 bits. If two or more child nodes are occupied, the process 500 uses the NC=0 probability distribution to code the occupancy pattern.

[0119] Reference is now made to FIG. 13, which illustrates in flowchart form an exemplary method 600 for decoding a bitstream of encoded point cloud data.

[0120] In operation 602, the decoder selects one of the probability distributions based on occupancy information from one or more nodes in the neighborhood of the current node. As explained above, the occupancy information can be a parent pattern from a parent node to the current node, i.e., the occupancy of the current node and its siblings, or it can be the occupancy of neighboring nodes of the current node, which can include some of the sibling nodes. Other or additional occupancy information can also be used in some implementations.

[0121] Once a probability distribution is selected, in operation 604, the decoder entropy decodes a portion of the bitstream using the selected probability distribution to reconstruct an occupancy pattern for the current node. The occupancy pattern is used by the decoder in reconstructing the tree to reconstruct the encoded point cloud data. Once the point cloud data is decoded, it may be output from the decoder for use, such as for rendering a view, segmentation / classification, or other uses.

[0122] In operation 606, the decoder updates the probability distribution based on the reconstructed occupancy pattern, and then, if there are more nodes to decode, it moves to the next node in the buffer and returns to operation 602.

[0123] An exemplary implementation of the method described above has proven to provide compression improvements with a negligible increase in coding complexity. Neighborhood-based selection exhibits better compression performance than parent-pattern-based selection, but it has additional computational complexity and memory usage. In some tests, the relative improvement in bits per point over the MPEG point cloud test model is 4-20%. Note that initializing the probability distribution based on the distribution arrived at using test data leads to improved performance compared to initializing with a uniform distribution.

[0124] Some of the above examples are based on tree coding processes that use non-binary coders to represent occupancy patterns. New developments to employ binary entropy coders are presented further below.

[0125] In one variation of neighborhood-based probability distribution selection, the number of distributions can be reduced by exploiting neighborhood symmetry. By replacing neighborhoods or replacing pattern distributions, structurally similar configurations with symmetry lines can reuse the same distribution. In other words, neighborhood configurations that can use the same pattern distribution can be grouped into a class. A class containing two or more neighborhoods can be referred to herein as a "neighborhood configuration," in that one of the neighborhood configurations effectively subsumes the other neighborhood configurations with a reflection or replacement of the other configurations.

[0126] As an example, consider eight corner patterns NC ∈ [21, 22, 25, 26, 37, 38, 41, 42], each representing a corner neighborhood pattern symmetry. These values ​​of NC are node specific patterns, but are likely to be significantly correlated with different patterns. Furthermore, these correlated patterns are likely to follow the same symmetry as the neighborhood patterns. As an example, a method can be implemented that reuses a single distribution to represent multiple instances of NC by substituting the probabilities of that distribution.

[0127] An encoder derives a pattern number for a node based on the occupancy of its child nodes. The encoder selects a distribution and a replacement function according to the neighborhood configuration. The encoder reorders the probabilities contained in the distribution according to the replacement function and then uses the replaced distribution to arithmetically encode the pattern number. The arithmetic encoder's updates of the probabilities in the replaced distribution are mapped back to the original distribution using the inverse replacement function.

[0128] The corresponding decoder first selects the same distribution and substitution function according to the neighborhood configuration. A substitution distribution is generated in the same manner as the encoder, and the substitution distribution is used by the arithmetic decoder to entropy decode the pattern number. The bits comprising the pattern number are then each assigned to the corresponding child.

[0129] Note that the same permutation can be achieved without reordering the data in the distribution itself, but rather by introducing a level of indirection and using a permutation function to permutate the lookup of a given index within the distribution.

[0130] An alternative embodiment considers permutations of the patterns themselves rather than the distributions, allowing for shuffling prior to entropy encoding or after decoding. Such methods are likely more suitable for efficient implementation through a bitwise shuffle operation. In this case, no reordering of the distributions is performed by either the encoder or decoder; rather, the computation of the encoded pattern number is

number

number

number

[0131] The method of deriving the required replacement can be based on the rotational symmetry of the neighboring structures or on the reflection along a particular axis. Furthermore, it is not necessary that the replacement replaces all positions, e.g., according to the symmetry; partial replacement can be used instead. For example, when replacing NC=22 with NC=41, positions on the symmetry axis may not be replaced, and the mapping

number

[0132] An example of an embodiment based on rotational symmetry and reflection is provided below for the specific case of an octree with six neighbors that share a common face with the current cube. Without loss of generality, as shown in Figure 16, the Z axis runs perpendicular to the direction of viewing the figure. The relative positions of neighbors such as "above" ("below") should then be understood as along the Z axis in increasing (decreasing) Z directions. The same applies with respect to left / right sides along the X axis and front / back along the Y axis.

[0133] 16 shows three rotations 2102, 2104, and 2106 along the Z, Y, and X axes. The angles of these three rotations are 90 degrees, i.e., they perform a quarter turn rotation along their respective axes.

[0134] FIG. 17 shows classes of invariance of neighborhood configurations under one or several iterations of rotation 2102 along the Z axis. This invariance represents the same statistical behavior of the point cloud geometry along any direction belonging to the XY plane. This is particularly true for the use case of a car moving on the Earth's surface, which is locally approximated by the XY plane. A horizontal configuration is a given occupancy of four neighbors (located on the left, right, front, and back sides of the current cube) independent of the occupancy of the upper neighbors (2202) and the lower neighbors (2204). The four horizontal configurations 2206, 2208, 2210, and 2212 belong to the same class of invariance under rotation 2102. Similarly, two configurations 2214 and 2216 belong to the same class of invariance. There are only six classes of invariance under rotation 2102 (grouped under the set of classes 2218).

[0135] A vertical configuration is a given occupancy of two neighbors 2202 and 2204, independent of the occupancy of the four neighbors located on the left, right, front, and back sides of the current cube. There are four possible vertical configurations, as shown in Figure 18. Consequently, there are 6 x 4 = 24 possible configurations when considering invariance to rotation 2102 along the Z axis.

[0136] Reflection 2108 along the Z axis is shown in Figure 16. Vertical configurations 2302 and 2304 depicted in Figure 18 belong to the same class of invariance under reflection 2108. There are three classes of invariance under reflection 2108 (grouped under the set of classes 2306). Invariance under reflection 2108 means that the up and down directions behave essentially the same in terms of point cloud geometry statistics. This is an accurate assumption for a moving car on a road.

[0137] If we assume invariance under both rotation 2102 and reflection 2108, there are 18 classes of invariance resulting from the intersection of the two pairs 2218 and 2306. These 18 classes are represented in FIG.

[0138] Applying further invariance under two other rotations 2104 and 2106, the two configurations 2401 and 2402 belong to the same class of invariance. Furthermore, the two configurations 2411 and 2412, the two configurations 2421 and 2422, the three configurations 2431, 2432, and 2433, the two configurations 2441 and 2442, the two configurations 2451 and 2452, and finally the two configurations 2461 and 2462 belong to the same class. Consequently, the invariance under the three rotations (2102, 2104, and 2106) and the reflection 2108 leads to 10 classes of invariance as shown in FIG. 20.

[0139] From the examples provided hereinabove, whether or not we assume invariance under three rotations and reflections, the number of valid neighborhoods, i.e., the number of classes into which the 64 neighborhoods can be grouped, is either 64, 24, 18, or 10.

[0140] Prior to entropy coding, the pattern undergoes the same transformations, i.e., rotation and reflection, as the neighborhood configurations do belong to one of the invariant classes, which preserves the statistical consistency between the invariant neighborhood configurations and the coded pattern.

[0141] It should be understood that during tree traversal, a child node may necessarily have certain neighboring nodes at the same tree depth that have been previously visited and may be used as dependents. For these same-level neighbors (i.e., at the same level as the child node), instead of referring to neighbors collocated with the parent, same-level neighbors may be used. Because same-level neighbors have half the dimension of the parent, one configuration considers a neighborhood to be occupied if any of its four directly adjacent neighboring child nodes (i.e., the four that share a face with the current node) are occupied. Thus, as will be described in more detail below, the neighborhood configuration of a current node may be determined based on the occupancy data of the current node's neighboring nodes and further based on occupancy data for at least one child node of the neighboring nodes. Thus, one or more probabilities associated with each entropy coder for entropy coding (e.g., binary entropy coding) the occupancy pattern of the current node may be selected not only based on occupancy data for multiple neighboring nodes of the current node (i.e., at the same level as the current node), but also based on occupancy data for at least one (and potentially all) child nodes of the multiple neighboring nodes (at the same level).

[0142] Reference is now made to FIG. 27, which illustrates a current node (i.e., its associated (sub)volume or current (sub)volume) 4000 and its six neighbors 4010, 4020, 4030, 4040, 4050, and 4060. With respect to this example of an octree, the neighbors of the current node may be those nodes (at the same level or depth of the tree) whose associated volumes share faces with the current volume. Other definitions of neighboring nodes are feasible as well. For example, the neighbors of the current node may be those nodes (at the same level or depth of the tree) whose associated volumes share an edge (or vertex) with the current volume. In general, regardless of the tree structure, neighboring nodes may be those nodes (at the same level or depth of the tree) whose associated volumes intersect with the current volume.

[0143] It should be understood that in the context of this application, volumes (nodes) that intersect with each other are adjacent volumes (nodes). Thus, the terms "having an intersection with" and "adjacent to" can be considered synonymous in the context of this application.

[0144] It should be noted that the expressions "volume" and "subvolume" may be used somewhat synonymously, in the sense that each subvolume is itself a volume that may be subdivided into subvolumes. In either case, the volume / subvolume relationship is understood to be explicit through the definition of a parent-child relationship between the nodes / volumes involved.

[0145] The traversal order of the nodes is assumed to be performed breadth-first, in increasing X order, then in increasing Y order, and finally in increasing Z order. By doing so, the three neighbors with the lowest X coordinate (i.e., neighbor 4010), the lowest Y coordinate (i.e., neighbor 4030), and the lowest Z coordinate (i.e., neighbor 4050) have already been coded. Thus, if one of these three neighbors is occupied, the configuration of the occupied subvolume associated with the occupied neighbor is known. While this example defines a traversal order in increasing X order, then in increasing Y order, and finally in increasing Z order, other breadth-first traversal orders can also be used for this purpose.

[0146] Reference is now made to Figure 28, which illustrates an exemplary current volume in which all three previously coded neighbors (i.e., neighbors 4010, 4030, and 4050) are occupied. The occupied subvolumes of neighbor 4010 are subvolumes 4011, 4012, and 4013, the occupied subvolumes of neighbor 4030 are subvolumes 4031, 4032, and 4033, and the occupied subvolumes of neighbor 4050 are subvolumes 4051 and 4052. While in this example all three previously coded neighbors are occupied, it should be understood that in general only two, one, or none of them may actually be occupied.

[0147] Knowledge of the occupied subvolumes of already coded occupied neighbors can be used to refine the occupancy status of the neighbors in calculating the neighborhood occupancy configuration. Reference is now made to FIG. 29(a), where neighbor 4010 has occupied subvolumes 4014 and 4015, neither of which shares a surface with current volume 4000. In this case, it may be advantageous to consider neighbor 4010 as unoccupied in calculating the neighborhood occupancy configuration. In FIG. 29(b), at least one of subvolumes 4016 and 4017 of neighbor 4010 shares a surface with current volume 4000. In this case, neighbor 4010 is considered occupied in calculating the neighborhood occupancy configuration.

[0148] Reference is now made to FIG. 30 , which illustrates in flowchart form an example of a method 4100 for encoding a point cloud to generate a bitstream of compressed point cloud data. A point cloud is defined as a tree structure (e.g., an octree) having multiple nodes with parent-child relationships, recursively divided into sub-volumes, and representing the geometry of a volumetric space containing the points of the point cloud. Each of the operations of method 4100 described below is performed with respect to a current node associated with a (sub)volume that is divided into further sub-volumes, each further sub-volume corresponding to a child node of the current node. In operation 4110, an occupancy pattern for the current node is determined based on the occupancy status of the child nodes. In operation 4120, one or more probabilities (e.g., contexts) associated with each entropy coder are selected for entropy encoding the occupancy pattern. This selection is based on occupancy data for multiple neighboring nodes of the current node and occupancy data for the child nodes of at least one of the multiple neighboring nodes (potentially all of the neighboring nodes). At operation 4130, the occupancy pattern is entropy encoded using the associated one or more entropy coders based on the selected one or more probabilities to generate encoded data for the bitstream.

[0149] In some implementations, the method 4100 may further include an act of updating one or more selected probabilities based on the occupancy pattern (not shown in FIG. 30).

[0150] The occupancy pattern of the current node may be entropy coded using a non-binary entropy coder. In this case, selecting one or more probabilities in operation 4120 of method 4100 may correspond to or involve selecting a probability distribution (and associated non-binary entropy coder) for entropy coding the occupancy pattern. Updating the one or more selected probabilities may then correspond to or involve updating the selected probability distribution.

[0151] On the other hand, as will be described in more detail below, the occupancy pattern of the current node may be entropy coded using a cascade of one or more binary entropy coders. Thus, operation 4120 of method 4100 may involve selecting, for each bit of the bit sequence representing the occupancy pattern, a respective probability (and correspondingly, an associated entropy coder) for coding that bit. Selecting this probability may be based on occupancy data for multiple neighboring nodes of the current node and occupancy data for at least one (possibly all) child node of the multiple neighboring nodes. In addition, selecting this probability may be based on a partial sequence of already coded bits of the bit sequence. In other words, for each bit of the bit sequence, a context may be selected based on occupancy data for multiple neighboring nodes of the current node and occupancy data for at least one (possibly all) child node of the multiple neighboring nodes. In addition, selecting the context may be based on a partial sequence of already coded bits of the bit sequence. From a context perspective, operation 4120 of method 4100 may be thought of as selecting a context for entropy coding an occupancy pattern based on occupancy data for multiple neighboring nodes of the current node and occupancy data for at least one (and potentially all) child nodes of the multiple neighboring nodes. In some implementations, this context may then be updated based on the occupancy pattern.

[0152] Reference is now made to FIG. 31 , which illustrates in flowchart form an example of a method 4200 for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud. The point cloud has multiple nodes with parent-child relationships, is recursively divided into subvolumes, and is defined in a tree structure (e.g., an octree) that represents the geometry of a volumetric space containing the points of the point cloud. Each of the operations of method 4200 described below is performed with respect to a current node associated with a subvolume that is divided into further subvolumes, each further subvolume corresponding to a child node of the current node. At operation 4210, one or more probabilities associated with respective entropy coders for entropy decoding the occupancy pattern are selected. This selection is based on occupancy data for multiple neighboring nodes of the current node and occupancy data of at least one child node of the multiple neighboring nodes. At operation 4220, the bitstream is entropy decoded using the one or more associated entropy coders based on the selected one or more probabilities to generate a reconstructed occupancy pattern for the current node indicative of the occupancy of the child node. In some implementations, the method 4200 may further include an act of updating one or more selected probabilities based on the reconstructed occupancy pattern (not shown in FIG. 31).

[0153] The occupancy pattern of the current node may be entropy coded using a non-binary entropy coder. In this case, selecting one or more probabilities at operation 4210 of method 4200 may correspond to or involve selecting a probability distribution (and associated non-binary entropy coder) for entropy coding the occupancy pattern. Updating the one or more selected probabilities may then correspond to or involve updating the selected probability distribution.

[0154] Meanwhile, the occupancy pattern of the current node may be entropy coded using a cascade of one or more binary entropy coders. Then, in the same manner as for encoding, operation 4210 of method 4200 may involve selecting, for each bit of the bit sequence representing the occupancy pattern, a respective probability (and correspondingly, an associated entropy coder) for coding that bit. Selecting this probability may be based on occupancy data for multiple neighboring nodes of the current node and occupancy data for at least one (possibly all) child node of the multiple neighboring nodes. In addition, selecting this probability may be based on a partial sequence of already coded bits in the bit sequence. In other words, for each bit of the bit sequence, a context may be selected based on occupancy data for multiple neighboring nodes of the current node and occupancy data for at least one (possibly all) child node of the multiple neighboring nodes. In addition, selecting the context may be based on a partial sequence of already coded bits in the bit sequence. From a context perspective, operation 4210 of method 4200 may be thought of as selecting a context for entropy coding an occupancy pattern based on occupancy data for multiple neighboring nodes of the current node and occupancy data for at least one (and potentially all) child nodes of the multiple neighboring nodes. In some implementations, this context may then be updated based on the reconstructed occupancy pattern.

[0155] In some implementations of method 4100 and method 4200, the respective selections in operations 4120 and 4210 may be based on a neighborhood configuration. As described above, the neighborhood configuration may be determined based on occupancy data for neighboring nodes (at the same level) of the current node. Furthermore, occupancy data for at least one (and potentially all) child nodes of multiple neighboring nodes may be used to adapt the neighborhood configuration calculation. In particular, occupancy data for a given child node of multiple neighboring nodes may be used to determine whether the given neighboring node should be considered occupied for purposes of calculating the neighborhood configuration. An example of such adaptation of the neighborhood configuration calculation will be described with reference to FIG. 32.

[0156] Reference is now made to FIG. 32 , which illustrates in flowchart form an example method 4300 for determining the occupancy of neighbors (neighboring nodes) in calculating a neighborhood configuration. The method is performed with respect to a current volume to determine the neighborhood configuration of the current volume. At operation 4310, neighbors of the current volume are selected. For each selected neighbor, the occupancy of the neighbor is checked at operation 4330. If the neighbor is not occupied ("No" at operation 4330), the method proceeds to operation 4340, and the selected neighbor will be considered as unoccupied (e.g., have an occupancy bit equal to zero) in the calculation of the neighborhood occupancy configuration. That is, the occupancy of such neighbor is taken as is for the determination of the neighborhood configuration. The method continues at operation 4320. If the selected neighbor is occupied ("Yes" at operation 4330), it is checked at operation 4350 whether this neighbor has already been coded. If it has not yet been coded ("No" at operation 4350), the method proceeds to operation 4360, where the not-yet-coded neighborhood will be considered occupied in the calculation of the neighborhood occupancy configuration. For such neighborhoods, their occupancy is still taken into account for the determination of the neighborhood configuration. The method continues to operation 4320. If the selected neighborhood has already been coded ("Yes" at operation 4350), it is checked in operation 4370 whether at least one of the occupied sub-volumes of the already-coded neighborhood shares a surface with the current volume. Generally speaking, it is checked in operation 4370 whether at least one of the occupied sub-volumes of the already-coded neighborhood intersects with the current volume. If that is the case ("Yes" at operation 4370), the method proceeds to operation 4360, where the already-coded neighborhood will be considered occupied in the calculation of the neighborhood configuration. Therefore, for such neighborhoods, their occupancy is still taken into account for the determination of the neighborhood configuration.Otherwise ("no" at operation 4370), the method proceeds to operation 4340, where the already coded neighbor will be considered unoccupied in the calculation of the neighborhood configuration. That is, the occupancy bit of that neighbor will be (intentionally / artificially) set to zero in the determination of the neighborhood configuration. The method continues to operation 4320, where it is checked whether there are any neighbors of the current volume that have not been selected. If so ("yes" at operation 4320), the method returns to operation 4310 to select the next neighbor of the current volume. Once all neighbors have been processed ("no" at operation 4320), the neighborhood configuration is calculated at operation 4380 depending on the occupancy (e.g., occupancy bit) of each of the neighbors determined at operation 4340 or operation 4360. This calculation may proceed in the same manner as described above, however, taking into account the occupancy of the neighbors of the current volume as determined at operations 4340 and 4360. In that respect, operation 4340 may be considered to modify the occupancy for a direct decision based solely on the occupancy data of the respective neighborhood. Similarly, the determined neighborhood configuration may be considered to be modified for a direct decision that does not consider occupancy data for neighboring sub-volumes that have already been coded.

[0157] The (modified) neighborhood configuration determined using the modified occupancy of the neighborhood may be used to select one or more probabilities in operation 4120 of method 4100 and operation 4210 of method 4200, unless a deactivation flag in the bitstream indicates that the original neighborhood configuration should be used. This is described in more detail below.

[0158] Methods 4100, 4200, and 4300 have been shown to provide compression gains of over 1% relative to point cloud geometry over direct determination of neighborhood configurations without considering already coded occupancy neighborhood subvolumes.

[0159] It should be understood that the methods described herein above are not limited to neighbors (or neighboring subvolumes) that share faces with the current volume. For example, the neighbors of the current volume could be all those same-level volumes that share faces or edges with the current volume. Thus, the criterion in operation 4370 of method 4300 would need to be replaced with a check whether the neighbors have occupying subvolumes that share faces or edges with the current volume. An example of such a neighborhood definition is illustrated in FIG. 33. As another example, the neighbors of the current volume could be all those same-level volumes that share faces, edges, or vertices with the current volume. Thus, the criterion in operation 4370 of method 4300 would need to be replaced with a check whether the neighbors have occupying subvolumes that share faces, edges, or vertices with the current volume. An example of such a neighborhood definition is illustrated in FIG. 34.

[0160] In the general case, the neighbors of a current volume may be all those same-level volumes that intersect with the current volume. Furthermore, regardless of the definition of the neighbors of the current volume, the criterion in operation 4370 of method 4300 may be replaced by checking whether the neighbors have an occupied subvolume that intersects with the current volume. In other words, it should be understood that methods 4100, 4200, and 4300 can be applied to any tree of nodes with associated volumes, and that the neighbors of a current node are defined as nodes that have the same depth (level) as the current node and whose associated volumes have a non-empty intersection with the current volume associated with the current node. For example, this intersection may be any non-empty set of faces, edges, vertices, or points. An already coded occupancy neighborhood will be considered occupied in the calculation of the neighborhood occupancy configuration if and only if at least one of its occupied child nodes has an associated volume that has a non-empty intersection with the current volume.

[0161] Reference is now made to Figure 33, which illustrates a neighboring volume 4070 that shares an edge with the current volume 4000. In Figure 33(a), the neighboring volume 4070 has occupied sub-volumes 4071 and 4072, neither of which shares an edge with the current volume 4000. In this case, the neighboring volume 4070 is considered unoccupied in the calculation of the neighborhood occupancy configuration. In Figure 33(b), at least one of the sub-volumes 4073 and 4074 of the neighboring volume 4070 shares an edge with the current volume 4000. In this case, the neighboring volume 4070 is considered occupied in the calculation of the neighborhood occupancy configuration.

[0162] Reference is now made to Figure 34, which illustrates a neighboring volume 4080 that shares vertices with the current volume 4000. In Figure 34(a), the neighbor 4080 has occupied sub-volumes 4081 and 4082, neither of which share vertices with the current volume 4000. In this case, the neighbor 4080 is considered unoccupied in the calculation of the neighborhood occupancy configuration. In Figure 34(b), at least one of the sub-volumes 4083 and 4084 of the neighbor 4070 shares vertices with the current volume 4000. In this case, the neighbor 4078 is considered occupied in the calculation of the neighborhood occupancy configuration.

[0163] Methods 4100, 4200, and 4300 have been observed to provide greater than 1% compression gains, i.e., greater than 1% reduction in compressed bitstream size, for dense virtual reality-oriented point clouds, which are notable gains given the simplicity of the methods.

[0164] However, for sparse point clouds captured, for example, by a LiDAR mounted on a moving vehicle, these methods may show little or no gain (or even slight loss in extremely sparse point clouds). Therefore, it may be advantageous to add a flag to the bitstream indicating activation (flag value 1) or deactivation (flag value 0) of the adaptation of the neighbor occupancy of the current volume. Deactivation means that the neighbor is considered occupied / unoccupied in the calculation of the neighbor occupancy configuration, independent of the location of its occupied subnodes.

[0165] (Entropy coding of tree occupancy patterns using binary coding) Some of the above-described techniques for using neighborhood occupancy information to code tree occupancy are detailed in European Patent Application No. 18305037.6. The described embodiments focus on the use of non-binary entropy coding of occupancy patterns, in which pattern distributions are selected based on neighborhood occupancy information. However, in some instances, using a binary coder may be more efficient from a hardware implementation perspective. Furthermore, on-the-fly updating of many probabilities may require fast-access memory and computations within the core of an arithmetic coder. Therefore, it may be advantageous to find a method and device for entropy encoding occupancy patterns using a binary arithmetic coder. If this can be done without significantly degrading compression performance and while avoiding having an overwhelming number of contexts to track, using a binary coder would be advantageous.

[0166] The use of a binary coder instead of a non-binary coder is reflected in the entropy formula below. H(X1,X2|Y)=H(X1|Y)H(X2|Y,X1)

[0167] where X = (X1, X2) is the non-binary information to be coded, and Y is the context for coding, i.e., the neighborhood configuration or selected pattern distribution. To convert the non-binary coding of X to binary coding, the information (X1, X2) is split into information X1 and X2, which can be coded separately without increasing entropy. To do this, one must code one of the two that depends on the other, where X2 depends on X1. This can be extended to n bits of information in X. For example, for n = 3, H(X1,X 2, X3|Y)=H(X1|Y)H(X2|Y,X1)H(X3|Y,X1,X2)

[0168] It should be understood that as the occupancy pattern, i.e., bit sequence X, becomes longer, more conditions exist for coding later bits in the sequence. For a binary coder (e.g., CABAC), this means a significant increase in the number of contexts to track and manage. Using an octree as an example, if the occupancy pattern is an 8-bit sequence b=b0···b7, the bit sequence may be divided into eight binary information bits b0···b7. The coding may use a neighborhood N (or N) to determine the context. As described above, assume that the neighborhood can be reduced to 10 valid neighborhoods through grouping the neighborhoods into invariant classes, where N is an integer belonging to {0, 1, 2,···, 9}. For brevity, a "class of invariant neighborhoods" may sometimes be simply referred to as a "neighborhood" herein, but it should be understood that this reduced number of neighborhoods may be realized based on a class-based grouping of neighborhoods based on invariance.

[0169] Figure 21 illustrates the division of an 8-bit pattern or sequence into eight individual bits for binary entropy coding. Note that the first bit of the sequence is encoded based on the neighborhood configuration; therefore, there are a total of 10 available contexts. The next bit of the sequence is encoded based on the neighborhood configuration and any previously encoded bit, namely, bit b0. This results in a total of 20 available contexts: obtained as the product of 10 from N and 2 from b0. The final bit, b7, is entropy encoded using a context selected from the 1,280 available contexts obtained as the product of 10 from N and 128 from the partial patterns given by previously encoded bits b0,...,b6. That is, for each bit, the number of contexts (i.e., possible combinations of conditions / subjects) is a function of the number of neighborhood configurations defined (10 in this example, based on the grouping of 64 neighborhood configurations into classes) and the number of possible partial patterns from the ordered sequence of n-1 previously encoded bits (2 n-1 (given by

[0170] As a result, there are a total of 2,550 contexts to maintain in relation to the binary coding of the occupancy patterns. This is an excessively large number of contexts to keep track of, and the relative shortage can cause poor performance due to context dilution, especially for later bits in the sequence.

[0171] Thus, in one aspect, this application discloses an encoder and decoder that determines whether a set of contexts can be reduced, and if so, applies a context reduction operation to achieve a smaller set of available contexts for entropy coding at least a portion of the occupancy pattern using a binary coder. In another aspect, this application further discloses an encoder and decoder that applies one or more steps of state reduction using the same context reduction operation to perform valid context selection from a fixed number of contexts. In some implementations, context reduction is applied a priori in generating a lookup table of contexts and / or algorithmic conditionals, which are then used by the encoder or decoder in selecting a suitable context. The reduction is based on testable conditions that the encoder and decoder evaluate to determine which lookup table to select or how to index / select from that lookup table to obtain the selected context.

[0172] Reference is now made to Figure 22, which illustrates, in flowchart form, an exemplary method 3000 for coding occupancy patterns in a tree-based point cloud coder using binary coding. The method 3000 may be implemented by an encoder or a decoder. In the case of an encoder, the coding operation is encoding, and in the case of a decoder, the coding operation is decoding. The encoding and decoding are context-based entropy encoding and decoding.

[0173] The exemplary method 3000 is for entropy coding an occupancy pattern, i.e., a bit sequence, for a particular node / volume. The occupancy pattern indicates the occupancy status of the node / volume's child nodes (subvolumes). In the case of an octree, there are eight child nodes / subvolumes. In operation 3002, a neighborhood is determined. The neighborhood is the occupancy status of one or more volumes located near the volume whose occupancy pattern is to be coded. As discussed above, there are various possible implementations for determining the neighborhood. In some examples, there are ten neighborhoods, and the neighborhood for the current volume is identified based on the occupancy of six volumes that share a surface with the current volume.

[0174] In operation 3004, an index i for the child node of the current volume is set to 0. Then, in operation 3006, an assessment is made as to whether context reduction is possible. Different possible context reduction operations are discussed in more detail below. The assessment of whether context reduction is possible may be based, for example, on the bits (e.g., index values) in the bit sequence being coded. In some cases, context reduction may be possible for later bits in the sequence but not for the first few bits. The assessment of whether context reduction is possible may be based, for example, on neighborhood configuration, since certain neighborhood configurations may allow for simplification. Additional factors may be used in assessing whether context reduction is possible in some implementations. For example, an upper bound Bo may be provided as the maximum number of contexts that a binary coder can use to code bits; if the initial number of contexts for coding bits is greater than Bo, context reduction is applied so that the number of contexts after reduction is at most Bo (otherwise, it is not applied). Such a bound Bo may be defined in the encoder and / or decoder specifications to ensure that a software or hardware implementation capable of handling Bo contexts will always be able to encode and / or decode the point cloud without overflowing in terms of the number of contexts. Knowing the bound Bo in advance also makes it possible to predict the complexity and memory footprint induced by a binary entropy coder, thus facilitating hardware design. Typical values ​​for Bo are in the tens to hundreds.

[0175] If it is determined that context reduction is available, then in operation 3008, a context reduction operation is applied. The context reduction operation reduces the number of available contexts in the set of available contexts to a smaller set containing fewer total contexts. Recall that the number of available contexts may depend in part on the bit position in the sequence, i.e., on the index, since the context may depend on partial patterns of previously coded bits from the bit sequence. In some implementations, the number of available contexts in the set before reduction may be based on the number of neighborhood configurations multiplied by the number of possible partial patterns with previously coded bits. For a bit at index i, where i ranges from 0 to n, the number of partial patterns is 2 i can be given by

[0176] As mentioned above, in some implementations, the context reduction operation is performed prior to coding, and the resulting reduced context set is the context set available for use by the encoder and decoder during the coding operation. The use and / or selection of the reduced context set during coding may be based on evaluation of one or more conditions prior to use of those reduced sets that correspond to the conditions evaluated in operation 3006 to determine that the number of contexts can be reduced. For example, in the case of a particular neighborhood configuration that enables use of a reduced context set, the encoder and / or decoder may first determine whether the neighborhood configuration condition is met and then, if applicable, use the corresponding reduced context set.

[0177] In operation 3010, bit b i The context for bit b is determined, i.e., iA context for the current bit is selected from the set of available contexts (or a reduced set, if present) based on the neighborhood configuration and the partial pattern of previously coded bits in the bit sequence. The current bit is then entropy encoded by a binary coder in operation 3012 using the selected context.

[0178] In operation 3014, the index i is determined to be equal to or less than i if the currently coded bit is the last bit in the sequence, i.e., i max If so, the coding process proceeds to the next node. Otherwise, the index i is incremented in operation 3016 and the process returns to operation 3006.

[0179] It should be understood that in some implementations, the context selection may not depend on the neighborhood configuration. In some cases, it may depend only on the partial pattern, if any, of previously coded bits in the sequence.

[0180] A simplified block diagram of a portion of an exemplary encoder 3100 is illustrated in Figure 23. In this illustration, it should be understood that the occupancy pattern 3102 is obtained as the corresponding volume is partitioned into child nodes and cycled through a FIFO buffer 3104 that holds the geometry of the point cloud. The coding of the occupancy pattern 3102 is illustrated as involving a cascade of binary coders 3106, one for each bit of the pattern. Between at least some of the binary coders 3106 are context reduction operations 3108 that operate to reduce the available contexts to a smaller set of available contexts.

[0181] 23 illustrates a series of binary coders 3106, in some implementations only one binary coder is used. If more than one coder is used, the coding may be (partially) parallelized. Given the contextual dependency of one bit on preceding bits in the bit sequence, the coding of a pattern cannot necessarily be fully parallelized, but it may be possible to improve pipelining through the use of cascading binary coders for certain patterns to achieve some degree of parallelism and speed improvement.

[0182] (Context reduction operation) The above example suggests that the coding process includes a context reduction operation on at least one bit of the occupancy pattern, so as to reduce the set of available contexts to a smaller set of available contexts. In this sense, a "context reduction operation" refers to the reduction of a particular bit b i This can be understood as identifying and consolidating contexts that may be considered overlapping or redundant in certain situations. As described above, the reduced context set may be determined prior to coding and provided to the encoder and decoder, which decide whether to use the reduced context set based on the same conditions described below for reducing the context set.

[0183] (Neighborhood formation reduction through screening / shielding) A first exemplary context reduction operation involves reducing the number of neighboring configurations based on screening / shielding. In principle, neighboring configuration incorporates the occupancy status of neighboring volumes into the context selection process, based on the neighboring volumes' usefulness in indicating whether the current volume or subvolume is likely occupied. As bits associated with subvolumes within the current volume are decoded, they are also incorporated into the context selection; however, information from nearby subvolumes may be more significant and useful than occupancy information for neighboring volumes located on the other side of the subvolume from the current subvolume. In this sense, previously decoded bits are associated with subvolumes that "screen" or "shield" the neighboring volumes. This may mean that in such situations, the occupancy of a neighboring volume can be ignored, since the relevance of its occupancy status is encompassed by the occupancy status of the subvolumes between the current subvolume and the neighboring volume, thereby enabling a reduction in the number of neighboring configurations.

[0184] Reference is now made to Figure 24, which diagrammatically illustrates an exemplary context reduction operation based on neighborhood screening. The example involves coding an occupancy pattern for a volume 3200. The occupancy pattern indicates the occupancy status of eight sub-volumes within the volume 3200. In this example, four sub-volumes in the upper half of the volume 3200 have been coded, and therefore their occupancy status is known. The bit of the occupancy pattern being coded is associated with a fifth sub-volume 3204, located in the lower half of the volume 3200, below the four previously coded sub-volumes.

[0185] Coding in this example involves determining a context based on neighborhood configurations. Ten neighborhood configurations 3202 are shown. Volume 3200, which includes a fifth sub-volume 3204 to be coded, is shown in light gray and designated by reference numeral 3200. Neighborhood configuration 3202 is based on the occupancy status of volumes adjacent to and sharing a surface with volume 3200. The neighborhood volumes include a top neighboring volume 3206.

[0186] In this example, the number of neighborhood configurations can be reduced from 10 to 7 by ignoring the top neighborhood volume 3206 in at least some of the configurations. As shown in FIG. 24 , three of the four configurations in which the top neighborhood volume 3206 is shown are subsumed under equivalent configurations that do not incorporate the top neighborhood volume 3206, thereby reducing the number of neighborhood configurations to a total of seven. It may still be advantageous to keep the configurations that show all six neighboring volumes, because there is no existing five-volume neighborhood configuration into which the six-volume configuration could be consolidated (eliminating the five-element one), which means that even if the top neighborhood volume were removed, a new five-element neighborhood configuration would result, and no overall reduction in context would occur.

[0187] The upper neighboring volume 3206 can be excluded from the neighborhood configuration in this example because the context decision for coding the occupancy bit associated with the fifth subvolume 3204 already takes into account the occupancy status of the four previously coded subvolumes directly above it, which is a better indication of the likelihood and direction of occupancy for the fifth subvolume than the occupancy status of the more distant upper neighboring volume 3206.

[0188] The above example in which the upper neighboring volume 3206 is screened or shielded by a previously coded subvolume when coding the occupancy bits corresponding to the fifth subvolume 3204 is only one example. Depending on the coding order within the volume 3200, several other possible screening / shielding situations may be realized and exploited to reduce the available neighborhood configurations.

[0189] Reference is now made to FIG. 25, which illustrates a second example of screening / shielding. In this example, the occupancy pattern for volume 3200 has been almost completely coded. The subvolume to be coded is the eighth subvolume, hidden (invisible) in the rear bottom corner of the figure. In this case, the occupancy status of all seven other subvolumes has been coded, particularly the subvolumes along the top (thus reducing the neighborhood to a total of seven) and along the right and front sides. Thus, in addition to screening the top neighboring volume, the subvolumes with previously coded occupancy bits shield the front neighboring volume 3210 and the right neighboring volume 3212. This may allow the neighborhood to be reduced from a total of seven to a total of five, as shown.

[0190] It should be understood that the two foregoing examples of shielding are illustrative, and that in some cases, different configurations may be merged to take into account different shielding situations. It should be understood that the context reduction operation based on shielding / screening by a previously coded sub-volume is general and not limited to these two examples, but cannot be applied in the case of the first sub-volume to be coded, because it requires that there be at least one previously coded occupied bit associated with a neighboring sub-volume for there to be any shielding / screening.

[0191] It should also be understood that the degree of shielding / screening required to justify neighborhood reduction may vary in different implementations. In the two above examples, all four subvolumes that share a surface with a neighboring volume were previously coded before that neighboring volume was considered shielded / screened, and therefore were removed from the neighborhood. In other examples, partial shielding / screening may be sufficient, for example, from one to three previously coded subvolumes that share a surface.

[0192] (Context reduction through special case handling) There are certain cases where context reduction can be performed without loss of useful information. In the exemplary context determination process described above, the context for coding the occupancy bits is based on neighborhood configurations, i.e., the pattern of occupancy of volumes located in the neighborhood of the current volume, and the partial patterns resulting from the occupancy of sub-volumes within the current volume that have been previously coded. The latter condition is due to the fact that for the eighth bit in the occupancy pattern bit sequence, the second bit to be tracked is 7 = 128 contexts. Even if the neighborhood configuration is reduced to a total of 5, that still means 640 contexts to track.

[0193] The number of contexts is large due to the fact that previously coded bits of a bit sequence have a certain order, and that order is relevant in assessing the context. However, in some cases, the order may not contain useful information. For example, if the neighborhood structure is empty, i.e., N 10If , ... j This means that the condition can be reduced to j+1 conditions, i.e., the context for coding one of the bits of a bit sequence is based on previously coded bits, but only on their sum, not their ordered pattern. In other words, the entropy representation in this special case can be expressed as follows: H(b|n)≒H(b0|0)H(b1|0,b0)H(b2|0,b0+b1)...H(b7|0,b0+b1+...+b6)

[0194] In some implementations, a similar observation may be made regarding a full neighborhood configuration. In some examples, a full neighborhood configuration lacks directionality, meaning that the order of previously coded bits need not be considered in determining the context. In some examples, this context reduction operation may be applied to only some of the bits in a bit sequence, such as some of the later bits in the sequence. In some cases, application of this context reduction operation to later bits may be conditional on determining that earlier bits associated with previously coded sub-volumes are also fully occupied.

[0195] (Statistical-based context reduction) Statistical analysis may be used to reduce contexts by determining those that lead to approximately the same statistical behavior and then combining them. This analysis may be performed a priori using test data to develop a reduced context set, which is then provided to both the encoder and decoder. In some cases, analysis may be performed on the current point cloud using two-pass coding to develop a custom reduced context set for the specific point cloud data. In some such cases, the mapping from the unreduced context set to the custom reduced context set may be indicated to the decoder by using dedicated syntax coded into the bitstream.

[0196] Two contexts can be compared through the concept of "distance". A first context c has a probability p that bit b is equal to zero, and a second context c' has a probability p' that bit b' is equal to zero. The distance between c and c' is given by: d(c,c')=|plog2p-p'log2p'|+|(1-p)log2(1-p)-(1-p')log2(1-p')|

[0197] Using this measure of similarity (distance), the contexts can then be grouped in a process such as:

[0198] 1. Start with the M1 context and fix the threshold level ε.

[0199] 2. For a given context, regroup into a class all contexts that have a distance from the given context that is less than a threshold level ε.

[0200] 3. Repeat step 2 for all unregrouped contexts until all are placed into a class.

[0201] 4. Label the M2 classes of 1~M2: This results in a brute-force reduction function that maps {1, 2, ···, M1] → [1, 2, ···, M2], where M1 ≥ M2.

[0202] The brute-force reduction function for mapping a set of contexts to a smaller set of contexts may be stored in memory to be applied by the encoder / decoder as a context reduction operation during coding. The mapping may be stored as a lookup table or other data structure. The brute-force reduction function may, for example, be applied only for later bits in a bit sequence (pattern).

[0203] (Combinations and subcombinations of context reduction actions) Three exemplary context reduction operations are described above. Each of them may be applied individually and independently in some implementations. Any two or more of them may be combined in some implementations. Additional context reduction operations may be implemented alone or in combination with any one or more of the context reduction operations described above.

[0204] 26 illustrates in flowchart form an example of a method 3300 for occupancy pattern binary coding with combined context reduction. The method 3300 is based on a 10-element neighborhood configuration N 10 Given N, we code the 8-bit binary pattern b0, b1, . . . , b7. The first condition evaluated is whether the neighborhood is empty, i.e., N 10 =0. If so, the bits are coded without reference to their order, as indicated by reference numeral 3302. If not, the bits are coded normally up to bit b4, at which point the encoder and decoder apply the brute force context reduction function BR ito reduce the number of contexts by mapping the set of contexts defined by neighborhood configurations and partial patterns of previously coded bits into a smaller set of contexts with substantially similar statistical performance.

[0205] In this example, the last two bits b6 and b7 are coded using a reduced neighborhood structure based on shielding / screening.

[0206] All functions can be implemented as look-up tables (LUTs) to reduce the size of the set of contexts. In one practical implementation, all reductions are simply incorporated into reduction functions, i.e., LUTs, that take contexts as input and provide reduced contexts as output. In this exemplary embodiment, the total number of contexts is reduced from 2,550 to 576, and each reduction function BR i The output sizes of are 70, 106, 110, and 119, respectively.

[0207] (Context selection in a system with a fixed number of contexts) Each of the above-described context reduction operations can be further used in compression systems with a static (fixed) minimum number of contexts. In such designs, for a given symbol in an 8-bit binary pattern, one or more reduction operations are applied to determine a context probability model for encoding or decoding the symbol.

[0208] (Impact on compression performance) The use of 10 neighbors and non-binary coding provides compression gains over the current implementation of the MPEG test model for point cloud coding. However, the use of 10 neighbors with cascaded binary coding using 2,550 contexts proposed above results in even greater improvements in compression efficiency. Even when context reduction is used, such as using the three techniques detailed above to reduce the contexts to a total of 576, binary coding compression is still slightly better than the implementation using non-binary coding and much better than the test model. This observation has been shown to be consistent across different test point cloud data.

[0209] Reference is now made to FIG. 14 , which illustrates a simplified block diagram of an exemplary embodiment of an encoder 1100. The encoder 1100 includes a processor 1102, a memory 1104, and an encoding application 1106. The encoding application 1106 may include a computer program or application stored in the memory 1104 and including instructions that, when executed, cause the processor 1102 to perform operations such as those described herein. For example, the encoding application 1106 may encode and output a bitstream encoded according to the processes described herein. It should be understood that the encoding application 1106 may be stored on a non-transitory computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard drive, or the like. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions to operate as a dedicated processor implementing the described processes. Such a processor may, in some examples, be referred to as a “processor circuit” or “processor circuitry.”

[0210] Reference is now also made to FIG. 15 , which illustrates a simplified block diagram of an exemplary embodiment of a decoder 1200. The decoder 1200 includes a processor 1202, a memory 1204, and a decoding application 1206. The decoding application 1206 may include a computer program or application stored in the memory 1204 and including instructions that, when executed, cause the processor 1202 to perform operations such as those described herein. It should be understood that the decoding application 1206 may be stored on a computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard drive, or the like. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions, such that it operates as a dedicated processor implementing the described processes. Such a processor may, in some examples, be referred to as a “processor circuit” or “processor circuitry.”

[0211] It should be understood that a decoder and / or encoder according to the present application may be implemented in a number of computing devices, including, but not limited to, a server, a suitably programmed general-purpose computer, a machine vision system, and a mobile device. The decoder or encoder may be implemented using software that includes instructions for configuring a processor or multiple processors to perform the functions described herein. The software instructions may be stored on any suitable non-transitory computer-readable memory, including a CD, RAM, ROM, flash memory, etc.

[0212] It should be understood that the modules, routines, processes, threads, or other software components implementing the decoders and / or encoders described herein and the described methods / processes for configuring the encoders or decoders can be realized using standard computer programming techniques and languages. The present application is not limited to a particular processor, computer language, computer programming notation, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated chip (ASIC), etc.

[0213] The present application also provides a computer-readable signal encoding data produced through application of an encoding process according to the present application.

[0214] Certain adaptations and modifications of the described embodiments can be made. Accordingly, the embodiments discussed above are considered to be illustrative rather than restrictive.

Claims

[Claim 1] The invention described in this specification.

Images