Method and device for binary entropy coding of point clouds

Abstract

Method and device for encoding a point cloud SOLUTION: A bit sequence indicative of an occupancy pattern for a sub-volume of the volume is coded using binary entropy coding. For a given bit in the bit sequence, the context may be based on the sub-volume neighborhood configuration for the sub-volume corresponding to that bit. The sub-volume neighborhood configuration depends on an occupancy pattern of a group of sub-volumes of neighboring volumes of the volume, the group of sub-volumes being located in a neighborhood of the sub-volume corresponding to the given bit. The context may be further based on a partial sequence of previously coded bits of the bit sequence.SELECTED DRAWING: None

Description

[Technical Field]

[0001] This application relates generally to point cloud compression, and more particularly to a method and device for binary entropy coding of point clouds. [Background technology]

[0002] Data compression is used in communications and computer networking to efficiently store, transmit, and reproduce information. There is growing interest in representing three-dimensional objects or spaces, which can involve large data sets, for which efficient and effective compression would be highly useful and beneficial. In some cases, a three-dimensional object or space can be represented using a point cloud, which is a set of points each having three coordinate locations (X, Y, Z) (and possibly other attributes such as color data (e.g., luminance and chrominance), transparency, reflectance, normal vectors, etc.). A point cloud can be static (a stationary object or a snapshot of an environment / object at a single point in time) or dynamic (a time-ordered sequence of point clouds).

[0003] Exemplary applications for point clouds include topography and mapping applications. Autonomous vehicles and other machine vision applications may rely on point cloud sensor data in the form of 3D scans of an environment, such as from LiDAR scanning devices. Virtual reality simulations may rely on point clouds.

[0004] It should be appreciated that point clouds may involve large amounts of data, and compressing (encoding and decoding) that data quickly and accurately is of great interest. Therefore, it would be advantageous to provide methods and devices for more efficiently and / or effectively compressing data related to point clouds. Furthermore, it would be advantageous to find methods and devices for coding point clouds that can be implemented using context-adaptive binary entropy coding without requiring management of an excessive number of contexts. Summary of the Invention [Means for solving the problem]

[0005] This application describes methods for encoding and decoding point clouds, and encoders and decoders for encoding and decoding point clouds. A bit sequence indicating an occupation pattern for sub-volumes of a current volume may be coded using entropy coding (e.g., binary entropy coding). Probabilities (e.g., context) for the entropy coding may be based on a neighborhood configuration of the current volume and a partial sequence of previously coded bits of the bit sequence. These probabilities may be further based on occupation data for sub-volumes of at least one neighboring volume of the current volume. In particular, probabilities for coding bits in the bit sequence may be selected at least in part based on a sub-volume neighborhood configuration, which depends on the occupation pattern of a group of sub-volumes of at least one neighboring volume located near the sub-volume of the current volume corresponding to the bit in the bit sequence.

[0006] In an example useful for understanding the present application, a decision may be made as to whether to apply a context reduction operation, and if so, the operation reduces the number of available contexts. Exemplary context reduction operations include shielding with sub-volumes associated with previously coded bits, special handling for empty neighbors, and reducing neighbors based on statistically-based context merging. The reduction may be applied before coding, and a decision may be made during coding as to whether the conditions for using a reduced context set are met.

[0007] In one aspect, the present application provides a method for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing the geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of the volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, and the volume having a plurality of neighboring volumes. The method includes determining a bit sequence indicating occupancy of subvolumes of a current volume with respect to a current node associated with a current volume, the current volume being divided into subvolumes, each subvolume corresponding to a child node of the current node. The method further includes determining, for at least one bit in the bit sequence of the current volume, a subvolume neighborhood configuration based on occupancy data for subvolumes of at least one neighboring volume of the current volume, the subvolume neighborhood configuration depending on an occupancy pattern of a group of subvolumes of the at least one neighboring volume located near the subvolume of the current volume corresponding to the bit in the bit sequence. The method further includes selecting probabilities (e.g., contexts) for use in entropy encoding of bits in the bit sequence, the selecting being based at least in part on the subvolume neighborhood configuration. The method further includes entropy encoding the bits in the bit sequence based on the selected probabilities using a binary entropy encoder to generate encoded data for a bitstream.

[0008] Another aspect relates to a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing geometry of a volume space recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of the volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in scan order within the volume, the volume having a plurality of neighboring volumes. The method includes determining a sub-volume neighborhood configuration based on occupancy data for sub-volumes of at least one neighboring volume of the current volume with respect to a current node associated with a current volume, the current volume being divided into sub-volumes, each sub-volume corresponding to a child node of the current node, and with respect to at least one bit in the bit sequence of the current volume, the sub-volume neighborhood configuration depending on an occupancy pattern of a group of sub-volumes of the at least one neighboring volume located near the sub-volume of the current volume corresponding to the bit in the bit sequence. The method further includes selecting a probability (e.g., a context) for use in entropy decoding of a bit in the bit sequence, the selecting being based at least in part on the sub-volume neighborhood configuration, and entropy decoding the at least one bit based on the selected probability using a binary entropy decoder to generate a reconstructed bit from the bitstream.

[0009] In some implementations, determining the subvolume neighborhood configuration may involve determining, based on occupancy data for the subvolumes of at least one neighboring volume of the current volume, a number of subvolumes of at least one neighboring volume that are located in proximity to the subvolume of the current volume that corresponds to a bit in the bit sequence, and may further involve applying a threshold function to the determined number.

[0010] In some implementations, the subvolume neighborhood configuration of a given subvolume within a given volume may correspond to the pattern of occupancy of subvolumes within neighboring volumes of the given volume that are located in the vicinity of the given subvolume.

[0011] In some implementations, determining the sub-volume neighborhood configuration may be based on occupancy data for sub-volumes of those neighboring volumes of the current volume that have already been coded.

[0012] In some implementations, selecting the probability may be further based on a partial sequence of bits of the already coded bit sequence and / or a neighborhood configuration of the current volume, which may correspond to a pattern of occupancy of neighboring volumes of the current volume.

[0013] In some implementations, determining the subvolume neighborhood configuration may involve determining, based on occupancy data for subvolumes of at least one neighboring volume of the current volume, all subvolumes of the at least one neighboring volume that intersect with a subvolume of the current volume that corresponds to a bit in the bit sequence. The determining may further involve applying respective weighting factors to the determined subvolumes, each weighting factor depending on the intersection of each determined subvolume with the subvolume of the current volume that corresponds to the bit in the bit sequence. The determining may further involve determining, based on the determined subvolumes and their respective weighting factors, a weighted number of subvolumes of the at least one neighboring volume that intersect with the subvolume of the current volume that corresponds to the bit in the bit sequence.

[0014] In some implementations, the scanning order within the current volume for determining the bit sequence indicating the occupancy pattern may be determined such that the maximum possible number of neighboring subvolumes in the already coded neighboring volumes of the current volume does not increase from one subvolume to the next subvolume in the scanning order.

[0015] In some implementations, the occupancy data for the sub-volumes of a given neighboring volume of the current volume may include an occupancy status for each of the sub-volumes of the given neighboring volume.

[0016] In some implementations, the tree structure may represent an octree.

[0017] In some implementations, the encoding method may further include encoding a flag indicating that the probability for use in entropy encoding of the at least one bit is selected based at least in part on the sub-volume neighborhood configuration.

[0018] In some implementations, the decoding method may further include decoding a flag from the bitstream, the flag indicating that a probability for use in entropy decoding of at least one bit should be selected based at least in part on the sub-volume neighborhood configuration.

[0019] In another aspect, this application provides a method for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure, the tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of a volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes, and a pattern of occupancy of the neighboring volumes being a neighborhood configuration. The method includes determining that a context reduction condition is met for at least one bit in a bit sequence of a volume, and based thereon, selecting a reduced context set including fewer contexts than the product of a count of neighboring configurations and the number of previously coded bits in the sequence; selecting a context from the reduced context set for coding the at least one bit based on the occupancy status of at least some of the neighboring volumes and at least one previously coded bit of the bit sequence; entropy encoding the at least one bit based on the selected context using a binary entropy encoder to generate encoded data for the bitstream; and updating the selected context.

[0020] In another aspect, this application provides a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure, the tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of a volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes, and a pattern of occupancy of the neighboring volumes being a neighborhood configuration. The decoding method includes determining that a context reduction condition is met for at least one bit in a bit sequence of a volume, and based thereon, selecting a reduced context set including fewer contexts than the product of the count of neighboring configurations and the number of previously coded bits in the sequence; selecting a context from the reduced context set for coding the at least one bit based on the occupancy status of at least some of the neighboring volumes and at least one previously coded bit of the bit sequence; entropy decoding the at least one bit based on the selected context using a binary entropy decoder to generate a reconstructed bit from the bitstream; and updating the selected context.

[0021] In some implementations, the context reduction condition may include determining that one or more previously coded occupied bits are associated with one or more respective sub-volumes that are positioned between the sub-volume associated with the at least one bit and one or more of the neighboring volumes. In some cases, this may involve determining that the four sub-volumes associated with a previously encoded bit share an aspect with a particular neighboring volume.

[0022] In some implementations, the context reduction condition may include determining that at least four bits of the bit sequence have been previously coded.

[0023] In some implementations, determining that a context reduction condition is met may include determining that a pattern of occupancy of neighboring volumes indicates that multiple neighboring volumes are unoccupied. In some of those cases, the selected reduced context set may include a number of contexts corresponding to a number of previously coded bits in the bit sequence, and optionally, selecting a context may include selecting a context based on a sum of previously coded bits in the bit sequence.

[0024] In some implementations, the context reduction condition may include determining that at least a certain threshold number of bits in the bit sequence have been previously coded, and the reduced context set may include a lookup table that maps each possible combination of neighborhood and previously coded bit patterns in the bit sequence to fewer contexts. In some examples, the lookup table may be generated based on iterative grouping of available contexts into multiple classes based on determining that a distance measure between each pair of available contexts is less than a certain threshold, and each class in the multiple classes may include a respective context in a smaller set, and there may be an available context for each possible combination of neighborhood and previously coded bit patterns in the bit sequence.

[0025] In some implementations, at least some of the neighboring volumes are neighboring volumes that share at least one face with the volume.

[0026] In a further aspect, the present application describes encoders and decoders configured to implement such methods of encoding and decoding.

[0027] In yet a further aspect, the present application describes a non-transitory computer-readable medium storing computer-executable program instructions that, when executed, cause one or more processors to perform the described methods of encoding and / or decoding.

[0028] In yet another aspect, this application describes computer-readable signals containing program instructions that, when executed by a computer, cause the computer to perform the described methods of encoding and / or decoding.

[0029] This application further describes computer-implemented applications, including topography applications, mapping applications, automotive industry applications, autonomous driving applications, virtual reality applications, cultural heritage applications, etc. These computer-implemented applications include processes for receiving a data stream or data file, decompressing the data stream or data file to obtain a bitstream of compressed point cloud data, and decoding the bitstream as described in the above aspects and implementations thereof, thereby utilizing point cloud compression techniques in accordance with aspects and implementations thereof described throughout this application.

[0030] The present application further describes methods for encoding and decoding point clouds, and encoders and decoders for encoding and decoding point clouds. In some implementations, a receiving unit receives multiplexed data, where the multiplexed data is obtained by multiplexing the coded point cloud data with other coded data types, such as metadata, images, video, audio, and / or graphics. The receiving unit includes a demultiplexing unit for separating the multiplexed data into the coded point data and the other coded data, and at least one decoding unit (or decoder) for decoding the coded point cloud data. In some other implementations, an emitting unit emits the multiplexed data, where the multiplexed data is obtained by multiplexing the coded point cloud data with other coded data types, such as metadata, images, video, audio, and / or graphics. The emitting unit includes at least one encoding unit (or encoder) for encoding the point cloud data and a multiplexing unit for combining the coded point cloud data and the other coded data into the multiplexed data.

[0031] The present invention provides, for example, the following items. (Item 1) 1. A method of encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing the geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of the volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in scan order within the volume, the volume having a plurality of neighboring volumes; The method comprises: determining, for a current node associated with a current volume divided into subvolumes, each subvolume corresponding to a child node of the current node, the bit sequence indicating the occupancy of the subvolumes of the current volume; For at least one bit in the bit sequence of the current volume, determining a sub-volume neighborhood configuration based on occupancy data for sub-volumes of at least one neighboring volume of the current volume, wherein the sub-volume neighborhood configuration depends on an occupancy pattern of a group of sub-volumes of the at least one neighboring volume that are located in the vicinity of the sub-volume of the current volume that corresponds to the bit in the bit sequence; selecting probabilities for use in entropy encoding of the bits in the bit sequence, said selecting being based at least in part on the sub-volume neighborhood configuration; and entropy encoding the bits in the bit sequence based on the selected probabilities using a binary entropy encoder to generate encoded data for the bitstream; A method comprising: (Item 2) 1. A method of decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of the volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes; The method comprises: For a current node associated with a current volume divided into sub-volumes, each sub-volume corresponding to a child node of the current node, For at least one bit in the bit sequence of the current volume, determining a sub-volume neighborhood configuration based on occupancy data for sub-volumes of at least one neighboring volume of the current volume, wherein the sub-volume neighborhood configuration depends on an occupancy pattern of a group of sub-volumes of the at least one neighboring volume that are located in the vicinity of the sub-volume of the current volume that corresponds to the bit in the bit sequence; selecting probabilities for use in entropy decoding of the bits in the bit sequence, said selecting being based at least in part on the sub-volume neighborhood configuration; and entropy decoding the at least one bit based on the selected probability using a binary entropy decoder to generate a reconstructed bit from the bitstream; A method comprising: (Item 3) Determining the sub-volume neighborhood configuration includes: determining, based on the occupancy data for the sub-volumes of the at least one neighboring volume of the current volume, the number of the sub-volumes of the at least one neighboring volume that are located adjacent to the sub-volume of the current volume that corresponds to the bit in the bit sequence; applying a threshold function to the determined number; and The method according to item 1 or 2, comprising: (Item 4) A method according to any one of items 1-3, wherein the subvolume neighborhood configuration of a given subvolume within a given volume corresponds to a pattern of occupation of subvolumes within neighboring volumes of the given volume that are located in the vicinity of the given subvolume. (Item 5) 5. The method according to any one of items 1-4, wherein determining the sub-volume neighborhood configuration is based on occupancy data for sub-volumes of those neighboring volumes of the current volume that have already been coded. (Item 6) 6. The method according to any one of items 1-5, wherein selecting the probability is further based on a partial sequence of bits of the bit sequence already coded and / or a neighborhood configuration of the current volume, the neighborhood configuration of the current volume corresponding to a pattern of occupation of the neighboring volumes of the current volume. (Item 7) Determining the sub-volume neighborhood configuration includes: determining, based on the occupancy data for the sub-volumes of the at least one neighboring volume of the current volume, all those sub-volumes of the at least one neighboring volume that intersect with the sub-volume of the current volume that corresponds to the bit in the bit sequence; applying respective weighting factors to the determined sub-volumes, each weighting factor depending on an intersection of the respective determined sub-volume with that sub-volume of the current volume corresponding to the bit in the bit sequence; determining a weighted number of sub-volumes of the at least one neighboring volume that intersect with the sub-volume of the current volume that corresponds to the bit in the bit sequence based on the determined sub-volumes and their respective weighting factors; 7. The method according to any one of items 1 to 6, comprising: (Item 8) 8. The method according to any one of items 1-7, wherein the scanning order within the current volume is determined such that the maximum possible number of neighboring subvolumes in already coded neighboring volumes of the current volume does not increase from one subvolume to the next subvolume in the scanning order. (Item 9) 9. The method of any one of items 1-8, wherein the occupancy data for the sub-volumes of a given neighboring volume of the current volume comprises an occupancy status for each of the sub-volumes of the given neighboring volume. (Item 10) 10. The method according to any one of items 1-9, wherein the tree structure represents an octree. (Item 11) Item 2 or the method of any one of items 3-10 when dependent on item 2, further comprising decoding a flag from the bitstream, the flag indicating that the probability for use in entropy decoding of at least one bit should be selected based at least in part on the sub-volume neighborhood configuration. (Item 12) 1. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of the volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes; The encoder comprises: a processor; Memory and an encoding application including instructions executable by said processor; Equipped with An encoder, wherein the instructions, when executed, cause the processor to perform the method of item 1 or any one of items 3-10 when dependent on item 1. (Item 13) 1. A decoder for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, the point cloud being defined in a tree structure having a plurality of nodes with a parent-child relationship, the plurality of nodes representing a geometry of a volume space, the volume space being recursively divided into sub-volumes containing points of the point cloud, the occupancy of the sub-volumes of the volume being indicated using a bit sequence, each bit of the bit sequence indicating the occupancy of a respective sub-volume in a scan order within the volume, the volume having a plurality of neighboring volumes; The decoder a processor; Memory and a decoding application including instructions executable by said processor; Equipped with The instructions, when executed, cause the processor to perform the method of item 2 or any one of items 3-11 when dependent on item 2. (Item 14) 12. A non-transitory processor-readable medium storing processor-executable instructions that, when executed by a processor, cause the processor to perform the method of any one of items 1-11. (Item 15) A computer-readable signal containing program instructions that, when executed by a computer, cause the computer to perform the method described in any one of items 1-11. Reference will now be made, by way of example, to the accompanying drawings which illustrate exemplary embodiments of the present application. [Brief explanation of the drawings]

[0032] [Figure 1] FIG. 1 shows a simplified block diagram of an exemplary point cloud encoder.

[0033] [Figure 2]FIG. 2 shows a simplified block diagram of an exemplary point cloud decoder.

[0034] [Figure 3] FIG. 3 shows an exemplary partial sub-volume and associated tree structure for coding.

[0035] [Figure 4] FIG. 4 illustrates the recursive partitioning and coding of an octree.

[0036] [Figure 5] FIG. 5 shows an example scanning pattern within an example cube from an octree.

[0037] [Figure 6] FIG. 6 shows an example occupation pattern within an example cube.

[0038] [Figure 7] FIG. 7 illustrates in flow chart form an exemplary method for encoding a point cloud.

[0039] [Figure 8] FIG. 8 illustrates a portion of an exemplary octree.

[0040] [Figure 9] FIG. 9 shows an example of a neighborhood sub-volume.

[0041] [Figure 10] FIG. 10 illustrates an exemplary neighborhood configuration showing occupancy among neighboring nodes.

[0042] [Figure 11] FIG. 11 graphically illustrates one illustrative embodiment of the process of point cloud entropy encoding using parent pattern dependent context.

[0043] [Figure 12]FIG. 12 shows an illustrative embodiment of a process for point cloud entropy encoding using neighborhood-dependent context.

[0044] [Figure 13] FIG. 13 illustrates in flow chart form an exemplary method for decoding a bitstream of compressed point cloud data.

[0045] [Figure 14] FIG. 14 shows an exemplary simplified block diagram of an encoder.

[0046] [Figure 15] FIG. 15 shows an exemplary simplified block diagram of a decoder.

[0047] [Figure 16] FIG. 16 illustrates an example Cartesian coordinate system and example rotations and / or reflections about axes.

[0048] [Figure 17] FIG. 17 shows the class of invariance of neighborhood configurations under one or several iterations of rotation around the Z axis.

[0049] [Figure 18] FIG. 18 shows the class of neighborhood configuration invariance with respect to vertical reflections.

[0050] [Figure 19] FIG. 19 shows the classes that are invariant with respect to both rotation and reflection.

[0051] [Figure 20] Figure 20 shows the classes of invariance under three rotations and reflections.

[0052] [Figure 21] FIG. 21 illustrates the equivalence between non-binary coding and cascaded binary coding with respect to occupancy patterns.

[0053] [Figure 22] FIG. 22 illustrates in flowchart form an exemplary method for coding occupancy patterns in a tree-based point cloud coder using binary coding.

[0054] [Figure 23] FIG. 23 shows a simplified block diagram of a portion of an exemplary encoder.

[0055] [Figure 24] FIG. 24 illustrates diagrammatically an exemplary context reduction operation based on neighborhood screening.

[0056] [Figure 25] FIG. 25 illustrates another exemplary context reduction operation based on neighborhood screening.

[0057] [Figure 26] FIG. 26 illustrates in flow chart form an example of a method for binary coding occupancy patterns using combined context reduction.

[0058] [Figure 27] FIG. 27 shows an example of neighboring subvolumes, some of which have already been coded.

[0059] [Figure 28] FIG. 28 shows an example of neighboring sub-volumes and their already coded sub-volumes.

[0060] [Figure 29] FIG. 29 illustrates in flow chart form a method for encoding a current node occupancy pattern based at least in part on a sub-volume neighborhood configuration.

[0061] [Figure 30]FIG. 30 illustrates in flow chart form a method for decoding a current node's occupancy pattern based at least in part on a sub-volume neighborhood configuration.

[0062] [Figure 31] FIG. 31 shows an example of how the volume associated with the current node is divided into sub-volumes.

[0063] [Figure 32] Figures 32-35 show examples of sub-volume neighborhood configurations for different sub-nodes of the current node. [Figure 33] Figures 32-35 show examples of sub-volume neighborhood configurations for different sub-nodes of the current node. [Figure 34] Figures 32-35 show examples of sub-volume neighborhood configurations for different sub-nodes of the current node. [Figure 35] Figures 32-35 show examples of sub-volume neighborhood configurations for different sub-nodes of the current node.

[0064] [Figure 36] FIG. 36 illustrates in flow chart form a method for determining sub-volume neighborhoods for subnodes of a current node.

[0065] [Figure 37] FIG. 37 illustrates in flow chart form an example method for binary coding of occupancy patterns using combined context reduction and context determination based on sub-volume neighborhood configurations. DETAILED DESCRIPTION OF THE INVENTION

[0066] Similar reference numbers may be used in different figures to represent similar components.

[0067] Other aspects and features of the present application will be understood by those skilled in the art from a review of the following description of examples in conjunction with the accompanying figures.

[0068] Any feature described in connection with one aspect or embodiment of the invention may also be used in connection with one or more other aspects / embodiments. These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described herein.

[0069] At times in the following description, the terms "node," "volume," and "subvolume" may be used interchangeably. It should be understood that a node is associated with a volume or subvolume. A node is a particular point on a tree, which may be an internal node or a leaf node. A volume or subvolume is a bounded physical space that a node represents. The term "volume" may be used in some cases to refer to the largest bounded space defined to contain a point cloud. A volume may be recursively subdivided into subvolumes for the purpose of constructing a tree structure of interconnected nodes for coding point cloud data.

[0070] As used herein, the term "and / or" is intended to include any one of the listed elements alone, any subcombination, or all of the elements, and to cover all possible combinations and subcombinations of the listed elements without necessarily excluding additional elements.

[0071] As used herein, the phrase "or at least one of" is intended to cover any one or more of the listed elements, including any one of the listed elements alone, any subcombination, or all of the elements, without necessarily excluding any additional elements, and without necessarily requiring all of the elements.

[0072] A point cloud is a set of points in a three-dimensional coordinate system. The points are often intended to represent the exterior surfaces of one or more objects. Each point has a location (position) in the three-dimensional coordinate system. The position may be represented by three coordinates (X, Y, Z), which may be Cartesian or any other coordinate system. Points may have other associated attributes, such as color, which in some cases may also be three-component values ​​such as R, G, B or Y, Cb, Cr. Other associated attributes may include transparency, reflectance, normal vectors, etc., depending on the desired use for the point cloud data.

[0073] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or topography can be static point cloud data. LiDAR-based scanning of an environment for machine vision purposes can be dynamic in that the point cloud (at least potentially) changes over time, e.g., with each successive scan of a volume. A dynamic point cloud is thus a time-ordered sequence of point clouds.

[0074] Point cloud data can be used in several applications, including preservation (scanning historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous automobiles), and virtual reality systems, to name a few. Dynamic point cloud data for applications such as machine vision can be quite different from static point cloud data such as that for preservation purposes. For example, automotive vision typically involves relatively low-resolution, colorless, and highly dynamic point clouds acquired through LiDAR (or similar) sensors with high acquisition frequencies. The purpose of such point clouds is not for human consumption or viewing, but rather for machine object detection / classification in decision processes. As an example, a typical LiDAR frame contains about several thousand points, while high-quality virtual reality applications require millions of points. It can be expected that there will be a demand for higher resolution data over time as computational speeds improve and new applications are discovered.

[0075] Although point cloud data is useful, the lack of effective and efficient compression, i.e., encoding and decoding processes, can hinder adoption and deployment. A particular challenge in coding point clouds that does not arise with other data compression, such as audio or video, is coding the geometry of the point cloud. Point clouds tend to be populated sparsely, which makes efficiently coding the location of the points much more difficult.

[0076] One of the more common mechanisms for coding point cloud data is through the use of a tree-based structure. In a tree-based structure, a bounded three-dimensional volume for the point cloud is recursively divided into subvolumes. Nodes of the tree correspond to the subvolumes. The decision to further divide a subvolume may be based on the resolution of the tree and / or whether there are any points contained within the subvolume. A leaf node may have an occupancy flag that indicates whether its associated subvolume contains points. A split flag may indicate whether the node has child nodes (i.e., whether the current volume has been further divided into subvolumes). These flags may be entropy coded in some cases, and predictive coding may be used in some cases.

[0077] A commonly used tree structure is the octree. In this structure, all volumes / subvolumes are cubes, and each division of a subvolume results in eight additional subvolumes / subcubes. Another commonly used tree structure is the KD tree, in which a volume (a cube or a rectangular cuboid) is recursively divided into two by a plane orthogonal to one of the axes. The octree is a special case of the KD tree, in which the volume is divided by three planes, each orthogonal to one of the three axes. Both of these examples relate to a cube or a rectangular cuboid; however, the present application is not limited to such tree structures, and volumes and subvolumes may have other shapes in some applications. The partitioning of a volume is not necessarily into two subvolumes (KD tree) or eight subvolumes (octree), but may involve other partitioning, including division into non-rectangular shapes, or non-adjacent subvolumes.

[0078] Although this application may refer to octrees for ease of explanation and because they are popular candidate tree structures for automotive applications, it should be understood that the methods and devices described herein may be implemented using other tree structures.

[0079] Reference is now made to Figure 1, which illustrates a simplified block diagram of a point cloud encoder 10, in accordance with an aspect of the present application. The point cloud encoder 10 includes a tree construction module 12 that receives point cloud data and generates a tree (in this example, an octree) that represents a geometry in volume space that contains the point cloud and indicates the locations or positions of points from the point cloud within that geometry.

[0080] The basic process for creating an octree for coding a point cloud may include: 1. Start with a bounded volume (a cube) containing the points in a coordinate system. 2. Divide the volume into 8 subvolumes (8 subcubes). 3. For each subvolume, mark the subvolume with 0 if it is empty, and with 1 if there is at least one point in it. For all subvolumes marked in 4.1, repeat (2) to split those subvolumes until the maximum depth of splitting is reached. 5. For every leaf subvolume (subcube) at maximum depth, if it is not empty, mark the leaf cube with 1, otherwise mark it with 0.

[0081] The above process can be described as an occupancy = split process, where splitting implies occupancy and there is a constraint that there is a maximum depth or resolution beyond which no further splitting will occur. In this case, a single flag indicates whether a node is split, and therefore whether it is occupied by at least one point, and vice versa. At the maximum depth, the flag indicates occupancy and no further splitting is possible.

[0082] In some implementations, partitioning and occupancy are independent, whereby a node can be occupied and a node can be partitioned or not. Two variations of this implementation exist: 1. Occupancy after split. A signal flag indicates whether a node is split. If it is split, the node must contain a point, i.e., splitting implies occupancy. Otherwise, if the node should not be split, a further occupancy flag indicates whether the node contains at least one point. Thus, when a node is not further split, i.e., it is a leaf node, the leaf node must have an associated occupancy flag to indicate whether it contains any points. 2. Occupy then split. A single flag indicates whether the node is occupied. If it is not occupied, no splitting occurs. If it is occupied, a split flag is coded to indicate whether the node will be further split.

[0083] Regardless of the above-described process used to construct the tree, it may be traversed in a predetermined order (breadth-first or depth-first, and according to a scanning pattern / order within each divided subvolume) to generate a sequence of bits from the flags (occupancy and / or split flags). This may be referred to as serializing or binarizing the tree. As shown in Figure 1, in this example, the point cloud encoder 10 includes a binarizer 14 for binarizing the octree and generating a bitstream of binarized data representing the tree.

[0084] This sequence of bits may then be encoded using an entropy encoder 16 to generate a compressed bitstream. The entropy encoder 16 may encode the sequence of bits using a context model 18, which defines probabilities for coding bits based on context decisions by the entropy encoder 16. The context model 18 may be adaptively updated after coding each bit or a defined set of bits. The entropy encoder 16 may, in some cases, be a binary arithmetic encoder. The binary arithmetic encoder may, in some implementations, employ context-adaptive binary arithmetic coding (CABAC). In some implementations, coders other than arithmetic coders may also be used.

[0085] In some cases, the entropy encoder 16 may not be a binary coder and may instead operate on non-binary data. The output octree data from the tree building module 12 may not be evaluated in binary form and may instead be encoded as non-binary data. For example, in the case of an octree, the eight flags (e.g., occupancy flags) in the subvolumes in their traversal order are 2 8It may be considered a 1-bit number (e.g., an integer having a value between 1 and 255, since the value 0 is not possible for a partitioned subvolume (i.e., it is not partitioned if it is not occupied at all). This number may, in some implementations, be encoded by an entropy encoder using a multi-symbol arithmetic coder. The sequence of flags that defines this integer within a subvolume (e.g., a cube) may be called a "pattern."

[0086] As in video or image coding, point cloud coding can involve prediction operations, in which an attempt is made to predict patterns for sub-volumes. Prediction can be spatial (relying on previously coded sub-volumes within the same point cloud) or temporal (relying on previously coded points in a time-ordered sequence of point clouds).

[0087] A block diagram of an exemplary point cloud decoder 50 corresponding to encoder 10 is shown in FIG. 2. Point cloud decoder 50 includes an entropy decoder 52 that uses the same context model 54 used by encoder 10. Entropy decoder 52 receives an input bitstream of compressed data and entropy decodes the data, producing an output sequence of decompressed bits. The sequence is then converted into reconstructed point cloud data by tree reconstructor 56. Tree reconstructor 56 reconstructs a tree structure from the decompressed data and knowledge of the scan order in which the tree data was binarized. Tree reconstructor 56 is therefore able to reconstruct point locations from the point cloud (given the resolution of the tree coding).

[0088] An exemplary partial subvolume 100 is shown in FIG. 3. In this example, the subvolume 100 is shown in two dimensions for ease of illustration, and the size of the subvolume 100 is 16×16. Note that the subvolume is divided into four 8×8 subsquares, two of which are further subdivided into 4×4 subsquares, three of which are further subdivided into 2×2 subsquares, and one of the 2×2 subsquares is then divided into a 1×1 square. The 1×1 square is the maximum depth of the tree and represents the finest resolution for the positional point data. Points from the point cloud are shown as dots in the figure.

[0089] The structure of the tree 102 is shown to the right of the subvolume 100. The sequence of split flags 104 and the corresponding sequence of occupancy flags 106 obtained in a given breadth-first traversal order are shown to the right of the tree 102. It will be observed that in this illustrative example, there is an occupancy flag for each subvolume (node) that is not split (i.e., has an associated split flag set to zero). These sequences may be entropy encoded.

[0090] Another example employing the occupancy = split condition is shown in FIG. 4. FIG. 4 illustrates the recursive splitting and coding of an octree 150. Only a portion of the octree 150 is shown in the figure. A FIFO 152 is shown processing a node for splitting to illustrate the breadth-first nature of this process. The FIFO 152 outputs an occupied node 154 that has been queued in the FIFO 152 for further splitting after processing its parent node 156. The tree builder splits the subvolume associated with the occupied node 154 into eight subvolumes (cubes) and determines their occupancy. The occupancy may be indicated by an occupancy flag for each subvolume. In a prescribed traversal order, the flags may be referred to as an occupancy pattern for the node 154. The pattern may be defined by an integer representing the sequence of occupancy flags associated with the subvolumes in the prescribed traversal order. For an octree, the pattern is an integer in the range [1, 255].

[0091] The entropy encoder then encodes the patterns using a non-binary arithmetic encoder based on the probabilities specified by the context model. In this example, the probabilities are pattern distributions based on an initial distribution model and can be adaptively updated. In one implementation, the pattern distribution is effectively a counter of the number of times each pattern (an integer between 1 and 255) is encountered during coding. The pattern distribution can be updated after each subvolume is coded. The pattern distribution can be normalized, if necessary, since the relative frequency of patterns is closely related to the probability assessment, not the absolute count.

[0092] Based on the pattern, those child nodes that are occupied (e.g., have flag=1) are then pushed into FIFO 152 for further splitting in order (provided the node is not at the maximum depth of the tree).

[0093] Reference is now made to Figure 5, which shows an example cube 180 from an octree. The cube 180 is subdivided into eight subcubes. The scan order for reading the flags results in an 8-bit string, which can be read as an integer [1,255] in binary. Based on the scan order and the resulting bit position of each subcube's flag within the string, the subcube has the value shown in Figure 5. The scan order can be any sequence of subcubes, provided that both the encoder and decoder use the same scan order.

[0094] As an example, Figure 6 shows cube 180 with four "front" subcubes occupied. This would correspond to pattern 85, based on the occupied subcubes being cubes 1 + 4 + 16 + 64. The integer pattern number defines the pattern of occupation within the subcubes.

[0095] Octree representations, or more generally, any tree representation, are efficient at representing spatially correlated points because trees tend to factor out the higher-order bits of point coordinates. With an octree, each level of depth refines the coordinates of a point within a subvolume by one bit for each component, at the expense of eight bits per refinement. Further compression is obtained by entropy coding the partitioning information (i.e., pattern) associated with each tree node. This further compression is possible because pattern distributions are not uniform; i.e., non-uniformity is another consequence of correlation.

[0096] One potential inefficiency in current systems is that a pattern distribution (e.g., a histogram of pattern numbers found in previously coded nodes of the tree) is developed over the course of coding the point cloud. In some cases, the pattern distribution can be initialized as equiprobable or to some other predetermined distribution, but the use of one pattern distribution means that the context model does not consider or exploit local geometric correlations.

[0097] In European Patent Application No. 18305037.6, the applicant described a method and device for selecting from available pattern distributions to be used in coding the pattern of occupancy of a particular node based on certain occupancy information from previously coded nodes in the vicinity of the particular node. In one exemplary implementation, the occupancy information is obtained from the pattern of occupancy of parents to the particular node. In another exemplary implementation, the occupancy information is obtained from one or more nodes located in the vicinity of the particular node. The contents of European Patent Application No. 18305037.6 are incorporated herein by reference.

[0098] Reference is now made to Figure 7, which illustrates in flowchart form an exemplary method 200 for encoding a point cloud. The method 200 in this example involves recursive partitioning of occupied nodes (subvolumes) and breadth-first traversal of the tree for coding.

[0099] In operation 202, the encoder determines a pattern of occupancy for the current node. The current node is an occupied node divided into eight child nodes, each corresponding to a respective subcube. The pattern of occupancy for the current node defines the occupancy of the eight child nodes in the traversal order. As explained above, this pattern of occupancy may be represented using integers between 1 and 255, e.g., an 8-bit binary string.

[0100] In operation 204, the encoder selects a probability distribution from a set of probability distributions. The selection of a probability distribution is based on occupancy information from nearby, previously coded nodes, i.e., at least one node that is a neighbor of the current node. In some embodiments, two nodes are neighbors if they are associated with respective subvolumes that share at least one face. In a broader definition, nodes are neighbors if they share at least one edge. In an even broader definition, two nodes are neighbors if they share at least one vertex. A parent pattern (in which the current node is a child node) provides occupancy data for the current node and its seven sibling nodes. In some implementations, the occupancy information is a parent pattern. In some implementations, the occupancy information is occupancy data for a set of neighboring nodes, which include nodes that have different parent nodes but are at the same tree depth level as the current node. In some cases, combinations of these are possible. For example, the set of neighboring nodes may include some sibling nodes and some non-sibling nodes.

[0101] Once a probability distribution is selected, the encoder entropy encodes the occupancy pattern for the current node using the selected probability distribution, as indicated by operation 206. It then updates the selected probability distribution based on the occupancy pattern (e.g., it may increment a count corresponding to the occupancy pattern) in operation 208. In operation 210, the encoder evaluates whether there are more nodes to code, and if so, returns to operation 202 to code the next node.

[0102] The probability distribution selection in operation 204 should be based on occupancy data for nearby, previously coded nodes. This allows both the encoder and decoder to make the same selection independently. For the following discussion of probability distribution selection, reference will be made to FIG. 8, which diagrammatically illustrates a partial octree 300 including a current node 302. The current node 302 is an occupied node and is being evaluated for coding. The current node 302 is one of eight children of a parent node 306, which is in turn a child of a grandparent node (not shown). The current node 302 is differentiated into eight child nodes 304. The occupancy pattern for the current node 302 is based on the occupancy of the child nodes 304. For example, as shown, using the notation where black dots are occupied nodes, the occupancy pattern could be 00110010, i.e., pattern 50.

[0103] The current node 302 has a sibling node 308 that has the same parent node 306. The parent pattern is the occupancy pattern for the parent node 306, which, as shown, would be 00110000, i.e., pattern 48. The parent pattern can serve as the basis for selecting a suitable probability distribution for entropy encoding the occupancy pattern for the current node.

[0104] FIG. 9 illustrates the set of neighbors surrounding a current node, where neighbors are defined as nodes that share a face. In this example, the node / subvolume is a cube, and the cube at the center of the image has six neighbors, one on each face. It should be understood that in an octree, the neighborhood of the current node would include three sibling nodes. It would also include three nodes that do not have the same parent node. Thus, occupancy data for some of the neighboring nodes will be available because they are siblings, but occupancy data for some neighboring nodes may or may not be available depending on whether those nodes have been previously coded. Special handling may be applied to deal with missing neighbors. In some implementations, missing neighbors may be presumed occupied or unoccupied. It should be understood that the neighborhood definition may be expanded to include neighboring nodes based on shared edges or shared vertices to include additional adjacent subvolumes in the assessment.

[0105] It should be appreciated that the foregoing process examines the occupancy of neighboring nodes in an attempt to determine the likelihood of occupancy of the current node 302 to select a more favorable context and use more accurate probabilities for entropy coding the occupancy data of the current node 302. It should be appreciated that the occupancy status of neighboring nodes that share faces with the current node 302 may be a more accurate assessment of whether the current node 302 is likely isolated than an assessment based on the occupancy status of sibling nodes, three of which will share only edges and one of which will share only vertices (in the case of an octree). However, assessing sibling occupancy status has the advantage of being modular in that all relevant data for the assessment is part of the parent node, which means it has a smaller memory footprint for implementation, but assessing neighbor occupancy status involves buffering tree occupancy data if it is needed when determining neighbor occupancy status in connection with coding future neighboring nodes.

[0106] Neighbor occupancy may be read in a scan order that effectively assigns a value to each neighbor, similar to that described above with respect to occupancy patterns. As shown, neighboring nodes effectively take on values ​​of 1, 2, 4, 8, 16, or 32, and therefore there are 64 (0-63) possible neighbor occupancy configurations. This value may be referred to herein as a "neighbor configuration." As an example, FIG. 10 illustrates an example neighborhood configuration 15 in which neighbors 1, 2, 4, and 8 are occupied and neighbors 16 and 32 are empty.

[0107] In some cases, both of the above criteria (parent pattern and neighborhood configuration) may be applied, or a choice may be made between them. For example, if neighborhoods are available, the probability distribution selection may be based on neighboring nodes; however, if one or more of the neighbors are not available because they are from a node that has not yet been coded, the probability distribution selection may fall back to an analysis based on sibling nodes (parent patterns).

[0108] In yet another embodiment, the probability distribution selection may alternatively or additionally be based on a grandparent pattern. In other words, the probability distribution selection may be based on the occupancy status of uncle nodes that are siblings of the parent node 306.

[0109] In still further implementations, additional or alternative assessments may be incorporated into the probability distribution selection. For example, the probability distribution selection may examine the occupancy status of the parent node's neighbors or the grandparent node's neighbors.

[0110] Any two or more of the above criteria for assessing local occupancy status may be used in combination in some implementations.

[0111] For non-binary entropy coders, the occupancy data for the current node can be coded by selecting a probability distribution. The probability distribution contains a number of probabilities corresponding to the number of possible occupancy patterns for the current node. For example, when coding the occupancy patterns of an octree, 2 8 -1 = 255 possible patterns, meaning that each probability distribution contains 255 probabilities. In some embodiments, the number of probability distributions may be equal to the number of possible occupancy outcomes in the selection criteria (i.e., using neighbor, sibling, and / or parent occupancy data). For example, if the parent pattern for the octree is used as the selection criteria to determine the probability distribution to use, there will be 255 probability distributions, each with 255 probabilities. For neighborhood configurations, if neighborhoods are defined as sharing a face, there will be 64 probability distributions, each containing 255 probabilities.

[0112] It should be understood that too many distributions can result in slow adaptation due to a lack of data (i.e., context dilution). Thus, in some embodiments, similar patterns can be grouped to use the same probability distribution. For example, separate distributions can be used for patterns corresponding to fully occupied cases, vertically oriented cases, horizontally oriented cases, mostly empty cases, and all other cases. This can reduce the number of probability distributions to approximately five. It should be understood that different groupings of patterns can be formed to result in different numbers of probability distributions.

[0113] Reference is now made to Figure 11, which diagrammatically illustrates one exemplary embodiment of a process 400 for point cloud entropy encoding using parent pattern dependent context. In this example, a current node 402 is split into eight child nodes, whose occupancy patterns 404 are to be encoded using a non-binary entropy encoder 406. The non-binary entropy encoder 406 uses a probability distribution selected from one of six possible probability distributions 408. The selection is based on the parent pattern, i.e., the selection is based on occupancy information from parent nodes for the current node 402. The parent patterns are identified by integers between 1 and 255.

[0114] The selection of the probability distribution may be a decision tree that assesses whether the pattern corresponds to a full node (e.g., pattern = 255), a horizontal structure (e.g., pattern = 170 or 85, assuming the Z axis is vertical), a vertical structure (e.g., pattern = 3, 12, 48, 192), a sparsely populated distribution (e.g., pattern = 1, 2, 4, 8, 16, 32, 64, or 128, i.e., none of the sibling nodes are occupied), a semi-sparsely populated distribution (total number of occupied nodes between the current node and sibling nodes ≦ 3), and all other cases. The exemplary patterns shown for different categories are merely examples. For example, the "horizontal" category may include patterns with two or three occupied cubes on the same horizontal level. The "vertical" category may include patterns with three or four occupied cubes in a wall-like arrangement. It should be understood that finer gradations may also be used. For example, the "horizontal" category may be further subdivided horizontally in the top portion of the cube and horizontally in the bottom portion of the cube, with different probability distributions for each. Other groupings of occupancy patterns with certain correlations may also be made and assigned corresponding probability distributions. Further discussion of grouping patterns in the context of neighborhoods and invariance between neighborhoods is provided further below.

[0115] Figure 12 shows an illustrative embodiment of a process 500 for point cloud entropy encoding using neighborhood-dependent context. This example assumes the definitions of neighborhood and neighborhood numbering used above in connection with Figure 9. This example also assumes that each neighborhood has a dedicated probability distribution, meaning that there are 64 different probability distributions. The current node 502 has an occupancy pattern 504 to be encoded. The probability distribution is selected based on the neighbors of the current node 502. That is, the neighborhood NC in [0, 63] is found and used to select the associated probability distribution.

[0116] It should be understood that in some embodiments, neighborhoods may be grouped such that two or more neighborhoods use the same probability distribution based on similarities in patterns. In some embodiments, the process may use different arrangements of neighborhoods for contextualization (selection) of distributions. Additional neighborhoods may be added, such as eight diagonally adjacent neighbors on all three axes or twelve diagonally adjacent neighbors on two axes. Embodiments that avoid certain neighborhoods may also be used, for example, to avoid using neighborhoods that introduce additional dependencies in depth-first traversals, or to only introduce dependencies on certain axes to reduce codec states for large trees.

[0117] In this example, the case of NC=0 is handled in a specific manner. If there are no occupied neighbors, it may indicate that the current node 502 is isolated. Therefore, the process 500 further checks how many of the child nodes for the current node 502 are occupied. If only one child node is occupied (i.e., the occupation number (NO) is equal to 1), a flag indicating that a single child node is occupied is encoded, and the index for the node is coded using 3 bits. If two or more child nodes are occupied, the process 500 uses the NC=0 probability distribution to code the occupancy pattern.

[0118] Reference is now made to FIG. 13, which illustrates in flowchart form an exemplary method 600 for decoding a bitstream of encoded point cloud data.

[0119] In operation 602, the decoder selects one of the probability distributions based on occupancy information from one or more nodes in the neighborhood of the current node. As explained above, the occupancy information can be a parent pattern from a parent node to the current node, i.e., the occupancy of the current node and its siblings, or it can be the occupancy of neighboring nodes of the current node, which can include some of the sibling nodes. Other or additional occupancy information can also be used in some implementations.

[0120] Once a probability distribution is selected, in operation 604, the decoder entropy decodes a portion of the bitstream using the selected probability distribution to reconstruct an occupancy pattern for the current node. The occupancy pattern is used by the decoder in reconstructing the tree to reconstruct the encoded point cloud data. Once the point cloud data is decoded, it may be output from the decoder for use, such as for rendering a view, segmentation / classification, or other uses.

[0121] In operation 606, the decoder updates the probability distribution based on the reconstructed occupancy pattern, and then, if there are more nodes to decode, it moves to the next node in the buffer and returns to operation 602.

[0122] An exemplary implementation of the method described above has proven to provide compression improvements with a negligible increase in coding complexity. Neighborhood-based selection exhibits better compression performance than parent-pattern-based selection, but it has additional computational complexity and memory usage. In some tests, the relative improvement in bits per point over the MPEG point cloud test model is 4-20%. Note that initializing the probability distribution based on the distribution arrived at using test data leads to improved performance compared to initializing with a uniform distribution.

[0123] Some of the above examples are based on tree coding processes that use non-binary coders to represent occupancy patterns. New developments to employ binary entropy coders are presented further below.

[0124] In one variation of neighborhood-based probability distribution selection, the number of distributions can be reduced by exploiting neighborhood symmetry. By replacing neighborhoods or replacing pattern distributions, structurally similar configurations with symmetry lines can reuse the same distribution. In other words, neighborhood configurations that can use the same pattern distribution can be grouped into a class. A class containing two or more neighborhoods can be referred to herein as a "neighborhood configuration," in that one of the neighborhood configurations effectively subsumes the other neighborhood configurations with a reflection or replacement of the other configurations.

[0125] As an example, consider eight corner patterns NC ∈ [21, 22, 25, 26, 37, 38, 41, 42], each representing a corner neighborhood pattern symmetry. These values ​​of NC are node specific patterns, but are likely to be significantly correlated with different patterns. Furthermore, these correlated patterns are likely to follow the same symmetry as the neighborhood patterns. As an example, a method can be implemented that reuses a single distribution to represent multiple instances of NC by substituting the probabilities of that distribution.

[0126] An encoder derives a pattern number for a node based on the occupancy of its child nodes. The encoder selects a distribution and a replacement function according to the neighborhood configuration. The encoder reorders the probabilities contained in the distribution according to the replacement function and then uses the replaced distribution to arithmetically encode the pattern number. The arithmetic encoder's updates of the probabilities in the replaced distribution are mapped back to the original distribution using the inverse replacement function.

[0127] The corresponding decoder first selects the same distribution and substitution function according to the neighborhood configuration. A substitution distribution is generated in the same manner as the encoder, and the substitution distribution is used by the arithmetic decoder to entropy decode the pattern number. The bits comprising the pattern number are then each assigned to the corresponding child.

[0128] Note that the same permutation can be achieved without reordering the data in the distribution itself, but rather by introducing a level of indirection and using a permutation function to permutate the lookup of a given index within the distribution.

[0129] An alternative embodiment considers permutations of the patterns themselves rather than the distributions, allowing for shuffling prior to entropy encoding or after decoding. Such methods are likely more suitable for efficient implementation through a bitwise shuffle operation. In this case, no reordering of the distributions is performed by either the encoder or decoder; rather, the computation of the encoded pattern number is

number

number

number

[0130] The method of deriving the required replacement can be based on the rotational symmetry of the neighboring structures or on the reflection along a particular axis. Furthermore, it is not necessary that the replacement replaces all positions, e.g., according to the symmetry; partial replacement can be used instead. For example, when replacing NC=22 with NC=41, positions on the symmetry axis may not be replaced, and the mapping

number

[0131] An example of an embodiment based on rotational symmetry and reflection is provided below for the specific case of an octree with six neighbors that share a common face with the current cube. Without loss of generality, as shown in Figure 16, the Z axis runs perpendicular to the direction of viewing the figure. The relative positions of neighbors such as "above" ("below") should then be understood as along the Z axis in increasing (decreasing) Z directions. The same applies with respect to left / right sides along the X axis and front / back along the Y axis.

[0132] 16 shows three rotations 2102, 2104, and 2106 along the Z, Y, and X axes. The angles of these three rotations are 90 degrees, i.e., they perform a quarter turn rotation along their respective axes.

[0133] FIG. 17 shows classes of invariance of neighborhood configurations under one or several iterations of rotation 2102 along the Z axis. This invariance represents the same statistical behavior of the point cloud geometry along any direction belonging to the XY plane. This is particularly true for the use case of a car moving on the Earth's surface, which is locally approximated by the XY plane. A horizontal configuration is a given occupancy of four neighbors (located on the left, right, front, and back sides of the current cube) independent of the occupancy of the upper neighbors (2202) and the lower neighbors (2204). The four horizontal configurations 2206, 2208, 2210, and 2212 belong to the same class of invariance under rotation 2102. Similarly, two configurations 2214 and 2216 belong to the same class of invariance. There are only six classes of invariance under rotation 2102 (grouped under the set of classes 2218).

[0134] A vertical configuration is a given occupancy of two neighbors 2202 and 2204, independent of the occupancy of the four neighbors located on the left, right, front, and back sides of the current cube. There are four possible vertical configurations, as shown in Figure 18. Consequently, there are 6 x 4 = 24 possible configurations when considering invariance to rotation 2102 along the Z axis.

[0135] Reflection 2108 along the Z axis is shown in Figure 16. Vertical configurations 2302 and 2304 depicted in Figure 18 belong to the same class of invariance under reflection 2108. There are three classes of invariance under reflection 2108 (grouped under the set of classes 2306). Invariance under reflection 2108 means that the up and down directions behave essentially the same in terms of point cloud geometry statistics. This is an accurate assumption for a moving car on a road.

[0136] If we assume invariance under both rotation 2102 and reflection 2108, there are 18 classes of invariance resulting from the intersection of the two pairs 2218 and 2306. These 18 classes are represented in FIG.

[0137] Applying further invariance under two other rotations 2104 and 2106, the two configurations 2401 and 2402 belong to the same class of invariance. Furthermore, the two configurations 2411 and 2412, the two configurations 2421 and 2422, the three configurations 2431, 2432, and 2433, the two configurations 2441 and 2442, the two configurations 2451 and 2452, and finally the two configurations 2461 and 2462 belong to the same class. Consequently, the invariance under the three rotations (2102, 2104, and 2106) and the reflection 2108 leads to 10 classes of invariance as shown in FIG. 20.

[0138] From the examples provided hereinabove, whether or not we assume invariance under three rotations and reflections, the number of valid neighborhoods, i.e., the number of classes into which the 64 neighborhoods can be grouped, is either 64, 24, 18, or 10.

[0139] Prior to entropy coding, the pattern undergoes the same transformations, i.e., rotation and reflection, as the neighborhood configurations do belong to one of the invariant classes, which preserves the statistical consistency between the invariant neighborhood configurations and the coded pattern.

[0140] It should also be understood that during tree traversal, a child node may necessarily have some neighboring nodes at the same tree depth that have been previously visited and used as dependents. For these same-level neighbors, instead of referring to the parent's collocated neighbors, same-level neighbors may be used. Because same-level neighbors have half the dimensions of the parent, one configuration considers a neighbor to be occupied if any of its four directly adjacent neighboring child nodes (i.e., the four that share a face with the current node) are occupied.

[0141] (Entropy coding of tree occupancy patterns using binary coding) Some of the above-described techniques for using neighborhood occupancy information to code tree occupancy are detailed in European Patent Application No. 18305037.6. The described embodiments focus on the use of non-binary entropy coding of occupancy patterns, in which pattern distributions are selected based on neighborhood occupancy information. However, in some instances, using a binary coder may be more efficient from a hardware implementation perspective. Furthermore, on-the-fly updating of many probabilities may require fast-access memory and computations within the core of an arithmetic coder. Therefore, it may be advantageous to find a method and device for entropy encoding occupancy patterns using a binary arithmetic coder. If this can be done without significantly degrading compression performance and while avoiding having an overwhelming number of contexts to track, using a binary coder would be advantageous.

[0142] The use of a binary coder instead of a non-binary coder is reflected in the entropy formula below. H(X1,X2|Y)=H(X1|Y)H(X2|Y,X1)

[0143] where X = (X1, X2) is the non-binary information to be coded, and Y is the context for coding, i.e., the neighborhood configuration or selected pattern distribution. To convert the non-binary coding of X to binary coding, the information (X1, X2) is split into information X1 and X2, which can be coded separately without increasing entropy. To do this, one must code one of the two that depends on the other, where X2 depends on X1. This can be extended to n bits of information in X. For example, for n = 3, H(X1,X 2, X3|Y)=H(X1|Y)H(X2|Y,X1)H(X3|Y,X1,X2)

[0144] It should be understood that as the occupancy pattern, i.e., bit sequence X, becomes longer, more conditions exist for coding later bits in the sequence. For a binary coder (e.g., CABAC), this means a significant increase in the number of contexts to track and manage. Using an octree as an example, if the occupancy pattern is an 8-bit sequence b=b0···b7, the bit sequence may be divided into eight binary information bits b0···b7. The coding may use a neighborhood N (or N) to determine the context. As described above, assume that the neighborhood can be reduced to 10 valid neighborhoods through grouping the neighborhoods into invariant classes, where N is an integer belonging to {0, 1, 2,···, 9}. For brevity, a "class of invariant neighborhoods" may sometimes be simply referred to as a "neighborhood" herein, but it should be understood that this reduced number of neighborhoods may be realized based on a class-based grouping of neighborhoods based on invariance.

[0145] Figure 21 illustrates the division of an 8-bit pattern or sequence into eight individual bits for binary entropy coding. Note that the first bit of the sequence is encoded based on the neighborhood configuration; therefore, there are a total of 10 available contexts. The next bit of the sequence is encoded based on the neighborhood configuration and any previously encoded bit, namely, bit b0. This results in a total of 20 available contexts: obtained as the product of 10 from N and 2 from b0. The final bit, b7, is entropy encoded using a context selected from the 1,280 available contexts obtained as the product of 10 from N and 128 from the partial patterns given by previously encoded bits b0,...,b6. That is, for each bit, the number of contexts (i.e., possible combinations of conditions / subjects) is a function of the number of neighborhood configurations defined (10 in this example, based on the grouping of 64 neighborhood configurations into classes) and the number of possible partial patterns from the ordered sequence of n-1 previously encoded bits (2 n-1 (given by

[0146] As a result, there are a total of 2,550 contexts to maintain in relation to the binary coding of the occupancy patterns. This is an excessively large number of contexts to keep track of, and the relative shortage can cause poor performance due to context dilution, especially for later bits in the sequence.

[0147] Thus, in one aspect, this application discloses an encoder and decoder that determines whether a set of contexts can be reduced, and if so, applies a context reduction operation to achieve a smaller set of available contexts for entropy coding at least a portion of the occupancy pattern using a binary coder. In another aspect, this application further discloses an encoder and decoder that applies one or more steps of state reduction using the same context reduction operation to perform valid context selection from a fixed number of contexts. In some implementations, context reduction is applied a priori in generating a lookup table of contexts and / or algorithmic conditionals, which are then used by the encoder or decoder in selecting a suitable context. The reduction is based on testable conditions that the encoder and decoder evaluate to determine which lookup table to select or how to index / select from that lookup table to obtain the selected context.

[0148] Reference is now made to Figure 22, which illustrates, in flowchart form, an exemplary method 3000 for coding occupancy patterns in a tree-based point cloud coder using binary coding. The method 3000 may be implemented by an encoder or a decoder. In the case of an encoder, the coding operation is encoding, and in the case of a decoder, the coding operation is decoding. The encoding and decoding are context-based entropy encoding and decoding.

[0149] The exemplary method 3000 is for entropy coding an occupancy pattern, i.e., a bit sequence, for a particular node / volume. The occupancy pattern indicates the occupancy status of the node / volume's child nodes (subvolumes). In the case of an octree, there are eight child nodes / subvolumes. In operation 3002, a neighborhood is determined. The neighborhood is the occupancy status of one or more volumes located near the volume whose occupancy pattern is to be coded. As discussed above, there are various possible implementations for determining the neighborhood. In some examples, there are ten neighborhoods, and the neighborhood for the current volume is identified based on the occupancy of six volumes that share a surface with the current volume.

[0150] In operation 3004, an index i for the child node of the current volume is set to 0. Then, in operation 3006, an assessment is made as to whether context reduction is possible. Different possible context reduction operations are discussed in more detail below. The assessment of whether context reduction is possible may be based, for example, on the bits (e.g., index values) in the bit sequence being coded. In some cases, context reduction may be possible for later bits in the sequence but not for the first few bits. The assessment of whether context reduction is possible may be based, for example, on neighborhood configuration, since certain neighborhood configurations may allow for simplification. Additional factors may be used in assessing whether context reduction is possible in some implementations. For example, an upper bound Bo may be provided as the maximum number of contexts that a binary coder can use to code bits; if the initial number of contexts for coding bits is greater than Bo, context reduction is applied so that the number of contexts after reduction is at most Bo (otherwise, it is not applied). Such a bound Bo may be defined in the encoder and / or decoder specifications to ensure that a software or hardware implementation capable of handling Bo contexts will always be able to encode and / or decode the point cloud without overflowing in terms of the number of contexts. Knowing the bound Bo in advance also makes it possible to predict the complexity and memory footprint induced by a binary entropy coder, thus facilitating hardware design. Typical values ​​for Bo are in the tens to hundreds.

[0151] If it is determined that context reduction is available, then in operation 3008, a context reduction operation is applied. The context reduction operation reduces the number of available contexts in the set of available contexts to a smaller set containing fewer total contexts. Recall that the number of available contexts may depend in part on the bit position in the sequence, i.e., on the index, since the context may depend on partial patterns of previously coded bits from the bit sequence. In some implementations, the number of available contexts in the set before reduction may be based on the number of neighborhood configurations multiplied by the number of possible partial patterns with previously coded bits. For a bit at index i, where i ranges from 0 to n, the number of partial patterns is 2 i can be given by

[0152] As mentioned above, in some implementations, the context reduction operation is performed prior to coding, and the resulting reduced context set is the context set available for use by the encoder and decoder during the coding operation. The use and / or selection of the reduced context set during coding may be based on evaluation of one or more conditions prior to use of those reduced sets that correspond to the conditions evaluated in operation 3006 to determine that the number of contexts can be reduced. For example, in the case of a particular neighborhood configuration that enables use of a reduced context set, the encoder and / or decoder may first determine whether the neighborhood configuration condition is met and then, if applicable, use the corresponding reduced context set.

[0153] In operation 3010, bit b i The context for bit b is determined, i.e., iA context for the current bit is selected from the set of available contexts (or a reduced set, if present) based on the neighborhood configuration and the partial pattern of previously coded bits in the bit sequence. The current bit is then entropy encoded by a binary coder in operation 3012 using the selected context.

[0154] In operation 3014, the index i is determined to be equal to or less than i if the currently coded bit is the last bit in the sequence, i.e., i max If so, the coding process proceeds to the next node. Otherwise, the index i is incremented in operation 3016 and the process returns to operation 3006.

[0155] It should be understood that in some implementations, the context selection may not depend on the neighborhood configuration. In some cases, it may depend only on the partial pattern, if any, of previously coded bits in the sequence.

[0156] A simplified block diagram of a portion of an exemplary encoder 3100 is illustrated in Figure 23. In this illustration, it should be understood that the occupancy pattern 3102 is obtained as the corresponding volume is partitioned into child nodes and cycled through a FIFO buffer 3104 that holds the geometry of the point cloud. The coding of the occupancy pattern 3102 is illustrated as involving a cascade of binary coders 3106, one for each bit of the pattern. Between at least some of the binary coders 3106 are context reduction operations 3108 that operate to reduce the available contexts to a smaller set of available contexts.

[0157] 23 illustrates a series of binary coders 3106, in some implementations only one binary coder is used. If more than one coder is used, the coding may be (partially) parallelized. Given the contextual dependency of one bit on preceding bits in the bit sequence, the coding of a pattern cannot necessarily be fully parallelized, but it may be possible to improve pipelining through the use of cascading binary coders for certain patterns to achieve some degree of parallelism and speed improvement.

[0158] (Context determination based on subvolume neighborhood configuration) The current occupied bit b in the bit sequence is i A context for (i.e., a probability associated with each entropy coder that will be used for entropy coding) is determined (i.e., selected) from the set of available contexts (or a reduced set, if present) based on the neighborhood configuration and the partial pattern of previously coded bits in the bit sequence. The current bit is then entropy encoded by a binary coder using the selected context (probability). Additionally or alternatively, the occupancy status of a child node of a neighboring node of the current node (e.g., of an already coded occupied neighboring node) may be determined based on the occupancy status of bit b in the bit sequence of the current node. i The occupancy status of the child nodes of the neighboring nodes of the current node can be considered to determine the context (e.g., probability) for coding the current node. An implementation that considers the occupancy status of the child nodes of the neighboring nodes of the current node to determine the context will be described next.

[0159] In some implementations, the method for encoding or decoding a bit sequence may use knowledge of the occupancy status of the child nodes of the neighboring nodes of the current node to determine the context, i.e., knowledge of the occupancy of the subnodes of the already coded occupied neighboring nodes (possibly all neighboring nodes or all already coded neighboring nodes) may be used to determine the occupancy bit b of the current node. i (possibly reduced to 10 neighboring configurations N, e.g., N 10 It can be used to drive the

[0160] In the following, a method for encoding or decoding a bit sequence that uses knowledge of the occupancy status of the child nodes of the current node's neighbors to determine the context will be described with reference to an octree and a cube as the volume associated with a node of the octree, without intending to be limiting.

[0161] Reference is now made to FIG. 27, which illustrates a current node (i.e., its associated or current volume) 4000 and its six neighbors 4010, 4020, 4030, 4040, 4050, and 4060. With respect to this example of an octree, the neighbors of the current node may be defined as those nodes (at the same level or depth of the tree) whose associated volumes share faces with the current volume. Volumes that share faces with each other may also be referred to as volumes that touch each other. Other definitions of neighboring nodes are possible as well. For example, the neighbors of the current node may be those nodes (at the same level or depth of the tree) whose associated volumes share an edge (or vertex) with the current volume. In general, regardless of the tree structure, neighboring nodes may be those nodes (at the same level or depth of the tree) whose associated volumes intersect with the current volume (e.g., at least in some predetermined non-empty set).

[0162] It should be understood that in the context of this application, volumes (nodes) that intersect with each other are adjacent volumes (nodes). Thus, the terms "having an intersection with" and "adjacent to" can be considered synonymous in the context of this application.

[0163] It should be noted that the expressions "volume" and "subvolume" may be used somewhat synonymously, in the sense that each subvolume is a volume that may itself be subdivided into subvolumes. In either case, the volume / subvolume relationship is understood to be explicit through the definition of a parent-child relationship between the nodes / volumes involved.

[0164] The traversal order of the nodes is assumed to be performed breadth-first, in increasing X order, then in increasing Y order, and finally in increasing Z order. By doing so, the three neighbors with the lowest X coordinate (i.e., neighbor 4010), the lowest Y coordinate (i.e., neighbor 4030), and the lowest Z coordinate (i.e., neighbor 4050) have already been coded. Thus, if one of these three neighbors is occupied, the configuration of the occupied subvolume associated with the occupied neighbor is known. While this example defines a traversal order in increasing X order, then in increasing Y order, and finally in increasing Z order, other breadth-first traversal orders can also be used for this purpose. Regardless of the particular breadth-first traversal order used, three neighbors will already be coded for the current node that is not located on the boundary of 3D space.

[0165] Reference is now made to Figure 28, which illustrates an exemplary current volume in which all three previously coded neighbors (i.e., neighbors 4010, 4030, and 4050) are occupied. The occupied subvolumes of neighbor 4010 are subvolumes 4011, 4012, and 4013, the occupied subvolumes of neighbor 4030 are subvolumes 4031, 4032, and 4033, and the occupied subvolumes of neighbor 4050 are subvolumes 4051 and 4052. While in this example all three previously coded neighbors are occupied, it should be understood that in general only two, one, or even none of them may actually be occupied.

[0166] Knowledge of the occupied subvolumes of the already coded occupancy neighborhood can be used to define a set of subvolume neighborhood configurations (e.g., one for each subvolume of the current volume). A subvolume neighborhood configuration is a set of subvolume neighborhood configurations that are determined by the bit b i The subvolume neighborhood configuration for a given subvolume of the current volume may be based on occupancy data for subvolumes of those neighboring volumes of the current volume that have already been coded. Furthermore, the subvolume neighborhood configuration for a given subvolume in the current volume may correspond to the pattern of occupancy of all those subvolumes in neighboring (already coded) volumes of the given volume that are located in the vicinity of the given subvolume.

[0167] Reference is now made to FIG. 29, which illustrates in flowchart form an example of a method 4100 for encoding a point cloud to generate a bitstream of compressed point cloud data. The point cloud is defined in a tree structure having multiple nodes with parent-child relationships, where the multiple nodes represent the geometry of a volume space, which is recursively divided into sub-volumes. The occupancy of the sub-volumes of the volume is indicated using a bit sequence, where each bit of the bit sequence indicates the occupancy of a respective sub-volume in traversal order within the volume, and the volume has multiple neighboring volumes. Each of the operations of method 4100 described below is performed with respect to a current node associated with a current volume divided into sub-volumes, where each sub-volume corresponds to a child node of the current node. In operation 4110, a bit sequence indicating the occupancy of the sub-volumes of the current volume is determined. Then, with respect to at least one bit in the bit sequence of the current volume, the following operations are performed: In operation 4120, a sub-volume neighborhood configuration is determined based on occupancy data for the sub-volumes of at least one neighboring volume of the current volume. The subvolume neighborhood configuration depends on the occupation pattern of a group of subvolumes of at least one neighboring volume that are located in the neighborhood of a given subvolume of the current volume. A given subvolume of the current volume is that subvolume that corresponds to a bit in the bit sequence. For example, bit b of the bit sequence i , a given subvolume is i is that subvolume of the current volume corresponding to bit b iis that subvolume indicating occupancy status. A subvolume neighborhood configuration is then determined for the considered subvolume of the current volume. Notably, operation 4120 may be performed for any, some, or all of the bits of the bit sequence. In operation 4130, probabilities (e.g., contexts) for use in entropy encoding of bits in the bit sequence are selected, where the selecting is based at least in part on the subvolume neighborhood configurations that have been determined for subvolumes corresponding to the bits in the bit sequence. In operation 4140, the bits in the bit sequence are entropy encoded based on the selected probabilities (e.g., contexts) using a binary entropy encoder to generate encoded data for a bitstream.

[0168] In some implementations, the method 4100 may further include an act of updating the selected probability (eg, context) (not shown in FIG. 29).

[0169] As described above, the occupancy pattern of the current node may be entropy coded using a cascade of one or more binary entropy coders. Thus, operation 4130 of method 4100 may involve, for at least one bit of a bit sequence, selecting a respective probability (and correspondingly, an associated entropy coder) for coding that bit based, at least in part, on a subvolume neighborhood configuration that has been determined for a subvolume corresponding to the bit to be entropy coded. Additionally, selecting this probability may be based on a partial sequence of already coded bits of the bit sequence and / or a neighborhood configuration for the current volume. In other words, for each bit of the bit sequence, a context may be selected based on a subvolume neighborhood configuration, and additionally, selecting the context may be based on a partial sequence of already coded bits of the bit sequence and / or a neighborhood configuration for the current volume. From a context perspective, operation 4130 of method 4100 may be considered to relate to selecting a context for entropy coding the bit based, at least in part, on a subvolume neighborhood configuration that has been determined for a subvolume corresponding to the bit to be coded. In some implementations, this context may then be updated after operation 4140.

[0170] Reference is now made to FIG. 30 , which illustrates in flowchart form an example of a method 4200 for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud. The point cloud is defined in a tree structure (e.g., an octree) having multiple nodes with parent-child relationships, where the multiple nodes represent the geometry of a volume space, which is recursively divided into sub-volumes containing the points of the point cloud. The occupancy of the sub-volumes of the volume is indicated using a bit sequence, where each bit of the bit sequence indicates the occupancy of a respective sub-volume in traversal order within the volume, and the volume has multiple neighboring volumes. Each of the operations of method 4200 described below is performed with respect to a current node associated with a current volume divided into sub-volumes, each sub-volume corresponding to a child node of the current node, and with respect to at least one bit in the bit sequence of the current volume. In operation 4210, a sub-volume neighborhood configuration is determined based on occupancy data for the sub-volumes of at least one neighboring volume of the current volume. The subvolume neighborhood configuration depends on the occupation pattern of a group of subvolumes of at least one neighboring volume that are located in the neighborhood of a given subvolume of the current volume, where the given subvolume is that subvolume of the current volume that corresponds to a bit in the bit sequence. For example, bit b of the bit sequence i , a given subvolume is i is that subvolume of the current volume corresponding to bit b iis the subvolume under consideration indicating its occupancy status. A subvolume neighborhood configuration is then determined for the considered subvolume of the current volume. Notably, operation 4120 may be performed for any, some, or all of the bits of the bit sequence. In operation 4220, a probability (e.g., context) for use in entropy decoding of a bit in the bit sequence is selected. This selection is based at least in part on the subvolume neighborhood configuration. In operation 4230, the at least one bit is entropy decoded based on the selected probability (e.g., context) using a binary entropy decoder to generate a reconstructed bit from the bitstream.

[0171] In some implementations, the method 4200 may further include an act of updating the selected probability (eg, context) (not shown in FIG. 30).

[0172] In the same manner as for encoding, operation 4220 of method 4200 may involve selecting, for at least one bit of a bit sequence representing an occupation pattern, a respective probability (and correspondingly, an associated entropy coder) for coding that bit based, at least in part, on a subvolume neighborhood configuration that has been determined for a subvolume corresponding to the bit to be entropy coded. In addition, selecting this probability may be based on a partial sequence of already coded bits in the bit sequence and / or a neighborhood configuration for the current volume. In other words, for each bit of the bit sequence, a context may be selected based on a subvolume neighborhood configuration, and selecting the context may be based on a partial sequence of already coded bits in the bit sequence and / or a neighborhood configuration for the current volume. In terms of context, operation 4220 of method 4200 may be considered to relate to selecting a context for entropy coding the bit based, at least in part, on a subvolume neighborhood configuration that has been determined for a subvolume corresponding to the bit to be coded. Then, in some implementations, this context may be updated after operation 4230.

[0173] An example of determining a sub-volume neighborhood will be described herein below.

[0174] In some implementations, this may involve determining, based on occupancy data for subvolumes of at least one neighboring volume of the current volume, the number of subvolumes (possibly of all neighboring volumes or all already coded neighboring volumes) located near the subvolume of the current volume corresponding to the bit in the bit sequence. This number is referred to below as NT[i], where i is the number of subvolumes of the current volume (or the corresponding bit b in the bit sequence). i). In this sense, selecting a probability in operation 4130 of method 4100 and operation 4220 of method 4200 may be thought of as selecting a probability based at least in part on number NT[i].

[0175] In some implementations, determining the sub-volume neighborhood may further involve applying a threshold function to the determined number NT[i].

[0176] Reference is made to Figure 31, which illustrates an example of the division of the current volume (current node) B into eight subvolumes (subnodes) SB0-SB7. For this example, neighboring volumes are defined as volumes that share surfaces with each other (i.e., "touch" each other). Each of the subvolumes (or child volumes) of the current volume B is located in the neighborhood of (touched by) three subvolumes in the neighborhood of the current volume. The subvolume neighborhood configuration for each subvolume SB0-SB7 (SB1, i = 0,...,7) may depend on the occupation pattern of the group of subvolumes touching each subvolume SB1 of the current volume. Depending on the occupation pattern of the occupation neighborhood of the current volume, up to three occupied subvolumes of the occupation neighborhood of the current volume may be located in the neighborhood of (or touching) each subvolume SB1, where touching / neighborhood is defined as sharing a surface.

[0177] One exemplary process for determining the subvolume neighborhood configuration for a given subvolume SBi of the current volume using the above definition of a neighborhood volume involves determining the number of subvolumes in the neighborhood of the current volume that touch each subvolume SBi, 0≦NT[i]≦3. In other words, the number NT[i] for a subvolume SBi of the current volume indicates the number of occupied subvolumes in the neighborhood of the current volume that are located near (touching, i.e., sharing a surface with) the subvolume SBi.

[0178] By the above definition, the integer 0≦NT[i]≦3 for the subvolume SBi of the current volume is the number of occupied subvolumes (child nodes) located within the (already coded) neighborhood of the current volume and intersecting the subvolume SBi in a plane (i.e. touching the subvolume SBi).

[0179] In a preferred implementation, the neighbors are the six neighbors that share a face with the current cube, and a child node of an occupying neighbor is considered to be located neighbors of a subnode SBi if and only if one of its faces is contained within the face of the cube associated with the subnode SBi (i.e., if the child node of the occupying neighbor intersects with the subnode SBi at a face). Nevertheless, it should be understood that the present application is not limited to the above-described implementation of an octree and the definition of a neighbor as a node whose associated volume shares a face with the associated volume of the current same-level node. In fact, the present application relates to any tree representing the geometry of a point cloud. Furthermore, the neighbors of a current node may generally be defined as nodes that are at the same depth or level as the current node (relative to the root node of the tree) and whose associated volume intersects with the volume of the current node (e.g., at least in a predetermined non-empty set). In a tree (e.g., an octree, etc.) in which the volume associated with a node is a cube, this intersection may be any non-empty set of faces, edges, vertices, or points.

[0180] Similarly, a nearby occupying child node of the current node is considered to intersect with a subnode SB of the current node B if the intersection of the volume associated with the child node and the volume associated with the subnode SB is a non-empty set of points. Again, in a tree (e.g., an octree) where the volume associated with the node is a cube, this intersection can be any non-empty set of faces, edges, vertices, or points.

[0181] It should also be understood that the neighbor definition for a neighbor and for a child node of a neighbor may differ. For example, in an octree, a neighbor may be defined as a node that shares a face with the current node, thus leading to six neighbors; however, the number NT[i] may count the number of child nodes of an already coded occupied neighbor that share at least an edge with the subnode SBi. Nevertheless, a preferred implementation may use the definition of intersection as "shares a face with" for both the neighbor and the child node of the neighbor.

[0182] As mentioned above, different definitions of neighboring nodes or volumes may be used in the context of this application. Depending on the neighborhood definition, the possible range for the number NT[i] will vary from 0≦NT[i]≦3 (which applies when neighboring volumes are defined as volumes that share a face or are "touching"). For example, if neighboring volumes are defined as volumes that intersect each other at least at an edge, we will have 0≦NT[i]≦12. Similarly, if neighboring volumes are defined as volumes that intersect each other at least at a vertex, we will have 0≦T[i]≦19.

[0183] As described above, depending on the occupation pattern of the current volume's occupation neighborhood, up to three occupied subvolumes of the current volume's occupation neighborhood may be located in the neighborhood of each subvolume (subnode) SBi, using a neighborhood definition in which neighboring volumes are volumes that share a plane. Figure 32 shows that for subnode SB0 of current node B, assuming the traversal order described above with reference to Figure 27, there are up to three occupied child nodes of the neighboring nodes that may be located in the neighborhood of the subvolume associated with subnode SB0 (i.e., may intersect with the subvolume associated with subnode SB0 in the plane). Eight possible occupied child node configurations and associated values ​​NT[0] that are either 0, 1, 2, or 3 are depicted in the figure.

[0184] 33 shows that for subnode SB1 of current node B, there are up to two occupying child nodes of neighboring nodes that can be located in the vicinity of (i.e., intersect in a plane with) the sub-volume associated with subnode SB1. Four possible occupying child node configurations and associated values ​​NT[1] that are either 0, 1, or 2 are depicted in the figure.

[0185] Figure 34 shows that for subnode SB3 of current node B, there is at most one occupied child node of a neighboring node that can be located in the vicinity of (i.e., intersect in a plane with) the sub-volume associated with subnode SB3. Two possible occupied child node configurations and associated values ​​NT[3] that are either 0 or 1 are depicted in the figure.

[0186] Figure 35 shows that for subnode SB7 of the current node B, there are no occupying child nodes of neighboring nodes that may be located in the vicinity of (i.e., intersect in a plane with) the sub-volume associated with subnode SB7. Therefore, there is only one possible occupying child node configuration as depicted in the figure, i.e., the value of NT[7] is necessarily 0. In this case, the child nodes do not provide any special information for context determination in coding the occupancy information of subnode SB7.

[0187] Once the number NT[i] is determined for the subnodes SBi of the current node, the subvolume neighborhood configuration value C[i] can be determined from NT[i] by applying a threshold function. The threshold function can be a function that outputs its input values ​​up to a certain threshold and outputs a threshold for input values ​​above the threshold. For example, the threshold can be set to 2, thus If NT(i)=0, then C[i]=0 If NT(i)=1, then C[i]=1 If NT(i)>=2, then C[i]=2 In this thresholding operation, possible values ​​for C[0] include 0, 1, or 2, while C[7] is always 0. It should be understood that other thresholding operations and threshold functions are possible as well. In particular, for other neighborhood definitions, the value range for the numbers NT[i] may be different from 0≦NT[i]≦3, and different thresholds may be applied.

[0188] Then, selecting a probability (e.g., context) in operation 4130 of method 4100 and operation 4220 of method 4200 may involve selecting a probability based at least in part on the sub-volume neighborhood configuration value C[i]. In this sense, the sub-volume neighborhood configuration value C[i] may be considered to correspond to a sub-volume neighborhood configuration.

[0189] Reference is now made to Figure 36, which illustrates in flowchart form one example of a method 4300 for determining numbers NT[i] and subvolume neighborhood configuration values ​​C[i] for subvolumes of the current volume. That is, method 4300 operates on subvolumes of the current volume. In operation 4310, the numbers NT[i] are initialized to zero for all subvolumes of the current volume. Then, in operation 4320, a neighborhood of the current volume is selected. In operation 4330, it is checked whether the selected neighborhood is occupied. If the selected neighborhood is occupied ("yes" in operation 4330), the method proceeds to operation 4340. In operation 4340, it is checked whether the selected occupied neighborhood has already been coded. If so ("yes" in operation 4340), the method proceeds to operation 4350. In operation 4350, for each occupied child node (subvolume) in the neighborhood, the subvolumes of the current volume that are located in the neighborhood of that child node (e.g., intersected in at least a predetermined non-empty set, depending on the neighborhood definition) are determined. For each subvolume of the current volume that is located in the neighborhood of the child node, the corresponding number NT[i] is incremented by one. The method then proceeds to operation 4360. If the selected neighborhood is unoccupied ("No" in operation 4330) or has not yet been coded ("No" in operation 4340), the method also proceeds to operation 4360. In operation 4360, it is checked whether there are any neighbors of the current volume that have not yet been selected. If so ("Yes" in operation 4360), the method returns to operation 4320 to select the next neighbor of the current volume. Once all neighbors have been processed ("No" in operation 4360), a subvolume neighborhood configuration value C[i] is calculated for each subvolume of the current volume in operation 4370. This may involve applying a threshold function. Then, in operation 4380, each bit b of the bit sequence is imay be entropy coded using probabilities (e.g., context) based on the sub-volume neighborhood configuration values ​​C[i], and possibly based on partial sequences of already coded bits within the bit sequence and / or (reduced) neighborhood configurations.

[0190] In an alternative implementation, instead of a loop over the neighborhood of the current volume, a loop over the sub-volumes SBi of the current volume (or equivalently over the index i) can be performed. Then, for each sub-volume SBi, the already coded neighboring occupied child nodes are tested as to whether they are located in the neighborhood of the sub-volume SBi (e.g., intersect in at least a predetermined non-empty set), and NT[i] is obtained as the number of such child nodes located in the neighborhood of the sub-volume SBi.

[0191] In the above determination of NT'[i], all neighbors of a subvolume of the current volume are treated equally, regardless of their intersection with the subvolume. In some implementations, different weights may be applied to neighbors with different intersections with the subvolume. For example, applying a broader neighborhood definition, neighboring subvolumes that share faces with the subvolume may have a higher weight than neighboring subvolumes that share edges (only) with the subvolume, which in turn may have a higher weight than neighboring subvolumes that share vertices (only) with the subvolume. These different weights may then be applied when determining the weighted number NT'[i], with each neighbor of the subvolume being counted with its respective weight in the sum leading to the weighted number NT'[i].

[0192] Thus, determining the subvolume neighborhood configuration in operation 4120 of method 4100 and operation 4210 of method 4200 may involve determining, based on occupancy data for the subvolumes of at least one neighboring volume of the current volume, all subvolumes of at least one neighboring volume (possibly of all neighboring volumes or all already-coded neighboring volumes) that intersect with the subvolume of the current volume that corresponds to the bit in the bit sequence. Further, respective weighting factors may be applied to the determined subvolumes. Each weighting factor may depend on the intersection of each determined subvolume with the subvolume of the current volume that corresponds to the bit in the bit sequence. For example, a subvolume that intersects with the subvolume at a face may have the highest weight, a subvolume that intersects with the subvolume at (only) an edge may have a medium weight, and a subvolume that intersects with the subvolume at (only) a vertex may have the lowest weight. In some implementations, only two of the three neighborhood definitions may be considered (which effectively corresponds to setting the weighting factor for the neighbor of the third neighborhood definition to zero). For example, the respective weighting factors applied to faces, edges, and sub-volumes intersecting the sub-volume at vertices may be set to 2, 1, and 0, respectively.

[0193] Then, determining the subvolume neighborhood may further involve determining, based on the determined subvolumes and their respective weighting factors, a weighted number NT'[i] of subvolumes of at least one neighboring volume that intersect with the subvolume of the current volume corresponding to the bit in the bit sequence. That is, when summing over the determined subvolumes of the at least one neighboring volume that intersect with the subvolume of the current volume corresponding to the bit in the bit sequence, each determined subvolume may be counted with its respective weighting factor. Further, thresholding may be applied to the weighted number NT'[i] to obtain a weighted subvolume neighborhood value C'[i].

[0194] In some implementations, a given occupied bit b in a bit sequence i The probabilities (eg, contexts) associated with a binary entropy coder that codes {right arrow over (x)} may be chosen depending on the following values: ·b0,···,b i-1 N 10 C[i] That is, once determined, the subvolume neighborhood configuration value C[i] (or the weighted subvolume neighborhood configuration value C'[i]) is the occupied bit b associated with the subnode SBi of the current node B. i can be used as input to an entropy coder that codes

[0195] (Context reduction operation) The above example suggests that the coding process includes a context reduction operation on at least one bit of the occupancy pattern, so as to reduce the set of available contexts to a smaller set of available contexts. In this sense, a "context reduction operation" refers to the reduction of a particular bit b i This can be understood as identifying and consolidating contexts that may be considered overlapping or redundant in certain situations. As described above, the reduced context set may be determined prior to coding and provided to the encoder and decoder, which decide whether to use the reduced context set based on the same conditions described below for reducing the context set.

[0196] (Neighborhood formation reduction through screening / shielding) A first exemplary context reduction operation involves reducing the number of neighboring configurations based on screening / shielding. In principle, neighboring configuration incorporates the occupancy status of neighboring volumes into the context selection process, based on the neighboring volumes' usefulness in indicating whether the current volume or subvolume is likely occupied. As bits associated with subvolumes within the current volume are decoded, they are also incorporated into the context selection; however, information from nearby subvolumes may be more significant and useful than occupancy information for neighboring volumes located on the other side of the subvolume from the current subvolume. In this sense, previously decoded bits are associated with subvolumes that "screen" or "shield" the neighboring volumes. This may mean that in such situations, the occupancy of a neighboring volume can be ignored, since the relevance of its occupancy status is encompassed by the occupancy status of the subvolumes between the current subvolume and the neighboring volume, thereby enabling a reduction in the number of neighboring configurations.

[0197] Reference is now made to Figure 24, which diagrammatically illustrates an exemplary context reduction operation based on neighborhood screening. The example involves coding an occupancy pattern for a volume 3200. The occupancy pattern indicates the occupancy status of eight sub-volumes within the volume 3200. In this example, four sub-volumes in the upper half of the volume 3200 have been coded, and therefore their occupancy status is known. The bit of the occupancy pattern being coded is associated with a fifth sub-volume 3204, located in the lower half of the volume 3200, below the four previously coded sub-volumes.

[0198] Coding in this example involves determining a context based on neighborhood configurations. Ten neighborhood configurations 3202 are shown. Volume 3200, which includes a fifth sub-volume 3204 to be coded, is shown in light gray and designated by reference numeral 3200. Neighborhood configuration 3202 is based on the occupancy status of volumes adjacent to and sharing a surface with volume 3200. The neighborhood volumes include a top neighboring volume 3206.

[0199] In this example, the number of neighborhood configurations can be reduced from 10 to 7 by ignoring the top neighborhood volume 3206 in at least some of the configurations. As shown in FIG. 24 , three of the four configurations in which the top neighborhood volume 3206 is shown are subsumed under equivalent configurations that do not incorporate the top neighborhood volume 3206, thereby reducing the number of neighborhood configurations to a total of seven. It may still be advantageous to keep the configurations that show all six neighboring volumes, because there is no existing five-volume neighborhood configuration into which the six-volume configuration could be consolidated (eliminating the five-element one), which means that even if the top neighborhood volume were removed, a new five-element neighborhood configuration would result, and no overall reduction in context would occur.

[0200] The upper neighboring volume 3206 can be excluded from the neighborhood configuration in this example because the context decision for coding the occupancy bit associated with the fifth subvolume 3204 already takes into account the occupancy status of the four previously coded subvolumes directly above it, which is a better indication of the likelihood and direction of occupancy for the fifth subvolume than the occupancy status of the more distant upper neighboring volume 3206.

[0201] The above example in which the upper neighboring volume 3206 is screened or shielded by a previously coded subvolume when coding the occupancy bits corresponding to the fifth subvolume 3204 is only one example. Depending on the coding order within the volume 3200, several other possible screening / shielding situations may be realized and exploited to reduce the available neighborhood configurations.

[0202] Reference is now made to FIG. 25, which illustrates a second example of screening / shielding. In this example, the occupancy pattern for volume 3200 has been almost completely coded. The subvolume to be coded is the eighth subvolume, hidden (invisible) in the rear bottom corner of the figure. In this case, the occupancy status of all seven other subvolumes has been coded, particularly the subvolumes along the top (thus reducing the neighborhood to a total of seven) and along the right and front sides. Thus, in addition to screening the top neighboring volume, the subvolumes with previously coded occupancy bits shield the front neighboring volume 3210 and the right neighboring volume 3212. This may allow the neighborhood to be reduced from a total of seven to a total of five, as shown.

[0203] It should be understood that the two foregoing examples of shielding are illustrative, and that in some cases, different configurations may be merged to take into account different shielding situations. It should be understood that the context reduction operation based on shielding / screening by a previously coded sub-volume is general and not limited to these two examples, but cannot be applied in the case of the first sub-volume to be coded, because it requires that there be at least one previously coded occupied bit associated with a neighboring sub-volume for there to be any shielding / screening.

[0204] It should also be understood that the degree of shielding / screening required to justify neighborhood reduction may vary in different implementations. In the two above examples, all four subvolumes that share a surface with a neighboring volume were previously coded before that neighboring volume was considered shielded / screened, and therefore were removed from the neighborhood. In other examples, partial shielding / screening may be sufficient, for example, from one to three previously coded subvolumes that share a surface.

[0205] (Context reduction through special case handling) There are certain cases where context reduction can be performed without loss of useful information. In the exemplary context determination process described above, the context for coding the occupancy bits is based on neighborhood configurations, i.e., the pattern of occupancy of volumes located in the neighborhood of the current volume, and the partial patterns resulting from the occupancy of sub-volumes within the current volume that have been previously coded. The latter condition is due to the fact that for the eighth bit in the occupancy pattern bit sequence, the second bit to be tracked is 7 = 128 contexts. Even if the neighborhood configurations are reduced to a total of 5, that still means 640 contexts to track.

[0206] The number of contexts is large due to the fact that previously coded bits of a bit sequence have a certain order, and that order is relevant in assessing the context. However, in some cases, the order may not contain useful information. For example, if the neighborhood structure is empty, i.e., N 10If , ... j This means that the condition can be reduced to j+1 conditions, i.e., the context for coding one of the bits of a bit sequence is based on previously coded bits, but only on their sum, not their ordered pattern. In other words, the entropy representation in this special case can be expressed as follows: H(b|n)≒H(b0|0)H(b1|0,b0)H(b2|0,b0+b1)...H(b7|0,b0+b1+...+b6)

[0207] In some implementations, a similar observation may be made regarding a full neighborhood configuration. In some examples, a full neighborhood configuration lacks directionality, meaning that the order of previously coded bits need not be considered in determining the context. In some examples, this context reduction operation may be applied to only some of the bits in a bit sequence, such as some of the later bits in the sequence. In some cases, application of this context reduction operation to later bits may be conditional on determining that earlier bits associated with previously coded sub-volumes are also fully occupied.

[0208] (Statistical-based context reduction) Statistical analysis may be used to reduce contexts by determining those that lead to approximately the same statistical behavior and then combining them. This analysis may be performed a priori using test data to develop a reduced context set, which is then provided to both the encoder and decoder. In some cases, analysis may be performed on the current point cloud using two-pass coding to develop a custom reduced context set for the specific point cloud data. In some such cases, the mapping from the unreduced context set to the custom reduced context set may be indicated to the decoder by using dedicated syntax coded into the bitstream.

[0209] Two contexts can be compared through the concept of "distance". A first context c has a probability p that bit b is equal to zero, and a second context c' has a probability p' that bit b' is equal to zero. The distance between c and c' is given by: d(c,c')=|plog2p-p'log2p'|+|(1-p)log2(1-p)-(1-p')log2(1-p')|

[0210] Using this measure of similarity (distance), the contexts can then be grouped in a process such as:

[0211] 1. Start with the M1 context and fix the threshold level ε.

[0212] 2. For a given context, regroup into a class all contexts that have a distance from the given context that is less than a threshold level ε.

[0213] 3. Repeat step 2 for all unregrouped contexts until all are populated into a class.

[0214] 4. Label the M2 classes of 1~M2: This results in a brute-force reduction function that maps {1, 2, ···, M1] → [1, 2, ···, M2], where M1 ≥ M2.

[0215] The brute-force reduction function for mapping a set of contexts to a smaller set of contexts may be stored in memory to be applied by the encoder / decoder as a context reduction operation during coding. The mapping may be stored as a lookup table or other data structure. The brute-force reduction function may, for example, be applied only for later bits in a bit sequence (pattern).

[0216] (Combinations and subcombinations of context reduction actions) Three exemplary context reduction operations are described above. Each of them may be applied individually and independently in some implementations. Any two or more of them may be combined in some implementations. Additional context reduction operations may be implemented alone or in combination with any one or more of the context reduction operations described above.

[0217] 26 illustrates in flowchart form an example of a method 3300 for occupancy pattern binary coding with combined context reduction. The method 3300 is based on a 10-element neighborhood configuration N 10 Given N, we code the 8-bit binary pattern b0, b1, . . . , b7. The first condition evaluated is whether the neighborhood is empty, i.e., N 10 =0. If so, the bits are coded without reference to their order, as indicated by reference numeral 3302. If not, the bits are coded normally up to bit b4, at which point the encoder and decoder apply the brute force context reduction function BR ito reduce the number of contexts by mapping the set of contexts defined by neighborhood configurations and partial patterns of previously coded bits into a smaller set of contexts with substantially similar statistical performance.

[0218] In this example, the last two bits b6 and b7 are coded using a reduced neighborhood structure based on shielding / screening.

[0219] All functions can be implemented as look-up tables (LUTs) to reduce the size of the set of contexts. In one practical implementation, all reductions are simply incorporated into reduction functions, i.e., LUTs, that take contexts as input and provide reduced contexts as output. In this exemplary embodiment, the total number of contexts is reduced from 2,550 to 576, and each reduction function BR i The output sizes of are 70, 106, 110, and 119, respectively.

[0220] In summary, the following conditions (or conditions) for conditional entropy are H(b i |b0,...,b i-1 ,N 10 ) can be reduced by using a reduction function BRi to obtain a conditional entropy relation with fewer dependencies via: H(b i |BR i [b0, ,b i-1 ,N 10 ]) As explained above, a subvolume neighborhood configuration (e.g., a subvolume neighborhood configuration value C[i] or a weighted subvolume neighborhood configuration value C'[i]) is calculated by dividing the occupancy bits b i In this case, the conditional entropy relation is: H(b i |b0,...,bi-1 ,N 10 ,C[i]) The reduction is performed in the same way as above, and the following can be obtained: H(b i |BR i [b0, ,b i-1 ,N 10 ],C[i])

[0221] Above, the reduction function BR i BR i [b0, ,b i-1 ,N 10 ] to obtain the partial bit sequence b0,...,b i-1 and 10-element neighborhood configuration N 10 In some implementations, the reduction function BR i In addition, for example, BR i [b0, ,b i-1 ,N 10 , C[i]] to the subvolume neighborhood configuration (e.g., to the subvolume neighborhood configuration values ​​C[i] or the weighted subvolume neighborhood configuration values ​​C'[i]) to obtain the following conditional entropy relationship: H(b i |BR i [b0, ,b i-1 ,N 10 ,C[i]])

[0222] In this case, the occupied bit b i The choice of probability p (e.g., context) for the binary entropy coder to code is determined by (BR i [b0, ,b i-1 ,N 10 ], C[i]).

[0223] The subvolume neighborhood configuration is the occupancy bit b i, ... 10 and the subvolume neighborhood configuration value C[i] (or the weighted subvolume neighborhood configuration value C'[i]), we code an 8-bit binary pattern b0, b1,..., b7. The first condition evaluated is whether the neighborhood is empty, i.e., N 10 =0. If so, the bits are coded without reference to their order, as indicated by reference numeral 4402. If not, the bits are coded normally up to bit b4, at which point the encoder and decoder apply the brute force context reduction function BR i to reduce the number of contexts by mapping the set of contexts defined by neighborhood configurations and partial patterns of previously coded bits into a smaller set of contexts with substantially similar statistical performance.

[0224] In this example, the last two bits b6 and b7 are coded using a reduced neighborhood structure based on shielding / screening.

[0225] All functions can be implemented as look-up tables (LUTs) to reduce the size of the set of contexts. In one practical implementation, all reductions are simply incorporated into a reduction function, i.e., a LUT, that takes a context as input and provides a reduced context as output.

[0226] (Selection of traversal order within the current volume) Introducing subvolume neighborhood configurations (eg, subvolume neighborhood configuration values ​​C[i] or weighted subvolume neighborhood configuration values ​​C'[i]) in the entropy coder increases the possible configurations for constrained coding.

[0227] However, this increase can be limited by carefully choosing the occupation pattern and therefore the order of the scan order of the subnodes SBi in the current volume B to determine the bit sequence to be coded. i The coding of depends on: 2 i The bits b0-b that lead to this configuration i-1 10 Neighborhoods N 10 Subvolume neighborhood configuration value C[i] (or weighted subvolume neighborhood configuration value C'[i]) For subvolumes SB0-SB7, 10, 20, 39, 76, 149, 294, 391, and 520 configurations were included, respectively, with 10·2 from the first two bullets. i A method for reducing this configuration is shown above.

[0228] Here, it is clearly advantageous to first scan those subnodes where the number of possible values ​​of C[i] is the largest (e.g., 3 in some of the above examples). By doing so, the number of configurations for the first scanned subnode becomes 10 3 = 30, which can be considered a slight reduction. However, if subnode SB0 were scanned last, 520 3 = 1,560 configurations would be obtained. Therefore, by carefully choosing the scanning order within the current volume (to determine the occupancy pattern and thereby the bit sequence to be coded), the introduction of subvolume neighborhood configurations (e.g., subvolume neighborhood configuration values ​​C[i] or weighted subvolume neighborhood configuration values ​​C'[i]) in the entropy coder has a limited impact on the number of configurations (contexts) in entropy coding.

[0229] Therefore, in some implementations, the scan order within the current volume may be determined such that the maximum possible number of neighboring subvolumes in the already coded neighboring volumes of the current volume does not increase from one subvolume to the next in the scan order. In other words, the scan order may be determined such that the maximum possible value of the number NT[i] (or weighted number NT′[i]) does not increase from one subvolume to the next in the scan order.

[0230] (Flag indicating use of subvolume neighborhood configuration) To enable a low complexity profile of the coder, it may be advantageous to code a flag indicating the activation / deactivation of considering sub-volume neighborhood configurations in determining probabilities (e.g., context) in entropy coding bit sequences.

[0231] Thus, in some implementations, the encoding method may further include encoding a flag indicating that the probability for use in entropy encoding of the at least one bit is selected at least in part based on the sub-volume neighborhood configuration. Similarly, the decoding method may further include decoding a flag from the bitstream, the flag indicating that the probability for use in entropy decoding of the at least one bit should be selected at least in part based on the sub-volume neighborhood configuration.

[0232] (Impact on compression performance) Each of the above-described context reduction operations can be further used in compression systems with a static (fixed) minimum number of contexts. In such designs, for a given symbol in an 8-bit binary pattern, one or more reduction operations are applied to determine a context probability model for encoding or decoding the symbol.

[0233] (Impact on compression performance) The use of 10 neighbors and non-binary coding provides compression gains over the current implementation of the MPEG test model for point cloud coding. However, the use of 10 neighbors with cascaded binary coding using 2,550 contexts proposed above results in even greater improvements in compression efficiency. Even when context reduction is used, such as using the three techniques detailed above to reduce the contexts to a total of 576, binary coding compression is still slightly better than the implementation using non-binary coding and much better than the test model. This observation has been shown to be consistent across different test point cloud data.

[0234] Reference is now made to FIG. 14 , which illustrates a simplified block diagram of an exemplary embodiment of an encoder 1100. The encoder 1100 includes a processor 1102, a memory 1104, and an encoding application 1106. The encoding application 1106 may include a computer program or application stored in the memory 1104 and including instructions that, when executed, cause the processor 1102 to perform operations such as those described herein. For example, the encoding application 1106 may encode and output a bitstream encoded according to the processes described herein. It should be understood that the encoding application 1106 may be stored on a non-transitory computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard drive, or the like. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions to operate as a dedicated processor implementing the described processes. Such a processor may, in some examples, be referred to as a “processor circuit” or “processor circuitry.”

[0235] Reference is now also made to FIG. 15 , which illustrates a simplified block diagram of an exemplary embodiment of a decoder 1200. The decoder 1200 includes a processor 1202, a memory 1204, and a decoding application 1206. The decoding application 1206 may include a computer program or application stored in the memory 1204 and including instructions that, when executed, cause the processor 1202 to perform operations such as those described herein. It should be understood that the decoding application 1206 may be stored on a computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard drive, or the like. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions, such that it operates as a dedicated processor implementing the described processes. Such a processor may, in some examples, be referred to as a “processor circuit” or “processor circuitry.”

[0236] It should be understood that a decoder and / or encoder according to the present application may be implemented in a number of computing devices, including, but not limited to, a server, a suitably programmed general-purpose computer, a machine vision system, and a mobile device. The decoder or encoder may be implemented using software that includes instructions for configuring a processor or multiple processors to perform the functions described herein. The software instructions may be stored on any suitable non-transitory computer-readable memory, including a CD, RAM, ROM, flash memory, etc.

[0237] It should be understood that the modules, routines, processes, threads, or other software components implementing the decoders and / or encoders described herein and the described methods / processes for configuring the encoders or decoders can be realized using standard computer programming techniques and languages. The present application is not limited to a particular processor, computer language, computer programming notation, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated chip (ASIC), etc.

[0238] The present application also provides a computer-readable signal encoding data produced through application of an encoding process according to the present application.

[0239] Certain adaptations and modifications of the described embodiments can be made. Accordingly, the embodiments discussed above are considered to be illustrative rather than restrictive.

Claims

1. A method for encoding a point cloud to generate a bitstream of point cloud data, wherein the point cloud is defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes represent a volume space having a volume that is recursively divided into subvolumes containing points of the point cloud, the current volume is divided into a first subvolume, and the current volume has one or more neighbor volumes that are divided into a second subvolume, and the method is Determining the subvolume neighborhood configuration based on the occupied data for one or more second subvolumes of at least one neighboring volume and / or the encoded occupied data for one or more first subvolumes, Selecting probabilities based at least partially on the aforementioned subvolume neighborhood configuration, Based on the aforementioned probability, the occupied data relating to one or more of the first subvolumes is entropi encoded. Methods that include...

2. A method for decoding a bitstream of point cloud data to generate at least a portion of a reconstructed point cloud, wherein the reconstructed point cloud is defined in a tree structure having a plurality of nodes with parent-child relationships, the plurality of nodes represent a volume space having a volume that is recursively divided into subvolumes containing points of the reconstructed point cloud, the current volume having one or more neighbor volumes that are divided into a first subvolume, and the current volume having one or more neighbor volumes that are divided into a second subvolume, and the method Determining the subvolume neighborhood configuration based on the occupied data for one or more second subvolumes of at least one neighboring volume and / or the decoded occupied data for one or more first subvolumes, Selecting probabilities based at least partially on the aforementioned subvolume neighborhood configuration, Based on the aforementioned probability, the occupied data relating to one or more of the first subvolumes is entropi-decoded from the bitstream. Methods that include...

3. The method according to any one of claim 1 or 2, comprising rotating the subvolume neighborhood configuration.

4. The method according to any one of claims 1 to 3, comprising reflecting the subvolume neighbor configuration.

5. The method according to claim 3, wherein the rotation is performed around an arbitrary axis.

6. The method according to claim 3, wherein the rotation is a computer implementation operation that utilizes a lookup table with an index representing the subvolume neighborhood configuration.

7. The method according to any one of claims 1 to 6, wherein selecting the probability includes selecting the probability from a first probability distribution among a plurality of probability distributions, and at least one of the plurality of probability distributions corresponds to a plurality of possible subvolume neighborhood configurations.

8. The method according to any one of claims 1 to 7, comprising updating the aforementioned probability.

9. The aforementioned subvolume neighborhood configuration is a first subvolume neighborhood configuration, the probability is a first probability, and the method is Based on the decoded occupancy data for the first subvolume, the second subvolume neighborhood configuration is determined. Based on the second subvolume neighborhood configuration, a second probability is selected, Based on the second probability, the occupied data relating to another first subvolume is entropi-decoded from the bitstream. The method according to claim 2, including the method described in claim 2.

10. The method according to any one of claims 2 or 9, wherein the selection of the aforementioned probability is at least in part based on a volume neighborhood configuration determined based on occupancy data relating to a volume having subvolumes that have not yet been decoded.

11. The method according to any one of claims 2, 9, or 10, wherein the entropy decoding is performed for a single first subvolume, and determining the subvolume neighborhood configuration is based on occupancy data of three or more subvolumes, each comprising the first subvolume or the second subvolume, each sharing a surface with the single first subvolume.

12. The method according to any one of claims 2, 9, 10, or 11, wherein the entropy decoding is performed for a single first subvolume, and the determination of the subvolume neighborhood configuration is based on the occupancy data of 19 or fewer already decoded subvolumes, each sharing a face, edge, or vertex with the single first subvolume.

13. The method according to any one of claims 11 or 12, wherein determining the subvolume neighborhood configuration is at least partially based on the occupied data of three or more subvolumes, each comprising a first subvolume or a second subvolume, each sharing an edge with the single first subvolume.

14. The method according to any one of claims 11 to 13, wherein determining the subvolume neighborhood configuration is at least partially based on the occupancy data of one or more first or second subvolumes that share vertices with the single subvolume.

15. A method for encoding point cloud data of a first point cloud based on occupancy associated with a second point cloud, wherein the first point cloud and the second point cloud each comprise a first volume and a second volume, each divided into a first subvolume and a second subvolume, and the method is With respect to the current volume of the first volume of the first point cloud, The subvolume neighborhood configuration for the current volume is determined based on the occupancy data for one or more second subvolumes of the second volume of the second point cloud, wherein the second volume in the second point cloud corresponds to the current volume of the first point cloud. Selecting probabilities based at least partially on the aforementioned subvolume neighborhood configuration, Based on the aforementioned probability, the occupied data relating to one or more first subvolumes is entropi encoded. Methods that include...

16. A method for decoding a bitstream of point cloud data to reconstruct at least a portion of a first point cloud based on occupations associated with a second point cloud, wherein the first and second point clouds each comprise a first volume and a second volume, each divided into a first subvolume and a second subvolume, and the method With respect to the current volume of the first volume of the first point cloud, The subvolume neighborhood configuration for the current volume is determined based on the occupancy data for one or more second subvolumes of the second volume of the second point cloud, wherein the second volume corresponds to the current volume of the first point cloud. Selecting probabilities based at least partially on the aforementioned subvolume neighborhood configuration, Entropy decoding the bitstream to reconstruct the occupied data relating to one or more first subvolumes based on the aforementioned probabilities. Methods that include...

17. The method according to claim 15 or 16, wherein the first point cloud and the second point cloud are configured to form a chronological sequence of a plurality of point clouds.

18. The method according to any one of claims 15 to 17, wherein the second volume corresponds to the current volume based on the second volume which is located substantially in the same location as the current volume.

19. The aforementioned method, The second subvolume neighborhood configuration is determined based on the occupancy data relating to the third subvolume of at least one neighboring volume of the current volume. The method according to any one of claims 15 to 18, wherein the selection of the probability is at least partially based on the second subvolume neighborhood configuration.

20. Determining the motion vector associated with the current volume, Determining the base location within the second point cloud, wherein the base location substantially corresponds to the location of the first volume within the first point cloud, The location of the second volume in the second point cloud is determined by applying the motion vector to the base location. The method according to any one of claims 15 to 19, including the method described in any one of claims 15 to 19.

21. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, wherein the encoder Processor and Memory configured to store instructions and An encoder comprising, wherein, when the instruction is executed, causes the processor to perform the method according to claim 1, claims 3 to 8 when dependent on claim 1, claim 15, and claims 17 to 20 when dependent on claim 15.

22. A decoder for decoding a bitstream of compressed point cloud data to generate at least a portion of a reconstructed point cloud, wherein the decoder: Processor and Memory configured to store instructions and A decoder comprising, wherein, when the instruction is executed, causes the processor to perform the method according to claim 2, claims 3 to 14 when dependent on claim 2, claim 16, and claims 17 to 20 when dependent on claim 16.

23. A processor-readable medium for storing processor-executable instructions, wherein, when executed by a processor, the processor-executable instructions cause the processor to perform the method according to any one of claims 1 to 20.

24. A computer-executable program comprising instructions, wherein, when executed by a computer, the instructions cause the computer to perform the method according to any one of claims 1 to 20.

25. A bitstream of compressed point cloud data, wherein the bitstream is generated according to the method of claim 1, claims 3 to 8 when dependent on claim 1, claim 15, and claims 17 to 20 when dependent on claim 15.

Images