Method and device for predicting point cloud attribute coding

Abstract

Method and apparatus for attribute coding for point clouds. In a top-down coding process, a predicted attribute geometric weighted sum is determined for each occupied subvolume in a volume based on a prediction operation that uses at least the attribute geometric weighted sum of the volume. The prediction operation involves upsampling data from the previous (parent) depth of the coding tree. The predicted attribute geometric weighted sum and the corresponding original attribute geometric weighted sum are subtracted in the attribute domain or transform domain to produce residual coefficients. The AC residual is sparsely encoded in the bitstream. The transform used has a DC coefficient property so that the DC coefficient of the transform of the attribute geometric weighted sum of the set of child subvolumes is the attribute geometric weighted sum of the volume.

Description

Technical Field

[0001] The present application relates generally to data compression and, in one specific example, to methods and apparatus for point cloud encoding. The present application describes methods and apparatus for encoding point cloud attributes using predictive coding. Background Art

[0002] Data compression is used in communications and computer networking to efficiently store, transmit, and reproduce information. Three-dimensional representations of physical spaces are often stored as point clouds, which consist of multiple points, each with a geometric position in a given space. Point clouds have a range of applications. In one example, they can be used for virtual reality (VR) and augmented reality (AR) applications. In another example, point clouds can be used in computer vision applications, such as autonomous vehicles.

[0003] Point cloud data can be very large, especially when it is time-varying. For example, LiDAR scans can produce large amounts of sparse point cloud data that must be processed, analyzed, or transmitted very quickly for real-time vehicle control applications. As another example, sophisticated VR applications may involve dense point cloud data with rapid real-time user movement. Therefore, efficient compression of point cloud data becomes a challenge.

[0004] Much work has been done on mechanisms for efficiently encoding the geometry of point clouds. This typically involves recursively splitting the geometric space into smaller and smaller sub-cells, until each occupied sub-cell contains only a single point. An octree-based encoding process has been developed for efficiently encoding this positioning or geometry data.

[0005] In addition to encoding geometry, one or more attributes about a point can also be encoded. For example, in the case of VR, color or intensity (brightness) information can be encoded for each pixel. In the case of LiDAR scanning, reflectivity information can be encoded for each pixel. Pixels can have alternative or additional attributes.

[0006] It would be advantageous to provide methods and apparatus for efficiently and effectively compressing attribute data of a point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Reference will now be made by way of example to the accompanying drawings which show example embodiments of the present application, and in which:

[0008] Figure 1 An example of octree-based point cloud geometry is shown;

[0009] Figure 2 shows the application of the 2-point transform to an example child subvolume;

[0010] Figure 3illustrates the recursive application of region-adaptive hierarchical transformations in three directions for example child nodes;

[0011] Figure 4 A flowchart illustrating an example method of encoding attribute data of a point cloud;

[0012] Figure 5 A flowchart illustrating an example method of decoding compressed attribute data of a point cloud;

[0013] Figure 6 An example encoding process for point cloud attributes using top-down coding and inter-layer prediction is diagrammed.

[0014] Figure 7 Diagrammatically illustrates Figure 6 Determination of the residual AC coefficient in the process;

[0015] Figure 8 An example decoding process is diagrammatically illustrated;

[0016] Figure 9 An example encoder is shown in block diagram form;

[0017] Figure 10 An example decoder is shown in block diagram form;

[0018] Figure 11 The upsampling process of the example is shown in a two-dimensional diagram;

[0019] Figure 12 An example graph reflecting five nodes or points is illustrated;

[0020] Figure 13 shows an example graph of compression performance for one implementation in encoding YUV attributes of a dense point cloud;

[0021] Figure 14 Another example graph is shown for an implementation in encoding reflectivity of a sparse point cloud;

[0022] Figure 15 A simplified block diagram illustrating an example embodiment of an encoder; and

[0023] Figure 16 A simplified block diagram of an example embodiment of a decoder is shown.

[0024] Like reference numerals may be used in different drawings to denote like components. DETAILED DESCRIPTION

[0025] This application describes methods and apparatus for encoding attributes of a point cloud. The encoding may be top-down encoding. The method and apparatus may involve prediction operations based on upsampling of attribute data from a parent depth in a coding tree. The method and apparatus may employ a transform that conforms to the DC coefficient property, such that the set DC coefficient used to transform the geometry-weighted sum of attributes of a child subvolume is the geometry-weighted sum of the attributes of the parent volume.

[0026] In one aspect, the present application describes a method for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being located in a space that is recursively decomposed and contains points of the point cloud, each point having a corresponding attribute. The method may include: in a top-down encoding process with respect to the recursively decomposed space, for a volume containing subvolumes containing a plurality of points, determining a predicted attribute geometric weighted sum for each occupied subvolume of the volume based on a prediction operation, the prediction operation being based at least on the attribute geometric weighted sum of the volume. The method may also include: applying a transform to the predicted attribute geometric weighted sum to produce predicted coefficients, and applying the transform to corresponding original attribute geometric weighted sums of the occupied subvolumes to produce original coefficients; determining a plurality of AC coefficients from a difference between the original coefficients and the predicted coefficients, wherein the residual coefficients include a DC coefficient and the plurality of AC coefficients; and encoding the plurality of AC coefficients to output a bitstream of compressed point cloud data.

[0027] In another aspect, the present application describes a method for decoding a bitstream of encoded attributes of a point cloud, the point cloud being located in a space that is recursively decomposed and includes points in the point cloud, each point having a corresponding attribute. The method may include, in a top-down encoding process with respect to the recursively decomposed space, determining, for a volume including subvolumes containing a plurality of points, a predicted geometrically weighted sum of attributes for each occupied subvolume of the volume based on a prediction operation, the prediction operation being based on at least the geometrically weighted sum of attributes for the volume. The method may also include: applying a transform to the predicted geometrically weighted sum of attributes to produce predicted AC coefficients; decoding the bitstream to reconstruct residual AC coefficients; setting a DC coefficient to the geometrically weighted sum of attributes for the volume; adding the residual AC coefficient and the DC coefficient to the predicted AC coefficients to produce reconstructed coefficients; and inversely transforming the reconstructed coefficients to produce a reconstructed geometrically weighted sum of attributes for the occupied subvolumes. At a maximum depth, the reconstructed geometrically weighted sum of attributes for the occupied subvolumes is the reconstructed attribute.

[0028] In a further aspect, the present application describes encoders and decoders configured to implement such encoding and decoding methods.

[0029] In yet a further aspect, the present application describes a non-transitory computer-readable medium storing computer-executable program instructions that, when executed, cause one or more processors to perform the described encoding and / or decoding methods.

[0030] In yet another aspect, the present application describes a computer-readable signal containing program instructions that, when executed by a computer, cause the computer to perform the described encoding and / or decoding method.

[0031] This application also describes computer-implemented applications, including topographic applications, mapping applications, automotive industry applications, autonomous driving applications, virtual reality applications, and cultural heritage applications, among others. These computer-implemented applications include receiving a data stream or data file, unpacking the data stream or data file to obtain a bitstream of compressed point cloud data, and decoding the bitstream as described in the above aspects and implementations thereof. Thus, these computer-implemented applications utilize point cloud compression techniques according to the aspects and implementations described throughout this application.

[0032] This application also describes methods for encoding and decoding point clouds, as well as encoders and decoders for encoding and decoding point clouds. In some implementations, a receiving unit receives multiplexed data, where the multiplexed data is obtained by multiplexing encoded point cloud data with other encoded data types (such as metadata, images, video, audio, and / or graphics). The receiving unit includes a demultiplexing unit and at least one decoding unit (or decoder), where the demultiplexing unit is configured to separate the multiplexed data into encoded point data and other encoded data, and at least one decoding unit (or decoder) is configured to decode the encoded point cloud data. In some other implementations, a transmitting unit transmits the multiplexed data, where the multiplexed data is obtained by multiplexing encoded point cloud data with other encoded data types (such as metadata, images, video, audio, and / or graphics). The transmitting unit includes at least one encoding unit (or encoder) and a multiplexing unit, where the at least one encoding unit (or encoder) is configured to encode the point cloud data, and the multiplexing unit is configured to combine the encoded point cloud data with the other encoded data into the multiplexed data.

[0033] Those skilled in the art will appreciate other aspects and features of the present application by reviewing the following description of the examples in conjunction with the accompanying drawings.

[0034] Any feature described in relation to one aspect or embodiment of the invention may be used in relation to one or more other aspects / embodiments.These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described herein.

[0035] In the following description, the terms "node", "volume" and "subvolume" may be used interchangeably. It will be understood that a node is associated with a volume or subvolume. A node is a specific point on the tree, which may be an internal node or a leaf node. A volume or subvolume is the bounded physical space represented by the node. In some cases, the term "volume" may be used to refer to the largest bounded space defined to contain a point cloud. For the purpose of constructing a tree structure of interconnected nodes for encoding the point cloud structure, a volume may be recursively divided into subvolumes.

[0036] In the present application, the term “and / or” is intended to cover all possible combinations and subcombinations of the listed elements, including any element alone, in any subcombination, or all of the listed elements, and does not necessarily exclude additional elements.

[0037] In this application, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any of the listed elements alone, in any subcombination, or all of these elements, without necessarily excluding additional elements, and without necessarily requiring all of these elements.

[0038] Many of the examples below will refer to compression of point cloud data. Point clouds provide a suitable example for illustrating the advantages of the present application because the data in point clouds is voluminous and prediction operations can be used to improve compression. However, it will be understood that point clouds are used as an example and that the present application provides methods and apparatus that can be applied to compress other types of data for other purposes.

[0039] A point cloud is a collection of points in a three-dimensional coordinate system. These points are often intended to represent the surface of one or more objects. Each point has a position (location) in this three-dimensional coordinate system. This location can be represented by three coordinates (X, Y, Z), which can be a Cartesian coordinate system or any other coordinate system. The terms "location," "position," or "geometry" are used interchangeably herein to refer to the position of a point in space.

[0040] Points may have other associated attributes, such as color, which in some cases may also be a three-component value such as R, G, B or Y, Cb, Cr. Depending on the desired application for the point cloud data, other associated attributes may include transparency, reflectivity, normal vector, timestamp, etc.

[0041] Point clouds can be static or dynamic. For example, a detailed scan or map of an object or terrain can be static point cloud data. LiDAR-based scans of an environment for machine vision purposes can be dynamic, as the point cloud (at least potentially) changes over time, e.g., with each successive scan of a volume. A dynamic point cloud is therefore a time-ordered sequence of point clouds.

[0042] Point cloud data can be used in many applications, including conservation (scanning of historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous vehicles), and virtual reality systems, to name a few. Dynamic point cloud data used for applications such as machine vision can be very different from static point cloud data used for conservation purposes. For example, automotive vision often involves relatively low-resolution, non-color, highly dynamic point clouds acquired by LiDAR (or similar) sensors with high capture frequencies. Such point clouds are not intended for human consumption or viewing, but rather for machine object detection / classification in decision-making processes. Attributes can also be derived from detection / classification algorithms that segment the point cloud into detected / classified objects; in such cases, the attribute value is typically the label of the object to which the point belongs. As an example, a typical LiDAR frame contains tens of thousands of points, while high-quality virtual reality applications require millions of points. It is expected that the demand for higher-resolution data will continue to increase over time as computing speeds increase and new applications are discovered.

[0043] While point cloud data is useful, the lack of effective and efficient compression (i.e., encoding and decoding processes) may hinder its adoption and deployment.

[0044] A more common mechanism for encoding point cloud data is by using a tree-based structure. In a tree-based structure, a bounded three-dimensional volume of a point cloud is recursively divided into sub-volumes. The nodes of the tree correspond to sub-volumes. The decision whether to further divide a sub-volume can be based on the resolution of the tree and / or whether the sub-volume contains any points. Leaf nodes can have an occupancy flag that indicates whether their associated sub-volume contains points. A split flag can indicate whether a node has child nodes (i.e., whether the current volume has been further divided into sub-volumes). These flags can be entropy coded in some cases, and in some cases predictive coding can be used.

[0045] A commonly used tree structure is the octree. In this structure, the volumes / subvolumes are all cubes, and each split of the subvolume produces eight further subvolumes / subcubes. Another commonly used tree structure is the KD-tree, in which a volume (cube or rectangular cuboid) is recursively divided into two by a plane that is orthogonal to one of the axes. The octree is a special case of the KD-tree, in which the volume is divided by three planes, each of which is perpendicular to one of the three axes. Both examples involve cubes or rectangular cuboids; however, the present application is not limited to such tree structures, and the volumes and subvolumes may have other shapes in some applications. The partitioning of the volume does not necessarily have to be into two subvolumes (KD-tree) or eight subvolumes (octree), but may involve other partitionings, including partitioning into non-rectangular shapes or involving non-adjacent subvolumes.

[0046] For ease of explanation, this application may refer to octrees because they are a popular candidate tree structure for automotive applications, but it will be understood that the methods and apparatus described herein may be implemented using other tree structures.

[0047] In the description herein, reference may be made to "levels" or "depths" of a point cloud or its tree representation. In a tree-based recursive splitting of subvolumes, it will be understood that each successive splitting adds a further level or depth to the tree, which may be up to a depth or level where each occupied subvolume contains a point of the point cloud. Subvolumes may also be referred to as "nodes". By convention, in this application, the root or top node or level is the largest defined volume that contains at least a portion of the point cloud. For example, in some cases, the volume containing the point cloud may be partitioned into largest coding units (LCUs), and each LCU may be encoded independently. In some cases, an LCU may be the volume containing the entire point cloud. Also by convention, in this application, a subvolume at a depth d (or level) may be subdivided into subvolumes at a "larger" depth d+1, such that the depth d increases as the resolution of the tree increases. In this sense, "larger" depths or levels refer to higher resolution levels of smaller sub-volumes in a top-down construction of a tree with a root node at the top at d=0 or d=1, depending on the convention chosen for the depth index d.

[0048] This application may also refer to "upsampling" of attribute data, where attribute data from level d is used to construct a prediction of attribute data at level d+1, i.e., data is upsampled to predict data at a greater depth / higher resolution. These terminology or labeling conventions may be modified in some implementations without affecting the substantive operation of the methods and apparatus described herein.

[0049] The tree geometry is often losslessly encoded. Flags or other bits defining the tree structure can be serialized in some cases. A binary or non-binary encoder can be used. In some implementations, prediction operations can be used to attempt to further compress the data. Entropy coding can also improve compression. At the decoder, the compressed data is losslessly decoded to reconstruct the tree geometry, enabling the decoder to determine the position of each point in the point cloud.

[0050] In some cases, lossy compression can be used to encode geometry. In this case, the encoded octree represents an approximation of the original point cloud, which generally requires fewer bits than a lossless encoding of the octree representing the original point cloud, but at the expense of distortion between the encoded geometry and the original geometry. In lossy compression, during the encoding process, the attributes associated with a reconstructed point in the lossy encoded and reconstructed point cloud can be found using interpolation of the attributes of the original point closest to the reconstructed point.

[0051] Discussions on standards for point cloud compression (PCC) are ongoing within the Moving Picture Experts Group (MPEG) and the International Organization for Standardization (ISO). For example, current work is reflected in MPEG-1 Part 9 on geometry-based point cloud compression. Those skilled in the art will be familiar with the options for compression of geometric point cloud data.

[0052] Point clouds can include more than just the geometric locations of points. In some cases, points have attributes such as color, reflectivity, transparency, timestamps, or other properties that can be represented by some kind of value or parameter. For example, color can be a three-component color value, such as RGB or YUV, commonly used in video and images.

[0053] There are two competing attribute encoding methods under consideration: Level of Detail (LoD) and Region-Adaptive Hierarchical Transform (RAHT). LoD is described, for example, in "G-PCC codec description" (ISO / IEC JTC1 / SC29 / WG11, Macau, China, output document w18015, December 2018). RAHT is described, for example, in "Compression of 3D point Cloud Using a Region-Adaptive Hierarchical Transform" (RL de Queiroz, PAChou, IEEE Transactions on Image Processing, vol 25(8), August 2016).

[0054] Typically, LoD involves sampling a point cloud (PC) at several levels of detail from 1 to L to obtain an increasing set of points Attributes are encoded hierarchically, first for points in E1, then for points in E2 (not in E1), and so on. l-1 The encoded attributes in E lA predictor of the attribute in [ ] is obtained by, for example, computing a weighted average of adjacent coded attributes, subtracting the original attribute from the average to obtain a residual, and encoding the residual. This approach is similar to scalable video coding.

[0055] LoD finds efficient application in the case of dense point clouds, such as might be encountered in VR applications, for example. Due to the lack of positional correlation, it does not necessarily perform well in the case of sparse point clouds, such as those produced by LiDAR, for example. LoD is also relatively computationally demanding, as it works on unstructured geometry of points with neighboring encoded attributes, i.e., the search for neighboring points may be particularly necessary in sparse point clouds, as neighboring points do not necessarily belong to neighboring nodes in the octree, making the use of an octree to find neighboring points inefficient.

[0056] RAHT is a form of 3D transform that can be applied to a 2×2×2 cube. It is applied by performing a series of 2-point transforms in the X, Y, and Z directions to generate a set of AC coefficients, and ultimately reducing the cube to a single DC coefficient through these successive transforms. RAHT is orthogonal and relatively easy to implement in terms of computational complexity. RAHT finds efficient application in the case of sparse point clouds, but does not necessarily perform well in the case of dense point clouds. Further details on the RAHT transform are provided below.

[0057] According to one aspect of the present application, methods and apparatus for encoding point cloud attributes are described that perform well compared to both LoD and RAHT, regardless of whether the point cloud is dense or sparse. The methods and apparatus may involve using per-node transformations rather than per-direction transformations as in RAHT, although in some embodiments the transformations may be direction-specific, RAHT being one possible example. In some cases, the methods and apparatus may include prediction of attributes, particularly inter-depth prediction of attributes. In some cases, the sum of attributes of a parent node at one depth level is used, at least in part, to predict the sum of attribute values associated with its child subvolumes at the next depth level. In some such cases, attribute data from one or more neighbors of the parent node may be used in predicting the sum of attribute values for the child subvolumes. The attribute sum used may, in some cases, be a geometrically weighted sum or mean sum of the attributes. In at least one example, the prediction operation applies weights to neighboring attributes from neighboring nodes that reflect (the inverse of) the geometric distance of the neighboring node to the child subvolume for which the attribute value is to be predicted.

[0058] As seen in image and video coding, transforms offer the potential for compression gains by mapping pixel or voxel domain data to the spectral domain. The resulting transform domain data includes a DC component as well as multiple AC components. By concentrating the data in the DC component and several lower-frequency AC components, overall compression can be improved. In some cases, this is further combined with coefficient quantization in lossy coding schemes to further improve data compression at the expense of introducing distortion to the encoded data relative to the original data.

[0059] Compared to video or image compression, the difficulty of point cloud compression is that not every location in the partition volume may contain a point. Towards the leaf nodes of the coding tree, some sub-volumes may contain points, and some sub-volumes may not contain points.

[0060] Figure 1 An example of octree-based point cloud geometry is shown as reflected at depth d and a greater depth d+1. At depth d, the occupied subvolume is indicated by shading. The current subvolume 102 is indicated by darker shading. At depth d+1, the child subvolumes 104 of the current subvolume 102 are shown by shading. At the highest resolution depth, the occupied child subvolumes 104 each contain a corresponding point of the point cloud, and in this example, each point has a corresponding attribute value(s).

[0061] RAHT starts at the deepest level (i.e., highest resolution), where each occupied subvolume contains a single point. To perform attribute data compression using RAHT, a two-point transformation is first applied in one direction (x, y, or z). Figure 2 The application 201 of a 2-point transform to an example child subvolume 104 when applied along a direction 200 is graphically illustrated. If the two child subvolumes aligned in the direction of the transform are occupied, i.e. have corresponding property values, then RAHT converts them into DC and AC components. If the properties are given by c1 and c2 respectively, then RAHT can be expressed as:

[0062]

[0063] For example, Figure 2As depicted above, attributes c1 and c2 of two corresponding child sub-volumes 210 and 211 are transformed by a two-point RAHT transform into a DC coefficient and an AC coefficient 213 associated with a merged sub-volume 212. The same process is applied to the attributes of sub-volumes 220 and 221, which are transformed into a DC coefficient and an AC coefficient 223 associated with a merged sub-volume 222. Volume 230, which is not aligned with another sub-volume along direction 200, is not transformed (or equivalently transformed using a one-point transform as the identity transform to obtain the DC coefficient), and its (un)transformed coefficient (i.e., the DC coefficient) is associated with a merged sub-volume 232.

[0064] Elementary transformation RAHT(w1,w 2) Can be defined as:

[0065]

[0066] Where w1 is the number of points contained in the first child subvolume, and w2 is the number of points contained in the second child subvolume. In the case of the deepest level, the number w i is 1. After the elementary transform, the AC coefficients are encoded and the DC coefficient is kept as new information associated with the merge of the two sub-nodes. The merged sub-volume has the number of associated points given by the sum w1+w2.

[0067] By construction, the merged subvolumes form a set of volumes belonging to a 2D structure perpendicular to the transformation direction. For example, the merged subvolumes 212, 222, and 232 belong to a plane perpendicular to the direction 200. The method can then be recursively applied along a second direction 250 perpendicular to the first direction 200.

[0068] Figure 3 1 , a recursive application 201, 301, and 302 of RAHT along three consecutive orthogonal directions 200, 250, and 300 of an example child node 104 is illustrated. A second application 301 of RAHT along the second direction 250 to the merged subvolumes 212, 222, and 232 provides two DC coefficients and one AC coefficient 323 associated with further merged subvolumes 312 and 322. The two further merged subvolumes 312 and 322 belong to a 1D structure that is perpendicular to the two directions 200 and 250, i.e., the subvolumes are aligned along a direction 300 that is perpendicular to the first two directions 200 and 250. Then, a third application 302 of RAHT along the third direction 300 to the further merged subvolumes 312 and 322 provides a unique DC coefficient 332 and AC coefficient 333.

[0069] As a result, the recursive application of RAHT on the child node 104 provides a unique DC coefficient 332 and a set of AC coefficients 343 ( Figure 3 213, 223, 323, and 333 in the example of ). It will be appreciated that this property remains constant regardless of the configuration of occupied subvolumes in the child node 104.

[0070] The AC coefficients 343 obtained from the application of RAHT in three directions are encoded in the bitstream. The unique DC coefficient 332 obtained becomes the "attribute data" of the parent node in the next round of recursive encoding, in which the parent node and its seven siblings in the octree are encoded using the same RAHT process. This process continues recursively in a bottom-up manner until the root node is reached. At the root node, the AC coefficients and the final DC coefficient are encoded in the bitstream.

[0071] Top-down attribute encoding

[0072] In one aspect of the present application, instead of using a bottom-up recursive transform like RAHT, the encoding process is top-down, starting from the root node and proceeding downwards towards the level of sub-volumes containing individual points. Also, as mentioned above, in some implementations, a "whole node" transform is applied to find the DC coefficient and the associated AC coefficient of the sub-volume.

[0073] In another aspect of the present application, the transform is applied to the set of attributed geometric weighted sums of child nodes within a node; the decoded attributed geometric weighted sum per subvolume corresponds to the respective DC coefficient of the transform to the next level down.

[0074] In yet a further aspect of the present application, depth inter prediction may be incorporated. Depth inter prediction may be used to predict attribute values for child nodes. More specifically, the prediction may be a prediction of a geometrically weighted sum of the attribute values in the child node. This prediction of the attribute geometrically weighted sum is subtracted from the actual or original geometrically weighted sum of the attribute to obtain a residual geometrically weighted sum of the attribute. Note that the transform may be applied before or after the subtraction. After the transform of the residuals (or the transform of the predicted and original values before finding the difference), a set of residual AC coefficients is produced. The AC coefficients are encoded. Note that the DC coefficient does not need to be encoded since it is known from the inverse transform of the coefficients at the previous (parent) level.

[0075] At the encoder, the geometry is known, and thus the number of points per subvolume. At the decoder, the positions of the points in space are also known, based on decoding the compressed point cloud geometry. Therefore, both the encoder and decoder have the structural information to know whether a subvolume contains points, and can use a simple bottom-up process in the octree to find the number of points w contained by each node.

[0076] The number of points in any given subvolume can be specified as w, where w is the number of points in the subvolume at depth d. For each point p, the attribute value can be specified as attribute(p). The sum of the attribute values in a given node can be given by:

[0077]

[0078] The mean of the attribute in a node is therefore:

[0079] a node =A node / w node

[0080] Then the geometric weighted sum of attribute values can be defined as

[0081] The transformation of attribute information in the domain This domain is used because the construction of the orthogonal transform implies the "DC coefficient property", that is,

[0082] The quantity determined for each subvolume from the root node down to the leaf nodes is the geometrically weighted sum of the attributes: This corresponds to the DC coefficient of the node when transforming the geometrically weighted sum of the attributes of its set of child nodes. This allows a top-down encoding process where each layer inherits its DC value from the inverse transform at the parent node level, i.e. the geometrically weighted sum of the attributes of the parent node.

[0083] When encoding a set of child nodes within a node, the encoder determines the geometrically weighted sum of the attributes of each child node If there are k child nodes and the transform is specified as T, the encoder applies the transform to the set of k child nodes to produce transform-domain coefficients:

[0084]

[0085] The DC coefficient is known as the quantity from the parent node Thus, the encoder encodes the AC coefficients and then continues to perform the same process within each of the child nodes.

[0086] The decoder starts at the root node and decodes the root DC coefficient and AC coefficient, and then inverse transforms them to obtain the decoded quantities of the root node's child nodes. For each of these child nodes, the decoder then uses the decoded amount of the i-th child As the (decoded) DC coefficient of the child node, decode the AC coefficient and apply the inverse transform to obtain the decoded attribute geometric weighted sum for each of the grandchild nodes within the child node This process repeats until the decoding reaches a leaf node and the final decoded attribute value is obtained.

[0087] It will be appreciated that in some implementations, quantization may be applied to the transform domain coefficients to introduce lossy coding.

[0088] Now refer to Figure 4 , which shows a flow chart of an example method 400 for encoding attribute data of a point cloud. The method 400 assumes that the three-dimensional space in which the point cloud data is located has been appropriately partitioned using a tree-based recursive partitioning (such as an octree). From this partitioning and the encoding of the geometry, the encoder knows which leaf nodes contain points, and therefore the number w of points in each sub-volume (node) within the tree. Likewise, the encoder knows from the raw point cloud data the attribute value associated with each point in the point cloud. As mentioned above, the attribute can be color, reflectivity, or any other characteristic.

[0089] Method 400 begins with operation 402 by determining the geometrically weighted sum of the attributes of each child node at depth d+1 within the current (parent) node. Operation 402 is used to encode the current (parent) node at depth d. In operation 404, the set of geometrically weighted sums of the attributes of the child nodes is transformed to produce a set of transform domain coefficients. The transform T used in operation 404 conforms to the DC coefficient property, so that the DC coefficient

[0090] In operation 406, if the current node is a root node, the DC coefficient is encoded. The encoding may be entropy encoding. If the current node is not a root node, the DC coefficient is not encoded because the decoder will already know the DC coefficient from decoding the coefficient at the depth above.

[0091] The AC coefficients are encoded in operation 408. The encoding may be entropy encoding.

[0092] If the process is lossy, for example the transform includes a quantization step to quantize the transform domain coefficients, then in operation 410 the encoder reconstructs the (decoded) attribute geometric weighted sum of the child node by dequantizing and inverse transforming the quantized transform domain coefficients, just as the decoder would do. It then sets the DC coefficient of each child node to the geometrically weighted sum of its reconstructed attributes.

[0093] In operation 412, the encoder evaluates whether there are additional nodes to encode at the current depth d. If so, it moves to the next subvolume at depth d, as indicated in operation 414, and returns to operation 402. If not, it determines in operation 416 whether it is at the maximum depth. If so, it ends, and if not, it moves to the next lower depth d→d+1 in operation 418 and returns to operation 402 to continue encoding at the next level. It will be understood that this example is a breadth-first encoding example.

[0094] Now refer to Figure 5 , which shows a flow chart of an example method 500 for decoding compressed attribute data of a point cloud. Method 500 assumes that the decoder has already decoded the geometry of the point cloud, and therefore the positions of the points in the partitioned three-dimensional space have been determined. Therefore, the decoder knows the number w of points in each subvolume in the tree-based structure of the encoded point cloud.

[0095] Method 500 begins with operation 502. If the current node is a root node, the decoder decodes the DC coefficient. In operation 504, the decoder decodes the AC coefficients of the current node from the bitstream. It then combines the DC coefficient with the decoded AC coefficients and inverse transforms (and dequantizes, if applicable) the coefficients to produce the decoded attribute geometric weighted sum These are the decoded attribute geometry weighted sums associated with the child nodes of the current node. Furthermore, when encoding the nodes at the next level, each decoded attribute geometry weighted sum of the node serves as the (decoded) DC coefficient of that node. Thus, in operation 508, the decoder can set the (decoded) DC coefficient of each child node to its corresponding reconstructed attribute geometry weighted sum.

[0096] In operation 510, the decoder evaluates whether there are additional nodes at the current depth d. If so, it returns to operation 504 to continue decoding. If not, in operation 512, the decoder determines whether it is already at the maximum depth (e.g., a leaf node), and if so, it ends. If not, it moves to the next lower depth d→d+1 and returns to operation 504.

[0097] Top-down coding with depth-wise inter-prediction

[0098] As mentioned above, inter-depth prediction can be applied to improve compression performance. The inter-depth prediction process uses information from the parent depth, such as attribute information from nodes adjacent to the parent node, to predict the attribute information of the child node. This prediction is then subtracted from the actual attribute information at the child node level and the residual data is encoded. Using parent-level attribute information, such as from neighboring nodes, to predict the attribute information of the child depth can be called "upsampling."

[0099] In some examples, "neighboring nodes" may include nodes that are siblings of a parent node within a subvolume, such as seven siblings in an octree structure. In some examples, "neighboring nodes" may include nodes that share faces with a parent node. In some examples, "neighboring nodes" may include nodes that share edges with a parent node. In some examples, "neighboring nodes" may include nodes that share vertices with a parent node.

[0100] Since the data encoded in these examples is the geometrically weighted sum of attributes So the prediction operation aims to predict the geometrically weighted sum of the attributes. However, The magnitude of depends on the number of points w, which means When w is large, it is Thus, in some implementations, to perform depth inter prediction in a bounded domain, the upsampling process is performed in a mean attribute domain, such as a = A / w, which is naturally bounded by the attribute value range. The geometrically weighted sum of the attributes of a node can be obtained by dividing by And is converted to the mean attribute domain. That is, the mean and value of attribute a are obtained as

[0101] The bounded nature of the domain of the mean attribute is advantageous because it is associated with a more physical meaning (the mean attribute is a physical quantity, such as the mean color, but the geometric weighted sum of attributes is generally not) and numerical stability of the upsampling process, thus leading to more efficient predictions. Moreover, having a bounded domain simplifies fixed-point implementation.

[0102] The mean sum of the attribute values at depth d can then be used in the upsampling process to predict the mean sum of the upsampled attribute values at depth d+1. The mean sum of the attribute values can be the mean sum of the attribute values from the parent node and / or one or more neighboring nodes. Any of a variety of possible upsampling operations can be used, some examples of which are further described below. For example, for a child node, the predicted mean sum of the upsampled attribute values a up It can then be converted to a predicted upsampled attribute geometric weighted sum at depth d+1 Thus, the upsampling produces a predicted attribute geometric weighted sum for the set of child nodes. The encoder subtracts the predicted attribute geometric weighted sum from the actual (original) attribute geometric weighted sum to obtain residual values. These are then transformed to find the AC coefficients used for encoding.

[0103] In some cases, a transform is applied to the predicted attribute geometric weighted sum to obtain predicted coefficients, and the transform is applied to the original attribute geometric weighted sum to obtain the original coefficients. The predicted coefficients are then subtracted from the original coefficients to obtain the AC coefficients for encoding.

[0104] Now refer to Figure 6 , which diagrammatically illustrates an example encoding process 600 for point cloud attributes using top-down coding and inter-layer prediction. The process 600 is applied to encode attribute information of nodes within a parent-child volume 602. The parent-child volume 602 is partitioned into a set of child sub-volumes 604 at a depth d.

[0105] At depth d-1, the parent subvolume 602 has an adjacent occupied subvolume 606. In this example, the set of adjacent subvolumes 606 may include any occupied subvolumes at depth d-1 that share vertices with the parent subvolume 602. The encoder determines the attribute geometry weighted sum for the parent node and each of the adjacent nodes. These attribute geometric weighted sums can be "reconstructed" (decoded) attribute geometric weighted sums obtained by decoding the encoded coefficient data of their corresponding subvolumes and inversely transforming the coefficients to reconstruct the geometric weighted sums, especially if the encoding is lossy due to the use of quantization in the transform process. In this way, the encoder ensures that it is working with the same data that the decoder can use.

[0106] The encoder then applies "normalization" to The attribute geometric weighted sum of each adjacent sub-volume in the parent sub-volume 602 and the adjacent sub-volume 606 at depth d-1 is calculated as These values are known to both the encoder and decoder because the geometry of the point cloud is known to both. As a result, for each of the parent nodes and each of its occupied neighbor nodes, the encoder determines the attribute mean and A i / w i Using these values, the encoder applies an upsampling operation to generate predicted attribute means and A for each occupied child subvolume 604 of the parent subvolume 602. i，up / w i .

[0107] The encoder then inversely normalizes the predicted attribute means and s to obtain the predicted attribute geometric weighted sum A for each occupied child subvolume 604 i，up / w i .

[0108] In this example, the encoder then transforms the predicted attribute geometric weighted sum to obtain predicted transform domain coefficients. is determined by the encoder and transformed to generate the original transform domain coefficients. The predicted AC coefficients are subtracted from the original AC coefficients to achieve residual AC coefficients, which the encoder then entropy encodes to output a bitstream of encoded data for the parent node 602.

[0109] Figure 7 The determination of the residual AC coefficients by subtracting the predicted coefficients from the original coefficients is illustrated diagrammatically.

[0110] Figure 8 An example of a decoding process 800 is illustrated diagrammatically. The decoder employs the same prediction process to generate the predicted attribute geometric weighted sum It also reconstructs the residual AC coefficients by entropy decoding and inverse quantization. Note that the DC coefficient is not obtained directly from the bitstream, but is known to the decoder through the coefficient at the parent depth (depth) d-1 that has been reconstructed and inverse transformed. The (decoded) DC component is given by the geometric weighted sum of the reconstructed attributes of the parent node,

[0111] The predicted AC coefficients are then added to the reconstructed residual AC coefficients to produce reconstructed AC coefficients. These may alternatively be referred to herein as decoded coefficients. The DC coefficients obtained from the parent depth and the reconstructed AC coefficients are then subjected to an inverse transform to obtain the reconstructed attribute geometric weighted sum of the child subvolume 604 at depth d=1.

[0112] Now refer to Figure 9 , which shows in block diagram form an example encoder 900 for encoding attribute data of a point cloud. The encoder 900 can be implemented using a combination of hardware and software, such as one or more processing units, memory, and processor-readable instructions. For clarity and ease of explanation, portions of the encoder 900 related to encoding point cloud geometry or auxiliary elements (such as inputs, outputs, user interface devices), or other such components are not shown.

[0113] The encoder 900 has raw point cloud attribute information 902 and point cloud geometry 904. The encoder 900 includes a DC coefficient property compatible transform operator 906 to transform the raw point cloud attributes in the form of a weighted sum of attribute geometry into transform domain coefficients.

[0114] The same transform operator 908 is applied to the predicted attribute geometry weighted sum obtained from the prediction / up-sampling operator 910, which uses the attribute geometry weighted sum from the parent node and one or more of its neighbors (i.e., at the depth above) as its input. The output of the transform operator 908 is a set of predicted AC coefficients, which are then subtracted from the original AC coefficients from the transform operator 906 to produce residual AC coefficients. These residual AC coefficients are quantized and encoded by the quantizer and encoder 920 to produce an output bitstream of compressed point cloud attribute data.

[0115] The input to the prediction / upsampling operator 910 is provided via a decoding feedback loop 912, where the quantized residual AC coefficients are inverse quantized in an inverse quantizer 922 and then added to the predicted AC coefficients to produce reconstructed AC coefficients. This set of coefficients, along with the DC coefficient, is then inverse transformed by an inverse transform operator 914 to produce a reconstructed attributed geometric weighted sum for the child nodes of the current node, which will then serve as input to the prediction operation at the next level down.

[0116] Figure 10 An example of a corresponding decoder 1000 is shown in block diagram form in FIG. The decoder 1000 has its point cloud geometry available from previously decoded point cloud position data. For ease of explanation, elements related to decoding are not necessarily shown.

[0117] The decoder 1000 includes a decoder and an inverse quantizer 1002 to decode and inverse quantize the residual AC coefficients encoded in the bitstream. The decoder 1002 includes a prediction / up-sampling operator 1010, which reflects the residual AC coefficients from the encoder 900 ( Figure 9 ) of the same component 910( Figure 9). The prediction / upsampling operator 1010 takes the decoded / reconstructed geometric weighted sum of attribute data from the depth above (e.g., the parent node and one or more neighboring nodes) and produces a predicted attribute geometric weighted sum. Those predicted attribute geometric weighted sums are subjected to a transform 1004 to produce predicted AC coefficients. Those predicted AC coefficients are added to the reconstructed residual AC coefficients from the decoder and inverse quantizer 1002 to produce reconstructed AC coefficients. An inverse transform 1006 is applied to obtain the reconstructed attribute geometric weighted sum for the current node. Since the decoder 1000 processes data with top-down reconstruction, once it reaches a leaf node, the decoded and inverse transformed attribute data produces the reconstructed attribute information for each point.

[0118] Upsampling operation

[0119] As mentioned above, the prediction operation employs an upsampling of the attribute information from depth d-1 to produce a predicted attribute geometric weighted sum for the node at d. In these examples, the upsampling is performed in the domain of the attribute mean and A / w to ensure bounds on the values to be upsampled and numerical stability, but it will be understood that in some implementations the domain The prediction operation in .

[0120] Now refer to Figure 11 , which illustrates an upsampling process 1100 in two-dimensional diagram form. At depth d-1, there is a parent node 1102 and neighboring nodes 1104a, 1104b, and 1104c (collectively referred to as 1104). For clarity and ease of explanation, this example is illustrated in two dimensions, but extensions to three dimensions will be understood in light of the description herein.

[0121] The parent node 1102 has a child node 1106 whose attribute information is to be predicted. The DC coefficients of the parent node 1102 and each of the neighboring nodes 1104 are known. Since the geometry of the tree is known by encoding / decoding the point cloud geometry, the number of points w in any node is known to both the encoder and decoder. Therefore, by dividing the corresponding node by the corresponding value, which is equivalent to the geometric weighted sum of the attributes of the parent node 1102 and the adjacent node 1104 The DC coefficient can be converted into attribute mean and A / w.

[0122] The upsampling operation is then applied to generate the predicted attribute value mean and a for the child node 1106 upThe upsampling operation takes as input the mean sum of the attributes of the parent node 1102 and its occupied neighboring nodes 1104. In this example implementation, the upsampling operation also considers a distance metric that associates the child node 1106 with the corresponding parent node 1102 and neighboring node 1104. The distance metric may reflect the distance between the center point of the subvolume corresponding to the child node 1106 and the center point of the subvolume corresponding to the corresponding parent node 1102 or neighboring node 1104. The inverse of the distance The relative weight of the correlation between the attribute information from the node at depth d-1 and the node at depth d can be reflected. Other weighting factors or additional weighting factors can be used in other implementations of the upsampling operation. In one example, the predicted attribute mean sum of the child node 1102 can be given by the following weighted sum:

[0123]

[0124] In some implementations, upsampling may be achieved using FIR (finite impulse response) filters.

[0125] Full Node Transformation

[0126] As mentioned above, a two-point RAHT transform may be applied successively in the x, y, and z directions to find the DC transform domain coefficients and AC coefficients for the octree-based subvolume. Recall that the formula for the RAHT transform is given by:

[0127]

[0128] In practical implementations, the transform is performed in three cascaded steps, for example by applying it in the X direction, then in the Y direction, then in the Z direction. However, it is mathematically possible to combine two elementary RAHT transforms to obtain a single orthogonal three-point transform. The generalization to more points is obtained by induction. Let For each containing w i The first elementary RAHT transform is applied to the first two nodes to obtain the first DC coefficient DC 2p , the first AC coefficient AC1 and the unchanged third node information

[0129]

[0130] Then the second primary RAHT transform is applied to the first DC coefficient and the third node information. As a result, the second DC coefficient combining the three points is obtained. and two AC coefficients.

[0131]

[0132] Then the orthogonal matrix RAHT(w1, w2, w3) of the three-point orthogonal transformation of the combination of two elementary two-point RAHT transformations is the product:

[0133]

[0134] Keep in mind that we might next consider graph transformations. Graph transformations are a general framework for constructing transformations on any set of points. This mathematical construction is described in detail below.

[0135] Now refer to Figure 12 , which illustrates an example graph reflecting five nodes or points. The set of points can be designated as E. For two different points p in E i ≠p j , the correlation factor d between these two points ij can be determined. In many example implementations, the correlation factor is a decreasing function of the distance between the two points. For example, the inverse of the Euclidean distance (with a negative sign by convention) can be taken:

[0136] d ij =-1 / ||p j -p i ||2

[0137] The diagonal entries can be obtained by summing the entries row by row as follows:

[0138] d ii =-∑ j≠i d ij

[0139] By construction, this matrix is symmetric and diagonally dominant. Therefore, it is diagonalizable in an orthogonal basis:

[0140] D=VAV T , where V T =V -1

[0141] The Laplace bilinear operator can be defined to operate on pairs (a, b) of attributes associated with a set E of points by:

[0142] Lap(a,b):=a T Db

[0143] The norm and distance can be derived:

[0144] ||a|| Lap :=Lap(a,a)=a T Da

[0145] d Lap(ab):=||ba|| Lap

[0146] For example, the distance can measure the difference between the original attribute and its encoded version attribute code The distortion Δ between (in the following expressions, the symbol α is used instead of attribute):

[0147] Δ=d Lap (α, α code )=(α-αcode) T D(α-α code )

[0148] Using the orthogonal decomposition of matrix D, it is possible to obtain:

[0149]

[0150] Where V i is the i-th column of V. Therefore, the graph transformation GT for the set of points E is naturally:

[0151] GT(E):=V T

[0152] By construction, the transform is orthogonal and well-suited for attribute compression by first applying a constant-step quantizer to the transformed attribute coefficients and then entropy coding the quantized coefficients. When applied to large sets of points, the graph transform is powerful in terms of compression efficiency, but is less practical than RAHT because the quantization process must be performed with a typical complexity of O(N) for N points. 2 ) to perform a diagonal decomposition of the matrix.

[0153] Still considering Figure 12 A graph is considered to have N nodes, where each node contains a corresponding number w i points. The graph γ is constructed from the centers of the cubes associated with the nodes. From the graph γ and the weights w i We can construct a “weighted graph transformation” WGT(w i ,γ). Such a transformation will transform the attribute information c associated with the node i Transformed into a DC coefficient and N-1 AC coefficients.

[0154]

[0155] In some embodiments, to be compatible with attribute encoding for point clouds, the transformation reflects this relationship (using the previous notation and w being w i and):

[0156]

[0157] This is referred to above as the "DC coefficient property." Therefore, the matrix D utilizes the elements d ij is defined as the Laplace matrix of the graph transformation obtained for the graph γ. Then, the matrix D W (i.e., the weighted Laplace matrix) can be defined as:

[0158]

[0159] Since this matrix is real and symmetric, it can be decomposed as follows:

[0160] D W =VΛV T , where V T =V -1

[0161] And similar to the unweighted graph transformation, the weighted graph transformation is defined as:

[0162] WGT(w i , Y): = V T

[0163] It can be shown that the matrix D W The kernel of is non-zero,

[0164]

[0165] And the associated column vector V in the orthogonal decomposition is:

[0166]

[0167] This indicates that the weighted graph transformation WGT(w i ,γ) satisfies the DC coefficient property stated above. If all weights w i With the same value w, the weighted graph transformation is identical to the well-known unweighted graph transformation. This is particularly true when applied to occupied leaf nodes, where systematically w = 1. Interestingly, this two-point weighted graph transformation is identical to the two-point elementary RAHT transformation:

[0168]

[0169] This means that RAHT is a special case of WGT.

[0170] The encoding and decoding process may employ any transform that complies with the DC coefficient properties. This includes the WGT described above, including but not limited to RAHT.

[0171] Applications to other tree structures

[0172] The examples described above are based on octree geometry, where each node has eight children that divide the volume associated with the node into eight subvolumes. The most common octree has cubes (or cuboids) associated with nodes and all cubes (or cuboids) of the same depth have a common size.

[0173] Another popular tree structure for point cloud representation is the KD tree. They essentially split a volume into two sub-volumes, for example by splitting the initial cuboid along a plane parallel to one of its faces. The two sub-cubes do not necessarily have the same size; the partitions can be unequal. The direction of the split (among the three directions) and the unequal nature of the split (if any) are information needed to represent the KD tree.

[0174] The transformation T is obtained by directly applying the elementary RAHT transformation to the two subvolumes node , the encoding and decoding processes described in this paper can be applied in KD-trees. The upsampling process is naturally performed in the direction perpendicular to the splitting plane.

[0175] More generally, the encoding and decoding process can be applied to any tree structure, as the upsampling process using weighted sums can be applied to any configuration of volumes and subvolumes. For example, all neighboring nodes within a fixed threshold distance from the current node can be used as predictors of the current node's occupied children for upsampling the mean attribute. The principles described above can be applied to other structures, attributes, transformations, etc., as long as the DC coefficient properties remain unchanged. The generalized weighted graph transformation can be applied to any tree and ensures the properties of the DC coefficient, thus providing another embodiment involving any tree structure.

[0176] Impact on compression performance

[0177] The described techniques can perform well for both dense and sparse point clouds. They can be compared with RAHT-based and LoD-based processes.

[0178] Figure 13 An example graph 1300 is shown for compression performance of one implementation in encoding YUV attributes of a dense point cloud. Graph 1300 indicates bits per point on the x-axis and peak signal-to-noise ratio (PSNR) on the y-axis. The graph includes the present process, RAHT, and LoD. It will be noted that on the combined YUV metric, embodiments of the present process perform at least as well as or better than LoD, and significantly better than RAHT.

[0179] Figure 14Another example graph 1400 is shown for one implementation in encoding reflectivity of a sparse point cloud. Graph 1400 again shows bits per point on the x-axis and PSNR on the y-axis. It will be noted that embodiments of this process perform roughly as well as RAHT and much better than LoD.

[0180] Complexity advantage of the proposed combined transform and upsampling method

[0181] The complexity of the proposed method is the sum of the complexity of the transformation process and the upsampling process. In some embodiments, due to the simplicity of the two-point RAHT transform and its recursive nature, the complexity of the transformation process can be kept very low. The upsampling process is very local in space, considering neighboring nodes. Nodes often share faces, edges, or vertices with the parent node whose attributes are to be upsampled.

[0182] In contrast, the LoD method is much more computationally demanding, especially for sparse point clouds, because it must find long-range attribute correlations to allow efficient attribute prediction. This means computationally intensive long-range neighbor search.

[0183] In the method described so far, the combination of transformation and upsampling automatically benefits from long-range correlation, because two points that are far away must belong to two adjacent nodes at a low enough depth (i.e., close to the root node). At this low enough depth, the transformation and upsampling ensure that the correlation between the two points is used.

[0184] Now refer to Figure 15 , which shows a simplified block diagram of an example embodiment of an encoder 1500. The decoder 1500 includes a processor 1502, a memory 1504, and an encoding application 1506. The encoding application 1506 may include a computer program or application stored in the memory 1504 and containing instructions that, when executed, cause the processor 1502 to perform operations such as those described herein. For example, the encoding application 1506 may encode and output a bitstream encoded according to the process described herein. It will be understood that the encoding application 1506 may be stored on a non-transitory computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard disk, etc. When the instructions are executed, the processor 1502 performs the operations and functions specified in the instructions, thereby operating as a dedicated processor that implements the described process(es). Such a processor may be referred to as a "processor circuit" or "processor circuit device" in some examples.

[0185] Now also refer to Figure 16, which shows a simplified block diagram of an example embodiment of a decoder 1600. Decoder 1600 includes a processor 1602, a memory 1604, and a decoding application 1606. Decoding application 1606 may include a computer program or application stored in memory 1604 and containing instructions that, when executed, cause processor 1602 to perform operations such as those described herein. It will be understood that decoding application 1606 may be stored on a computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard disk, etc. When the instructions are executed, processor 1602 performs the operations and functions specified in the instructions, thereby operating as a dedicated processor that implements the described process(es). Such a processor may be referred to as a "processor circuit" or "processor circuitry" in some examples.

[0186] It will be understood that the decoder and / or encoder according to the present application can be implemented in a variety of computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented in software that includes instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transient computer-readable memory, including CD, RAM, ROM, flash memory, etc.

[0187] It will be understood that the decoders and / or encoders described herein and the modules, routines, processes, threads, or other software components that implement the described methods / processes for configuring the encoders or decoders can be implemented using standard computer programming techniques and languages. The present application is not limited to a specific processor, computer language, computer programming environment, data structure, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of a computer executable code stored in a volatile or non-volatile memory, as part of an application specific integrated circuit (ASIC), or the like.

[0188] The present application also provides a computer-readable signal encoding data generated by applying the encoding process according to the present application.

[0189] Certain adaptations and modifications of the described embodiments may be made.Therefore, the embodiments discussed above are considered to be illustrative rather than restrictive.

Claims

1. A method for encoding a point cloud to generate a bitstream of compressed point cloud data, wherein the point cloud is located in a space that is recursively decomposed and contains points of the point cloud, each point having corresponding attributes, the method comprising: In a top-down encoding process of the recursively split space, for a volume containing sub-volumes containing a plurality of points, determining a predicted geometrically weighted sum of attributes for each occupied subvolume of the volume based on a prediction operation based at least on the geometrically weighted sum of attributes of the volume; Applying a transform to the predicted attribute geometric weighted sum to produce predicted coefficients, and applying the transform to the corresponding original attribute geometric weighted sum of the occupied subvolume to produce original coefficients, wherein the attribute geometric weighted sum is defined as where w node is the number of points in the subvolume at a particular depth of the space that is recursively split, and A node is the sum of the attribute values in a given node corresponding to said subvolume at said particular depth; determining a plurality of AC coefficients from differences between the original coefficients and the predicted coefficients, wherein residual coefficients include a DC coefficient and the plurality of AC coefficients; as well as The plurality of AC coefficients are encoded to output the bitstream of compressed point cloud data. 2 . The method of claim 1 , wherein the one DC coefficient is obtained from a geometrically weighted sum of the attributes of the volume.

3. The method according to claim 2, further comprising: The attribute geometrically weighted sum of the volume is determined by deriving the attribute geometrically weighted sum of the volume from an encoding of a parent volume, the volume being part of the parent volume.

4. The method according to any one of claims 1 to 3, further comprising: The attribute geometrically weighted sum is determined by summing the attributes of all points within the volume and dividing by the square root of the count of points within the volume.

5. The method of any one of claims 1 to 3, wherein the prediction operation is further based on a geometrically weighted sum of respective attributes of at least one neighboring volume, the at least one neighboring volume sharing at least one vertex with the volume. 6 . The method of claim 5 , wherein the prediction operation is based on upsampling the attribute geometric weighted sum of the volume and at least one attribute geometric weighted sum of the neighboring volumes.

7. The method of claim 6, wherein upsampling the attribute geometric weighted sum of the volume and at least one attribute geometric weighted sum of the neighboring volume comprises: Normalizing the attribute geometrically weighted sums by dividing by respective square roots of counts of points of the attribute geometrically weighted sums in the respective sums to obtain respective attribute mean sums, upsampling the respective attribute mean sums to generate a predicted attribute mean sum for each occupied subvolume, and denormalizing the predicted attribute mean sum for each occupied subvolume to obtain the predicted attribute geometrically weighted sum for each occupied subvolume of the volume.

8. A method according to any one of claims 1 to 3, wherein the transformation conforms to the DC coefficient property, so that the DC coefficient resulting from the transformation of the set of attribute geometric weighted sums of the child volumes of the volume is the attribute geometric weighted sum of the volume.

9. A method for decoding a bitstream of encoded attributes of a point cloud, the point cloud being located in a space recursively decomposed and containing points of the point cloud, each point having a corresponding attribute, the method comprising: In a top-down encoding process of the recursively split space, for a volume containing sub-volumes containing a plurality of points, determining a predicted geometrically weighted sum of attributes for each occupied subvolume of the volume based on a prediction operation based at least on the geometrically weighted sum of attributes of the volume; A transform is applied to the predicted attribute geometric weighted sum to produce predicted AC coefficients, where the attribute geometric weighted sum is defined as where w node is the number of points in the subvolume at a particular depth of the space that is recursively split, and A node is the sum of the attribute values in a given node corresponding to said subvolume at said particular depth; decoding the bitstream to reconstruct residual AC coefficients; setting a DC coefficient to a geometrically weighted sum of the attributes of the volume; adding the residual AC coefficients and the DC coefficients to the predicted AC coefficients to produce reconstructed coefficients; as well as performing an inverse transformation on the reconstructed coefficients to produce a geometrically weighted sum of the reconstructed attributes of the occupied subvolume, Thus at the maximum depth the geometrically weighted sum of the reconstructed properties of the occupied sub-volume is the reconstructed property.

10. The method according to claim 9, further comprising: The attribute geometric weighted sum of the volume is determined by obtaining the attribute geometric weighted sum of the volume from a decoding of a parent volume, the volume being part of the parent volume.

11. The method of claim 9 or 10, wherein the geometrically weighted sum of attributes of the volume is based on the sum of the attributes of all points located within the volume divided by the square root of the count of points located within the volume.

12. The method of claim 9 or 10, wherein the prediction operation is further based on a geometrically weighted sum of respective attributes of at least one neighboring volume that shares at least one vertex with the volume.

13. The method of claim 12, wherein the prediction operation is based on upsampling the geometrically weighted sum of the attributes of the volume and at least one of the geometrically weighted sums of the neighboring volumes.

14. The method of claim 13, wherein upsampling the geometrically weighted sum of the attributes of the volume and at least one of the geometrically weighted sums of the neighboring volumes comprises: Normalizing the attribute geometric weighted sums by dividing by respective square roots of counts of points of the attribute geometric weighted sums in the respective sums to obtain respective attribute mean sums, upsampling the respective attribute mean sums to generate a predicted attribute mean sum for each occupied subvolume, and denormalizing the predicted attribute mean sum for each occupied subvolume to obtain the predicted attribute geometric weighted sum for each occupied subvolume of the volume.

15. The method of claim 9 or 10, wherein the transformation conforms to a DC coefficient property such that a DC coefficient resulting from the transformation of the set of attribute geometrically weighted sums of the volume's child volumes is the attribute geometrically weighted sum of the volume.

16. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being located in a space recursively decomposed and containing points of the point cloud, each point having a corresponding attribute, the encoder comprising: Memory; at least one processor; a coded application stored in the memory and comprising processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: In a top-down encoding process of the recursively split space, for a volume containing sub-volumes containing a plurality of points, determining a predicted geometrically weighted sum of attributes for each occupied subvolume of the volume based on a prediction operation based at least on the geometrically weighted sum of attributes of the volume; applying a transform to the predicted attributed geometric weighted sum to produce predicted coefficients, and applying the transform to the corresponding original attributed geometric weighted sum of the occupied subvolume to produce original coefficients; A plurality of AC coefficients are determined from differences between the original coefficients and the predicted coefficients, wherein the residual coefficients include a DC coefficient and the plurality of AC coefficients, wherein the attribute geometric weighted sum is defined as where w node is the number of points in the subvolume at a particular depth of the space that is recursively split, and A node is the sum of the attribute values in a given node corresponding to said subvolume at said particular depth; as well as The plurality of AC coefficients are encoded to output a bit stream of compressed point cloud data.

17. The encoder of claim 16, wherein the one DC coefficient is obtained from a geometrically weighted sum of the attributes of the volume.

18. The encoder of claim 17, wherein the processor-executable instructions, when executed, further cause the at least one processor to: determine the attribute geometric weighted sum of the volume by obtaining the attribute geometric weighted sum of the volume from an encoding of a parent volume, the volume being part of the parent volume.

19. An encoder according to any one of claims 16 to 18, wherein the processor executable instructions, when executed, further cause the at least one processor to determine the attribute geometric weighted sum by summing the attributes of all points within the volume and dividing by the square root of the count of points within the volume.

20. An encoder according to any one of claims 16 to 18, wherein the prediction operation is further based on a geometrically weighted sum of respective attributes of at least one neighbouring volume that shares at least one vertex with the volume.

21. The encoder of claim 20, wherein the prediction operation is based on upsampling the attribute geometric weighted sum of the volume and at least one of the attribute geometric weighted sums of the neighboring volumes.

22. The encoder of claim 21 , wherein upsampling the attribute geometric weighted sum of the volume and at least one of the attribute geometric weighted sums of the neighboring volumes comprises: Normalizing the attribute geometrically weighted sums by dividing by respective square roots of counts of points of the attribute geometrically weighted sums in the respective sums to obtain respective attribute mean sums, upsampling the respective attribute mean sums to generate a predicted attribute mean sum for each occupied subvolume, and denormalizing the predicted attribute mean sum for each occupied subvolume to obtain the predicted attribute geometrically weighted sum for each occupied subvolume of the volume.

23. An encoder according to any one of claims 16 to 18, wherein the transform conforms to the DC coefficient property, such that a DC coefficient resulting from the transform of the set of attribute geometrically weighted sums of the volume's child volumes is the attribute geometrically weighted sum of the volume.

24. A decoder for decoding a bitstream of encoded attributes of a point cloud, the point cloud being located in a recursively decomposed space containing points in the point cloud, each point having a corresponding attribute, the decoder comprising: Memory; at least one processor; a coded application stored in the memory and comprising processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: In a top-down encoding process of the recursively split space, for a volume containing sub-volumes containing a plurality of points, determining a predicted geometrically weighted sum of attributes for each occupied subvolume of the volume based on a prediction operation based at least on the geometrically weighted sum of attributes of the volume; A transform is applied to the predicted attribute geometric weighted sum to produce predicted AC coefficients, where the attribute geometric weighted sum is defined as where w node is the number of points in the subvolume at a particular depth of the space that is recursively split, and A node is the sum of the attribute values in a given node corresponding to the subvolume at the specific depth; decoding the bitstream to reconstruct residual AC coefficients; setting a DC coefficient to a geometrically weighted sum of the attributes of the volume; adding the residual AC coefficients and the DC coefficients to the predicted AC coefficients to produce reconstructed coefficients; as well as performing an inverse transformation on the reconstructed coefficients to produce a geometrically weighted sum of the reconstructed attributes of the occupied subvolume, Thus at the maximum depth the geometrically weighted sum of the reconstructed properties of the occupied sub-volume is the reconstructed property.

25. The decoder of claim 24, wherein the processor-executable instructions, when executed, further cause the at least one processor to: determine the attribute geometric weighted sum of the volume by obtaining the attribute geometric weighted sum of the volume from a decoding of a parent volume, the volume being part of the parent volume.

26. The decoder of claim 24 or 25, wherein the geometrically weighted sum of attributes of the volume is based on the sum of the attributes of all points located within the volume divided by the square root of the count of points located within the volume.

27. A decoder according to claim 24 or 25, wherein the prediction operation is further based on a geometrically weighted sum of respective attributes of at least one neighbouring volume, the at least one neighbouring volume sharing at least one vertex with the volume.

28. The decoder of claim 27, wherein the prediction operation is based on upsampling the attribute geometric weighted sum of the volume and at least one of the geometric weighted sums of the neighboring volumes.

29. The decoder of claim 28, wherein upsampling the attribute geometric weighted sum of the volume and at least one of the geometric weighted sums of the neighboring volumes comprises: Normalizing the attribute geometric weighted sums by dividing by respective square roots of counts of points of the attribute geometric weighted sums in the respective sums to obtain respective attribute mean sums, upsampling the respective attribute mean sums to generate a predicted attribute mean sum for each occupied subvolume, and denormalizing the predicted attribute mean sum for each occupied subvolume to obtain the predicted attribute geometric weighted sum for each occupied subvolume of the volume.

30. A decoder according to claim 24 or 25, wherein the transform conforms to the DC coefficient property, such that a DC coefficient resulting from the transform of the set of attributed geometrically weighted sums of the volume's child volumes is the attributed geometrically weighted sum of the volume.

Images