Method for encoding and decoding a point cloud

Abstract

A method of encoding attributes of points of a point cloud to generate a bitstream of compressed point cloud data, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of: determining whether a first number of occupied nodes is greater than or equal to a first threshold; when the first number of occupied nodes is less than the first threshold, performing a first encoding of attributes of a current node, the first encoding not including a prediction process using second nodes comprising parent nodes of the current node and nodes belonging to a same level as the parent nodes; when the first number of occupied nodes is greater than or equal to the first threshold, performing a second encoding of attributes of the current node, the second encoding including the prediction process using the second nodes.

Description

TECHNICAL FIELD

[0001] The present application relates generally to compression of attributes of points of a point cloud. In particular, the present application relates to a method of encoding and decoding and an encoder and a decoder for improved coding of attributes of a point cloud. BACKGROUND

[0002] As a representation format of 3D data, point clouds have recently gained attraction due to their ability to flexibly represent all kinds of 3D objects or scenes. As a result, many use cases can be handled by point clouds, including:

[0003] • movie post-production,

[0004] • real-time 3D immersive telepresence or VR / AR applications,

[0005] • videos with free viewpoint (e.g. watching a sport event),

[0006] • geographic information systems (aka cartography),

[0007] • cultural heritage (storing scans of rare artifacts in digital form),

[0008] • autonomous driving, including 3D mapping of the environment and real-time laser radar data acquisition.

[0009] A point cloud is a collection of points in a 3D space, optionally with additional values attached to each of them. These additional values are often referred to as point attributes. Thus, a point cloud is a combination of geometry (3D position of each point) and attributes.

[0010] The attributes can be, for example, three-component colors, material attributes like reflectance, and / or two-component normal vectors of a surface associated with the point.

[0011] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, laser radars and scanners, or can be generated by computers (e.g. in movie post-production). Depending on the use case, a point cloud can have thousands to billions of points for mapping applications.

[0012] Each point in the raw representation of a point cloud requires a very high number of bits, with at least a dozen bits for each spatial component X, Y or Z. Optionally, more bits are used for attributes, like three times 10 bits for colors. Practical deployment of point cloud based applications requires compression techniques in order to store and distribute point clouds by reasonable storage and transmission infrastructure.

[0013] Compression can be lossy, such as video compression, to be distributed to end users and visualized by them, e.g. on AR / VR glasses or any other device with 3D capabilities. Other use cases do require lossless compression, e.g. medical applications or autonomous driving, to avoid changes in decision results due to analysis of the point cloud after compression and transmission.

[0014] Until recently, the point cloud compression (also called PCC) problem was not solved for the mass market and no standardized point cloud codec was available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also called Moving Picture Experts Group or MPEG, started a work item on point cloud compression. This led to two standards, namely:

[0015] • MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC), and

[0016] • MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-based Point Cloud Compression (G-PCC).

[0017] The V-PCC codec approach compresses point clouds by projecting them multiple times onto 3D objects to obtain 2D patches that are packed into images (or videos when dealing with moving point clouds). The resulting images or videos are then compressed using existing image / video codecs, which allows to leverage already deployed image and video solutions. By its nature, V-PCC is only efficient on dense and continuous point clouds, as image / video codecs cannot compress non-smooth patches, e.g. obtained from projections of sparse geometry data acquired from a LiDAR.

[0018] The G-PCC codec approach has two schemes for geometry compression.

[0019] The first scheme is based on an occupancy tree (octree / quadtrees / binary tree) representation of the point cloud geometry. Occupied nodes are split down until a certain size is reached and the occupied leaf nodes provide the positions of the points, usually at the center of these nodes. By using neighbor-based prediction techniques, a high level of compression can be obtained for dense point clouds.

[0020] The problem of sparse point clouds can also be handled by directly coding the positions of the points within non-minimal sized nodes by stopping the tree construction when there are only isolated points in a node; this technique is called direct coding mode (DCM).

[0021] The second approach is based on a prediction tree, where each node represents a 3D position of a point and the relationship between nodes is a spatial prediction from a parent node to a child node. This method can only handle sparse point clouds and has the advantage of lower latency and simpler decoding compared to the occupancy tree. However, it only slightly outperforms the first occupancy-based approach in terms of compression performance and is complex to encode, as it needs to find the best prediction factor among a long list of potential prediction factors when building the prediction tree.

[0022] In both approaches, the encoding / decoding of attributes is performed after the geometry encoding / decoding, resulting in a two-pass codec. Therefore, low latency is achieved by using slices that decompose the 3D space into sub-volumes that can be independently encoded / decoded. This can severely impact the compression performance when many slices are used.

[0023] One important use case is the transmission of LiDAR data acquired by moving vehicles. This often requires a simple low-latency on-board encoder. The need for simplicity is due to the fact that the encoder is likely to be deployed on a computing unit that performs other processing in parallel, such as (semi-)autonomous driving, thus limiting the processing power allocated to the point cloud encoder. Low latency is also required to allow fast transmission from the car to the cloud in order to allow real-time viewing of local traffic conditions based on multi-vehicle acquisition and to make appropriate and rapid decisions based on traffic information. Although the use of 5G can reduce transmission latency, the encoder itself should not introduce too much encoding / decoding latency. Moreover, compression performance should not be sacrificed, even though the data stream from millions of cars to the cloud is expected to be very heavy.

[0024] The combination of simplicity, low latency and compression performance for the encoder and decoder remains an unsatisfactory problem for current point cloud codecs.

[0025] Encoding point attributes based on already encoded geometry coordinates that are used to help de-correlate the attribute information according to the spatial relationship / distance between points. In G-PCC, there are mainly two methods for de-correlating and encoding attributes: the first one is called RAHT for Region Adaptive Hierarchical Transform and the second one uses one or more levels of detail that are sometimes referred to as LoD or predlift because it can be configured to work as a predictive de-correlation method or as a lifting-based de-correlation method. SUMMARY

[0026] In one aspect of the present application, a method of encoding properties of points of a point cloud to generate a bitstream of compressed point cloud data is provided, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of:

[0027] determining whether a first occupied node number is greater than or equal to a first threshold, the first occupied node number being a total number of occupied nodes, each of the occupied nodes comprising at least one three-dimensional point, the occupied nodes included in the first occupied node number being occupied child nodes of a current node in an N-ary tree structure of three-dimensional points included in the point cloud data, N being an integer greater than or equal to 2;

[0028] when the first occupied node number is less than the first threshold, performing a first encoding of properties of the current node, the first encoding not including a prediction process using third nodes, the third nodes including a parent node of the current node and nodes belonging to a same level as the parent node; and

[0029] when the first occupied node number is greater than or equal to the first threshold, performing a second encoding of properties of the current node, the second encoding including the prediction process using the third nodes.

[0030] In which a prediction coding can be used to encode the properties. According to the present application, before applying the prediction coding, it is first checked whether the total number of occupied child nodes is less than a first threshold. For example, if the total number of occupied child nodes is 1 and less than the first threshold, then the normal prediction coding will be disabled. In other words, the properties will be coded in another way, instead of implementing a prediction for 8 child nodes. According to the proposed encoding method, it can be appropriately chosen whether to use a prediction process for property encoding, so that the encoding efficiency can be improved.

[0031] Preferably, the first threshold is 2. Thus, in the case where there is only one occupied child node in the current node, the transformation of the original property values and the predicted property values only results in one DC coefficient, and no AC coefficients, so that no coding of AC coefficient residuals is needed. This avoids unnecessary time-consuming prediction processes, and further improves the coding efficiency by providing such an appropriate first threshold.

[0032] Preferably, the method further comprises: determining whether a second occupied node number is greater than or equal to a second threshold value before performing the second encoding on the attribute of the current node, the second occupied node number being a total number of occupied nodes included in a second node, the second node including a grandparent node of the current node and nodes belonging to a same layer as the grandparent node; performing the first encoding on the attribute of the current node and skipping the second encoding when the second occupied node number is less than the second threshold value; searching for nodes belonging to a same layer as a parent node of the current node when the second occupied node number is greater than or equal to the second threshold value; calculating a third occupied node number, the third occupied node number being a total number of occupied nodes included in a third node, the third node including the parent node of the current node and nodes belonging to a same layer as the parent node; determining whether the third occupied node number is greater than or equal to a third threshold value; performing the first encoding on the attribute of the current node and skipping the second encoding when the third occupied node number is less than the third threshold value; performing the second encoding on the attribute of the current node when the third occupied node number is greater than or equal to the third threshold value.

[0033] In this way, the whole process of determining whether to apply the predictive encoding (i.e. the second encoding) is further optimized. Specifically, firstly, it is checked whether the number of occupied child nodes is large enough (i.e. greater than or equal to the first threshold value), and only when this number is large enough, the judgment of whether to apply the predictive encoding is further performed. Secondly, it is checked whether the number of occupied grandparent neighbor nodes is greater than or equal to the second threshold value. If yes, the parent neighbor nodes are searched and the number of occupied parent neighbor nodes is counted. Then, if the number of occupied parent neighbor nodes is greater than or equal to the third threshold value, the predictive encoding is applied. Therefore, in other words, if the number of occupied grandparent neighbor nodes or the number of occupied parent neighbor nodes is less than the corresponding threshold value, the time-consuming attribute prediction will be terminated in advance. The application of the two additional condition settings (i.e. the second threshold value and the third threshold value) together with the check of the first condition (i.e. the first threshold value) produces a synergistic technical effect, i.e. the process of predictive encoding can be terminated at an earlier stage to avoid the time-consuming parent node search when the first condition is not satisfied. Therefore, the whole technical solution provides an overall optimal encoding process, and the predictive encoding is applied only when necessary.

[0034] Preferably, when the first occupied node number is less than the first threshold value, the third occupied node number is set to be greater than the second threshold value, preferably, the third occupied node number is set to be 19.

[0035] wherein, when the first occupied nodes number is less than the first threshold, the search for the parent node is skipped. Thus, the default count of parent nodes can be 0. Even so, by setting the third occupied nodes number to be greater than the second threshold, the prediction of the children is not forced to terminate. Preferably, the third occupied nodes number is set to 19, which guarantees that this number is higher than the second threshold, since the maximum number of neighbor nodes (nodes sharing a face or an edge with the current node) is 18.

[0036] In one aspect of the application, a method of decoding a bitstream of compressed point cloud data to generate attributes of points in a reconstructed point cloud is provided, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of:

[0037] determining whether a first occupied nodes number is greater than or equal to a first threshold, the first occupied nodes number being a total number of occupied nodes, each of the occupied nodes comprising at least one three-dimensional point, the occupied nodes comprised in the first occupied nodes number being occupied children nodes of a current node in an N-ary tree structure of three-dimensional points comprised in the point cloud data, N being an integer greater than or equal to 2;

[0038] when the first occupied nodes number is less than the first threshold, performing a first decoding of attributes of the current node, the first decoding not comprising a prediction process using third nodes, the third nodes comprising parent nodes of the current node and nodes belonging to the same level as the parent nodes; and

[0039] when the first occupied nodes number is greater than or equal to the first threshold, performing a second decoding of attributes of the current node, the second decoding comprising the prediction process using third nodes.

[0040] Preferably, the decoding method is further built according to the features described above for the encoding method, which features can be freely combined with the decoding method.

[0041] In one aspect of the application, an encoder for encoding a point cloud to generate a bitstream of compressed point cloud data is provided, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the encoder comprising:

[0042] a processor, and

[0043] a memory storage device, wherein the memory storage device has stored therein executable instructions of a processor that, when executed, cause the processor to perform the method according to the above encoding method.

[0044] In one aspect of the application, a decoder is provided for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the decoder comprising:

[0045] a processor, and

[0046] a memory storage device, wherein the memory storage device has stored therein executable instructions of a processor that, when executed, cause the processor to perform the method according to the above decoding method.

[0047] In one aspect of the application, a non-transitory computer readable storage medium is provided for storing instructions for execution by a processor that, when executed by the processor, cause the processor to perform the above encoding and / or decoding method. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 for an embodiment of the encoding method according to the application,

[0049] Figure 2 for an embodiment of the decoding method according to the application,

[0050] Figure 3 for an example of transform domain prediction and parameter definition according to the application,

[0051] Figure 4 for a detailed embodiment of the application,

[0052] Figure 5 for a schematic diagram of an encoder apparatus, and

[0053] Figure 6 for a schematic diagram of a decoder apparatus. DETAILED DESCRIPTION

[0054] The present application describes an encoding method and an encoder for encoding attributes of points in a point cloud, and a decoding method and a decoder for decoding a bitstream into attributes of points in a point cloud.

[0055] The present application relates to a method of encoding attributes of points of a point cloud to generate a bitstream of compressed point cloud data, wherein the point cloud geometry is represented by a voxel-based structure having a plurality of nodes in parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of: determining whether a first number of occupied nodes is greater than or equal to a first threshold, the first number of occupied nodes being a total number of occupied nodes, each of said occupied nodes comprising at least one three-dimensional point, the occupied nodes comprised in the first number of occupied nodes being occupied children nodes of a current node in an N-ary tree structure of three-dimensional points comprised in the point cloud data, N being an integer greater than or equal to 2; when the first number of occupied nodes is less than the first threshold, performing a first encoding of attributes of the current node, the first encoding not comprising a prediction process using third nodes, the third nodes comprising a parent node of the current node and nodes belonging to a same level as the parent node; and when the first number of occupied nodes is greater than or equal to the first threshold, performing a second encoding of attributes of the current node, the second encoding comprising a prediction process using third nodes.

[0056] Other aspects and features of the present application will become apparent to those of ordinary skill in the art upon reading the following description of the embodiments in conjunction with the accompanying drawings.

[0057] In the following description, the terms "node" and "sub-volume" are sometimes used interchangeably. It can be understood that a node is associated with a sub-volume. A node is a particular point on the tree, which can be an internal node or a leaf node. A sub-volume is the bounded physical space represented by a node. The term "volume" can be used to refer to the largest bounded space defined for containing the point cloud. The volume is recursively divided into sub-volumes to build the tree structure of interconnected nodes for coding and decoding the point cloud data. Furthermore, the term "parent node" refers to a node at the next higher level of the tree. A node can be at a level or depth D in the tree, while a parent node is a node at level or depth D-1.

[0058] A point cloud is a set of points in a three-dimensional coordinate system. These points are generally intended to represent the outer surface of one or more objects. Each point has a position in the three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be a Cartesian coordinate system or any other coordinate system. The points have further associated attributes, such as color, which in some cases can also be a three-component value, such as R, G, B or Y, Cb, Cr. Other associated attributes can include transparency, reflectance, normal vector, etc., depending on the desired application of the point cloud data.

[0059] A point cloud can be static or dynamic. For example, a detailed scan or mapping of an object or terrain can be static point cloud data. Laser-radar based scans of an environment for machine vision purposes can be dynamic, as the point cloud (at least potentially) changes over time, e.g., for each successive scan of a volume. Dynamic point clouds are thus temporally ordered sequences of point clouds.

[0060] Point cloud data can be used in many applications, including preservation (scanning of historical or cultural artifacts), mapping, machine vision (such as self-driving or semi- autonomous cars), and virtual reality systems, to give some examples. Dynamic point cloud data for applications such as machine vision can be quite different from static point cloud data, such as for preservation purposes. For example, car vision typically involves relatively small resolution, non-color, and highly dynamic point clouds obtained through laser-radar (or similar) sensors with high capture frequency. The purpose of such point clouds is not for human use or viewing, but for machine object detection / classification in a decision process. As an example, a typical laser-radar frame contains on the order of tens of thousands of points, while high-quality virtual reality applications require millions of points. It can be expected that there will be a demand for higher resolution data over time as computing speeds increase and new applications are discovered.

[0061] While point cloud data is useful, a lack of efficient and effective compression, i.e., encoding and decoding processes, of the properties and geometry of such point clouds can hinder adoption and deployment.

[0062] One of the more common mechanisms for coding point cloud data is through the use of tree-based structures. In a tree-based structure, the bounding three-dimensional volume of a point cloud is recursively divided into sub-volumes. The nodes of the tree correspond to the sub-volumes. The decision of whether to further divide a sub-volume can be based on the resolution of the tree and / or whether there are any points contained in the sub-volume. Leaf nodes can have an occupancy flag that indicates whether its associated sub-volume contains points. A split flag can signal whether a node has child nodes (i.e., whether the current volume has been further partitioned into sub-volumes). These flags can be entropy coded in some cases and prediction coded in some cases. A commonly used tree structure is an octree. In this structure, the volumes / sub-volumes are all cubes, and each partitioning of a sub-volume results in eight additional sub-volumes / sub-cubes.

[0063] The basic process of creating an octree to code a point cloud can include:

[0064] Starting with a bounding volume (cube) containing the point cloud in one coordinate system;

[0065] 1. Partitioning the volume into 8 sub-volumes (eight sub-cubes);

[0066] 2. For each child, if the child is empty, marking the child as 0, or if there is at least one point in the child, marking the child as 1 ;

[0067] 3. For all children marked as 1, repeating (2) to split those children until a maximum split depth is reached; and

[0068] 4. For all leaf children (sub-cubes) of maximum depth, marking the leaf cube as 1 if it is non-empty, or 0 otherwise.

[0069] The tree can be traversed in a predefined order (breadth-first or depth-first, and according to a scan pattern / order within each split child) to produce a bit sequence representing the occupancy pattern of each node.

[0070] As mentioned above, points in the point cloud can comprise attributes. These attributes are coded independently of the coding of the point cloud geometry. Thus, each occupied node, i.e. a node comprising at least one point of the point cloud, is associated with one or more attributes in order to further specify the attributes of the point cloud.

[0071] The present application provides a method for encoding attributes of points in a point cloud. The method is as shown in Figure 1

[0072] A method of encoding attributes of points of a point cloud to generate a bitstream of compressed point cloud data, wherein the point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively splitting a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of:

[0073] S01: determining whether a first occupied node number is greater than or equal to a first threshold, the first occupied node number being a total number of occupied nodes, each occupied node comprising at least one three-dimensional point, the occupied nodes included in the first occupied node number being occupied child nodes of a current node in an N-ary tree structure of three-dimensional points included in the point cloud data, N being an integer greater than or equal to 2;

[0074] S02: when the first occupied node number is less than the first threshold, performing a first encoding of attributes of the current node, the first encoding not including a prediction process using third nodes, the third nodes including a parent node of the current node and nodes belonging to a same level as the parent node;

[0075] S03: when the first occupied node number is greater than or equal to the first threshold, performing a second encoding of attributes of the current node, the second encoding including a prediction process using third nodes.

[0076] ​According to step S01, it is first checked whether the total number of occupied child nodes is less than a first threshold. If it is less than the first threshold, the attribute is encoded in a way other than predictive coding. Only when it is greater than or equal to the first threshold, predictive coding can be performed. Thus, it can be appropriately selected whether to use the prediction process for attribute encoding, and if not needed, the prediction process can be terminated at an early stage, thereby improving the coding efficiency.

[0077] In one embodiment, the first threshold is 2. Thus, if there is only one occupied child node in the current node, the transformation of the original attribute value and the predicted attribute value only produces one DC coefficient and no AC coefficients, and thus there is no need to encode the AC coefficient residual. This avoids unnecessary time-consuming prediction process, and further improves the coding efficiency by providing such appropriate first threshold. The details of the prediction process are well known in the art. For example, it can be known from “G-PCC CE13.18 report on upsampled transform domain prediction in RAHT, ISO / IEC JTC1 / SC29 WG11 Doc. M49380, Gothenburg, SE, July 2019”, which is incorporated herein by reference.

[0078] In one embodiment, the method further comprises: before performing the second encoding on the attribute of the current node, determining whether a second occupied node number is greater than or equal to a second threshold, the second occupied node number being a total number of occupied nodes included in a second node, the second node including an ancestor node of the current node and a node belonging to the same layer as the ancestor node; when the second occupied node number is less than the second threshold, performing the first encoding on the attribute of the current node and skipping the second encoding; when the second occupied node number is greater than or equal to the second threshold, finding a node belonging to the same layer as a parent node of the current node; calculating a third occupied node number, the third occupied node number being a total number of occupied nodes included in a third node, the third node including the parent node of the current node and the node belonging to the same layer as the parent node; determining whether the third occupied node number is greater than or equal to a third threshold; when the third occupied node number is less than the third threshold, performing the first encoding on the attribute of the current node and skipping the second encoding; when the third occupied node number is greater than or equal to the third threshold, performing the second encoding on the attribute of the current node.

[0079] Thus, the whole process of determining whether to apply the prediction coding (i.e. the second encoding) is further optimized. Specifically, firstly, it is checked whether the number of occupied child nodes is large enough (i.e. larger than or equal to the first threshold), and only when the number is large enough, the judgment of whether to apply the prediction coding is further performed. Secondly, it is checked whether the number of occupied grandparent neighbor nodes is larger than or equal to the second threshold. If yes, the parent neighbor nodes are searched and the number of occupied parent neighbor nodes is counted. Then, if the number of occupied parent neighbor nodes is larger than or equal to the third threshold, the prediction coding is applied. Thus, in other words, if the number of occupied grandparent neighbor nodes or the number of occupied parent neighbor nodes is smaller than the corresponding threshold, the time-consuming attribute prediction will be terminated in advance. The two additional conditions (i.e. the second threshold and the third threshold) applied together with the check of the first condition (i.e. the first threshold) produce a synergistic technical effect that the process of prediction coding can be terminated at an earlier stage to avoid the time-consuming parent node search when the first condition is not satisfied. Thus, the whole technical solution provides an overall optimal encoding process, and the prediction coding is applied only when necessary.

[0080] Referring to Figure 3 , an example of the transform domain prediction and the partial parameter definition of the present application is shown. In which, the occupied nodes (i.e. valid nodes) are shown in shade, and the unoccupied nodes are transparent. Thus, in this example, the number of occupied nodes in the grandparent level (i.e. the grandparent level of the target 8 child nodes to be encoded) NumValidNGP (i.e. the second occupied node number) is 2. The number of occupied nodes in the parent level (i.e. the parent level of the target 8 child nodes to be encoded) NumValidNP (i.e. the third occupied node number) is 11. The corresponding thresholds TH1 (i.e. the second threshold) and TH2 (i.e. the third threshold) are set to 2 and 6 respectively. Thus, when the number of occupied child nodes is larger than or equal to the first threshold, the other two conditions of the grandparent nodes and the parent nodes are also satisfied. Thus, the encoding process continues with the prediction coding, and the attribute values of the parent nodes are used for prediction. From Figure 3 As can be seen from the lower left subgraph, the occupied parent nodes are shown in shade, which are used to predict the 8 target child nodes shown on the right. The details of the prediction are well known in the art. For the sake of brevity, the technique is not described in detail here, because the gist of the present application is to selectively apply the prediction coding technique rather than the prediction coding itself.

[0081] Referring to Figure 4, which shows a detailed embodiment of the present application. If NumValidC (i.e., the number of occupied child nodes of the current node) is equal to 1, attribute prediction is disabled. In this case, the value of NumValidP can have a default value (e.g., zero) since the parent neighbor search is skipped. This can cause the prediction of the child to be forced to terminate. To avoid this, NumValidP (i.e., the number of occupied parent nodes) is set to a value greater than TH1 (i.e., the second threshold), e.g., Val = 19 in the proposed method when NumValidC = 1.

[0082] The present application also provides a method of decoding attributes of points of a point cloud from a bitstream, the method being as shown in Figure 2

[0083] A method of decoding a bitstream of compressed point cloud data to generate attributes of points of a point cloud, wherein the point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of:

[0084] S10: determining whether a first occupied node number is greater than or equal to a first threshold, the first occupied node number being a total number of occupied nodes, each occupied node comprising at least one three-dimensional point, the occupied nodes included in the first occupied node number being occupied child nodes of a current node in an N-ary tree structure of three-dimensional points included in the point cloud data, N being an integer greater than or equal to 2;

[0085] S11: when the first occupied node number is less than the first threshold, performing a first decoding on attributes of the current node, the first decoding not including a prediction process using third nodes, the third nodes including parent nodes of the current node and nodes belonging to the same layer as the parent nodes;

[0086] S12: when the first occupied node number is greater than or equal to the first threshold, performing a second decoding on attributes of the current node, the second decoding including a prediction process using third nodes.

[0087] In some embodiments, the decoding method is further constructed according to the embodiments described above for the encoding method, and these features can be freely combined with the decoding method.

[0088] The different embodiments described above can be freely combined. In particular, the thresholds can be freely selected and combined to meet the needs of a specific implementation.

[0089] ​The simulation is performed on the TMC13v14 platform, and the results under C1 (lossless-geom-lossy-attrs) and C2 (lossy-geom-lossy-attrs) conditions are evaluated. The results show that the method proposed in the application can significantly reduce the encoding / decoding time, while having no impact on the performance.

[0090] Reference is now made to Figure 5 which shows a simplified block diagram of an example embodiment of an encoder 1100. The encoder 1100 includes a processor 1102 and a memory storage 1104. The memory storage 1104 can store a computer program or application containing instructions that, when executed, cause the processor 1102 to perform operations such as those described herein. For example, the instructions can encode and output a bitstream encoded according to the methods described herein. It will be appreciated that the instructions can be stored on a non-transitory computer readable medium such as an optical disc, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions in order to operate as a specialized processor implementing the described processes. In some examples, such a processor can be referred to as a “processor circuit” or “processor circuitry”.

[0091] Reference is now also made to Figure 6 which shows a simplified block diagram of an example embodiment of a decoder 1200. The decoder 1200 includes a processor 1202 and a memory storage 1204. The memory storage 1204 can include a computer program or application containing instructions that, when executed, cause the processor 1202 to perform operations such as those described herein. It will be appreciated that the instructions can be stored on a computer readable medium such as an optical disc, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions in order to operate as a specialized processor implementing the described processes and methods. In some examples, such a processor can be referred to as a “processor circuit” or “processor circuitry”.

[0092] It can be appreciated that a decoder and / or encoder according to the present application can be implemented in a number of computing devices, including but not limited to servers, suitably programmed general purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer readable memory, including CDs, RAM, ROM, flash memory, etc.

[0093] It should be appreciated that the decoders and / or encoders described herein, as well as the modules, routines, processes, threads or other software components implementing the described methods / processes for configuring an encoder or decoder, can be implemented using standard computer programming techniques and languages. The present application is not limited to a particular processor, computer language, computer programming conventions, data structures, and other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as a part of computer-executable code stored in volatile or non-volatile memory, as part of an application specific integrated circuit (ASIC), and so on.

[0094] Certain adjustments and modifications can be made to the described embodiments. Therefore, the above-discussed embodiments are to be considered illustrative and not restrictive, in particular, the embodiments can be freely combined with each other.

Claims

1. A method of encoding attributes of points of a point cloud to generate a bitstream of compressed point cloud data, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of: determining whether a first occupied node number is greater than or equal to a first threshold, the first occupied node number being a total number of occupied nodes, each of the occupied nodes comprising at least one three-dimensional point, the occupied nodes comprised in the first occupied node number being occupied children of a current node in an N-ary tree structure of three-dimensional points comprised in the point cloud data, N being an integer greater than or equal to 2; when the first occupied node number is less than the first threshold, performing a first encoding of attributes of the current node, the first encoding not comprising a prediction process using third nodes, the third nodes comprising a parent node of the current node and nodes belonging to a same level as the parent node; and when the first occupied node number is greater than or equal to the first threshold, performing a second encoding of attributes of the current node, the second encoding comprising the prediction process using third nodes.

2. The method of claim 1, wherein, the first threshold being 2.

3. The method of any one of claims 1 or 2, further comprising: before performing the second encoding of attributes of the current node, determining whether a second occupied node number is greater than or equal to a second threshold, the second occupied node number being a total number of occupied nodes comprised in a second node, the second node comprising a grandparent node of the current node and nodes belonging to a same level as the grandparent node; when the second occupied node number is less than the second threshold, performing the first encoding of attributes of the current node and skipping the second encoding; when the second occupied node number is greater than or equal to the second threshold, finding nodes belonging to a same level as a parent node of the current node; computing a third occupied node number, the third occupied node number being a total number of occupied nodes comprised in the third nodes; determining whether the third occupied node number is greater than or equal to a third threshold; when the third occupied node number is less than the third threshold, performing the first encoding of attributes of the current node and skipping the second encoding; when the third occupied node number is greater than or equal to the third threshold, performing the second encoding of attributes of the current node.

4. The method of claim 3, wherein, when the first occupied node number is less than the first threshold, setting the third occupied node number to be greater than the second threshold.

5. The method of claim 4, wherein, setting the third occupied node number to be 19.

6. A method of decoding a bitstream of compressed point cloud data to generate attributes of points of a point cloud, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the method comprising the steps of: determining whether a first occupied node number is greater than or equal to a first threshold, the first occupied node number being a total number of occupied nodes, each of the occupied nodes comprising at least one three-dimensional point, the occupied nodes included in the first occupied node number being occupied children nodes of a current node in an N-ary tree structure of three-dimensional points included in point cloud data, N being an integer greater than or equal to 2; when the first occupied node number is less than the first threshold, performing a first decoding on properties of the current node, the first decoding not including a prediction process using third nodes, the third nodes including a parent node of the current node and nodes belonging to a same level as the parent node; and when the first occupied node number is greater than or equal to the first threshold, performing a second decoding on the properties of the current node, the second decoding including the prediction process using the third nodes.

7. The method of claim 6, wherein, The first threshold is 2.

8. The method of any one of claims 6 or 7, further comprising: before performing the second decoding on the properties of the current node, determining whether a second occupied node number is greater than or equal to a second threshold, the second occupied node number being a total number of occupied nodes included in a second node, the second node including a grandparent node of the current node and nodes belonging to a same level as the grandparent node; when the second occupied node number is less than the second threshold, performing the first decoding on the properties of the current node and skipping the second decoding; when the second occupied node number is greater than or equal to the second threshold, finding nodes belonging to a same level as a parent node of the current node; calculating a third occupied node number, the third occupied node number being a total number of occupied nodes included in the third nodes; determining whether the third occupied node number is greater than or equal to a third threshold; when the third occupied node number is less than the third threshold, performing the first decoding on the properties of the current node and skipping the second decoding; when the third occupied node number is greater than or equal to the third threshold, performing the second decoding on the properties of the current node.

9. The method of claim 8, wherein, When the first occupied node number is less than the first threshold, the third occupied node number is set to be greater than the second threshold.

10. The method of claim 9, wherein, The third occupied node number is set to be 19.

11. An encoder configured to encode properties of points of a point cloud to generate a bitstream of compressed point cloud data, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the encoder comprising: a processor, and a memory storage having stored therein executable instructions of the processor that, when executed, cause the processor to perform the method of any one of claims 1 to 5.

12. A decoder configured to decode a bitstream of compressed point cloud data to generate attributes of points of a reconstructed point cloud, wherein a point cloud geometry is represented by a voxel-based structure having a plurality of nodes in a parent-child relationship, by recursively partitioning a volume space containing the point cloud into sub-volumes each associated with a node of the voxel-based structure, the decoder comprising: a processor, and a memory storage having stored therein executable instructions of a processor that, when executed, cause the processor to perform the method according to any one of claims 6 to 10.

13. A non-transitory computer-readable storage medium having stored therein processor-executable instructions that, when executed, cause a processor to perform the method according to any one of claims 1 to 10.

Images