Method, encoder and decoder for encoding and decoding 3d point clouds

Abstract

An encoding method and a decoding method, as well as an encoder and a decoder, for encoding a 3D point cloud into a bitstream. The geometry of the point cloud is defined by an octree structure comprising a plurality of nodes having a parent-child relationship and representing the three-dimensional position of an object, the point cloud being located within a volumetric space of a three-dimensional coordinate system, the volumetric space being segmented step by step into sub-volumes and containing points of the point cloud, wherein the volume is divided into a number of sub-volumes, each sub-level sub-volume being associated with one of the plurality of nodes of the octree structure, and wherein occupancy information associated with each of the sub-level sub-volumes indicates whether the respective sub-level sub-volume contains at least one of the points of the point cloud, the method comprises the following steps: along each axis X, Y and Z of a coordinate system, acquiring nodes which contain at least a part of point clouds and have the depth of d in a dictionary type sorting manner; occupancy information of child nodes of each node is entropy decoded from the bitstream based on the dictionalized ranking.

Description

Technical Field

[0001] The present application relates to a method for encoding a 3D point cloud into a bitstream. In addition, the object of the present application is to provide a method for decoding a 3D point cloud from a bitstream. In addition, the object of the present application is to provide an encoder and a decoder, a bitstream encoded according to the present application, and software. In particular, the object of the present application is to provide a method for improving coding efficiency and reducing processing delay. Background Art

[0002] Point cloud, as a format for representing 3D data, has attracted much attention due to its versatile ability to represent all types of 3D objects or scenes. As a result, point cloud can handle a variety of usage scenarios, including:

[0003] Film post-production,

[0004] Real-time 3D immersive telepresence or VR (virtual reality) / AR (augmented reality) applications,

[0005] Free viewpoint video (e.g. for watching sports events),

[0006] Geographic Information Systems (also known as Cartography),

[0007] Cultural heritage (scanning and storing digitally rare objects),

[0008] Autonomous driving, including 3D mapping of the environment and real-time lidar data collection.

[0009] A point cloud is a set of points located in 3D space, each of which can have values ​​attached to it. These values ​​are often called point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0010] These attributes may be, for example, three-component color, material properties (such as reflectivity), and / or a two-component normal vector of the surface associated with the point.

[0011] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, Lidar, scanners, or they can be computer-generated (for example, in movie post-production). Depending on the use case, point clouds in mapping applications may contain thousands to billions of points.

[0012] In the raw representation of point clouds, each point requires a very high number of bits, at least a dozen bits per spatial component X, Y or Z, and optionally more bits for attributes, such as three times 10 bits for color. Practical deployment of point cloud-based applications requires compression techniques that allow point clouds to be stored and distributed within reasonable storage and transmission infrastructure conditions.

[0013] The compression may be lossy (similar to lossy video compression) during the distribution of point clouds to end users and visualization by the user terminal, such as on AR / VR glasses, or any other 3D-enabled device. Other use cases require lossless compression, such as medical applications or autonomous driving, to avoid compromising the decision-making results obtained by analyzing the compressed and transmitted point cloud.

[0014] Until recently, point cloud compression (also known as PCC) has not been addressed by the mass market and no standardized point cloud codecs were available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11 (also known as the Moving Picture Experts Group or MPEG) has started a work item on point cloud compression.

[0015] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC); and

[0016] MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-based Point Cloud Compression (G-PCC).

[0017] The V-PCC and G-PCC standards have completed their first versions at the end of 2020 and will soon be available in the market.

[0018] The V-PCC coding method compresses point clouds by performing multiple projections of 3D objects to obtain 2D patches that are packed into images (or videos, for mobile point clouds). The obtained images or videos are then compressed using existing image / video codecs, making it possible to use already deployed image and video solutions. By its nature, V-PCC is only effective on dense and continuous point clouds, as image / video codecs cannot compress non-smooth patches obtained from the projection of sparse geometric data collected, such as Lidar.

[0019] The G-PCC coding method has two schemes for compression geometry.

[0020] The first scheme is based on an occupancy tree (octree / quadtree / binarytree) representation of the point cloud geometry. Occupied nodes are forked until a certain size is reached, at which point the occupied leaf nodes provide the positions of the points, usually the centers of these nodes. A high degree of compression can be obtained for dense point clouds by using neighborhood-based prediction techniques. Sparse point clouds are also addressed by directly encoding the positions of points within nodes with non-minimum sizes, by stopping the construction of the tree when there are only isolated points in the node; this technique is called Direct Coding Mode (DCM).

[0021] The second scheme is based on a prediction tree, where each node represents the 3D position of a point and the relationship between nodes is the spatial prediction from parent to child nodes. This method can only solve sparse point clouds and has the advantages of lower latency and simpler decoding than occupancy trees. However, the compression performance is only slightly better than the first occupancy-based scheme and has complex encoding, intensively searching for the best predictor (in a long list of potential subvolume predictors) when building the prediction tree.

[0022] In both schemes, attribute encoding (de)coding is performed after geometry encoding (de)coding is completed, resulting in two-pass coding. Therefore, low latency is achieved by using slices that decompose the 3D space into independently coded sub-volumes without prediction between sub-volumes. When many slices are used, compression performance may be severely affected.

[0023] An important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves with respect to time. In addition, AR / VR point clouds are usually locally 2D, as they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly correlated (or dense), in the sense that points in them are rarely isolated, but have many neighbors.

[0024] In GPCC, although the neighbors are reduced to close neighbors, the number of possible configurations is still too high to be directly used for selecting the context to be used for entropy encoding of the current node occupancy information.

[0025] Several techniques have been proposed to further reduce the neighborhood configurations to an achievable number of reduced configurations.

[0026] On the first aspect, some "physical" arguments are introduced, such as another occupied neighboring node placed between an occupied neighboring node and the current node shielding the occupied neighboring node; in this case, the information of the occupied neighboring node is discarded because the information should be weaker than the information of the other occupied neighboring node mentioned above.

[0027] Secondly, based on learning, a reduction method using a look up table (LUT) is proposed.

[0028] Reduced configuration = LUT[configuration]

[0029] The output range of the LUT is smaller than the number of entries. The construction of the LUT depends largely on the type of learning point cloud. During the occupancy encoding of the child nodes, the LUT is used. The LUT is a lookup table based on the encoder index of the occupancy information configuration of the currently encoded child node, and it maps the occupancy information configuration to select the encoder index, and the selected binary encoder is used to encode the current child node.

[0030] Thirdly, a more flexible technique called "Optimal Binary Coder with Update on the Fly (OBUF)" has been introduced to reduce the number of contexts of entropy encoders such as CABAC (Context-based Adaptive Binary Arithmetic Coding) by:

[0031] Use a limited number of adaptive entropy encoders (e.g. 32)

[0032] Associate an encoder index (e.g. 0 to 31) with each neighborhood configuration

[0033] After each encoding of the occupancy of the current node, update the encoder index associated with the neighborhood configuration of the current node.

[0034] OBUF is combined with the first two reduction techniques applied before OBUF to limit the number of encoder indices that depend on the reduction configuration.

[0035] A practical implementation is also proposed that allows entropy coding of the occupancy bits of the current geometry element by using pre-coded occupancy information of neighboring nodes, as follows:

[0036] Use easy-to-compute, low-memory reduction functions to reduce neighborhood occupancy configurations;

[0037] Use the reduced configuration as input to the LUT pointing to the small part entropy encoder; update the LUT after the encoding of the occupied bits. This is the so-called OBUF process in the prior art;

[0038] • Entropy encode the occupied bits using the entropy encoder pointed to by the entry of the LUT indicated by the reduction configuration.

[0039] This process works very well once a sufficient number of occupied bits have been encoded so that the LUT has been updated a sufficient number of times to converge and consistently point to an entropy encoder whose coding probabilities are well correlated with the reduced configuration.

[0040] However, when the point cloud is small or at the beginning of the encoding process for a large point cloud, the OBUF LUT has not yet converged and cannot select an adequate entropy encoder. This results in poor compression for small point clouds or compression degradation for large point clouds.

[0041] In a 3D octree representation of the geometry of a point cloud, the number of possible occupancy configurations of a neighborhood is 2N, where N is the number of nodes / voxels involved in the neighborhood. This number becomes very large even when a small number of N neighbors are involved.

[0042] Therefore, it is proposed to add an extra step of downscaling after the (optional) fixed downscaling function. This new downscaling will be called "dynamic downscaling" which is updated according to the encoding progress of the point cloud.

[0043] A typical example of the order of magnitude of the context information involved when processing neighboring nodes in an octree representing point cloud geometry is as follows.

[0044] There are 26 neighboring nodes that share faces, edges or vertices with the parent node of the current node, and the occupied bits of the current node are to be encoded. When using the breadth-first scan of the octree, all the occupied bits of these 26 neighboring nodes have been encoded and can be used as context information.

[0045] Depending on the scan order (Morton order of raster scan order), most of the time, the occupied bits of the child nodes of at least 7 of the 26 neighboring nodes have been encoded. This increases 8*7=56 occupied bits that can be used as context information.

[0046] Finally, some of the sibling nodes of the current node may have been encoded. There are at most 7 of them. Therefore, by considering only the closest neighboring nodes that touch the current node, a total of up to 26+56+7=89 already encoded occupied bits can be used as context information. This leads to 2 89 ≈1000 9 =10 27 possible neighborhood configurations are used as context information.

[0047] When encoding a point cloud, not all of these configurations are visited. However, they must be maintained to cover a wide variety of point clouds. For example, encoding an AR / VR dense point cloud will result in visiting some reduced context configurations, but encoding a Lidar sparse point cloud will result in visiting other reduced context configurations. This is the main goal of dynamic reduction: further reduce configurations that are rarely or never visited, and give full potential (i.e., no reduction or slight reduction) to frequently visited configurations. The dynamicity comes from the on-the-fly determination of frequently visited configurations during encoding and the updating of the dynamic reduction function based on that determination.

[0048] So basically, the dynamically reduced function DR is initialized with the initial function DR0 which has the maximum reduction of all configurations and then updated to DR 1 ,DR 2 , …, DR n etc. Each update depends on the configurations visited during encoding and is determined by relaxing the reduction on frequently visited configurations. A tower structure of the function DR is obtained with an ever increasing image size equal to the number of possible output dynamically reduced configurations.

[0049] #ImDR 0 ≤#Im DR 1 ≤…≤#Im DR n ≤#Im DR n+1 ≤…

[0050] It is further proposed to construct a dynamic reduction function DR that operates by keeping certain bits of contextual information CI. Assume that the contextual information CI is a word consisting of a series of K bits.

[0051] CI=β1…βK

[0052] Then we can get the CI by keeping the first k n (CI) bits to simply define the dynamic reduction function DR n , so that the dynamically reduced context information CI' is

[0053] CI'=β1…βkn(CI)

[0054] And the function DR n Completely composed of array k n (CI) Definition. The value of k0 can be initialized to zero.

[0055] k n (CI)=0

[0056] And by passing the array k n(CI) changes to the new array k n+1 (CI) to execute from DR n To DR n+1 This change is made by increasing the number of hold bits.

[0057] k n (CI)≤k n+1 (CI)

[0058] Based on the construction of binary tree, a series of dynamic reduction functions DR are proposed n Iterative construction (more than n). Initial function DR n =0 corresponds to a tree with only a root node.

[0059] exist Figure 6 An exemplary function DR is shown in n It is defined by a binary tree. For a node of depth d, each branch of the tree is selected according to the value (0 or 1) of the d-th bit βd of the context information CI. The leaf nodes correspond to the dynamically reduced configurations.

[0060] like Figure 6 As shown in the example tree of , there are three dynamically reduced configurations corresponding to the context information CI having the binary form 0xxxxxx, 10xxxxx and 11xxxxx. These three configurations are appended with three numbers N(0xxxxxx), N(10xxxxx) and N(11xxxxx), which track the number of visits so far during the encoding of the point cloud. These numbers count the number of visits to each reduced configuration CI' during encoding.

[0061] Because the function DR is used n The binary representation of k n (CI), so the function can be represented by the array k n (CI) is represented as follows:

[0062] kn(0xxxxxx)=1 and kn(1xxxxxx)=2

[0063] When the number N exceeds the threshold th, the function DR n Evolved into a newer function DR n+1 .

[0064] Figure 7 shows the update function DR in the case where N(10xxxxx) has exceeded the threshold th n+1 .

[0065] The above tree has been extended from the node corresponding to configuration 10xxxxx to two child nodes corresponding to configurations 100xxxx and 101xxxx. In general, when N(C) exceeds the threshold of configuration C, the node corresponding to configuration C is extended to two child nodes associated with two new configurations that replace configuration C. The two new configurations have one more bit added after reduction.

[0066] After the update, function DR n+1 It can be represented by the array k n (CI) is represented as follows:

[0067] k n (0xxxxxx)=1, k n (11xxxxx)=2, and k n (10xxxxx)=3.

[0068] However, according to the prior art, there is still room for improving the coding efficiency. In particular, the sub-occupancy of each node is encoded by Morton ordering, and the encoding order of the nodes is also Morton ordering.

[0069] Therefore, there are at least two disadvantages in the prior art, which are described as follows:

[0070] First, in the prior art, when considering the encoding of the sub-volume occupancy of the current node, not all adjacent sub-volume occupancy has been encoded. Therefore, the occupancy information of some sub-volumes of the adjacent nodes is lost. The following is a description.

[0071] Using breadth-first Morton scan sorting, the available adjacent child nodes are as follows Figure 8 As shown. The occupancy of the children of the 7 neighboring nodes at the lower xyz coordinate corner (a) has always been encoded and is therefore available. However, the occupancy of the children of the 7 neighboring nodes at the higher xyz coordinate corner (b) has never been encoded and is therefore not available. In between, the occupancy of the children of the remaining 12 neighboring nodes may or may not have been encoded and may or may not be available.

[0072] The availability of these 12 neighbor nodes for occupation depends on the position of the current node in the octree and the position of its corresponding Morton code. In particular, according to the first sorting, it depends on the position of the current node in its parent node.

[0073] Therefore, one of the main drawbacks of using Morton ordering is that the neighboring patterns used to determine entropy context or make predictions may vary a lot with different subvolume positions in the Morton ordering. It complicates the design of efficient entropy coding of occupancy trees and may also lead to suboptimal compression.

[0074] Second point, one known advantage of using the Morton ordering (or some other space-filling curve known to be similar) is that this ordering tends to allow processing of nodes that have close distances in 3D, but have close distances within the 1D Morton ordering (i.e. distances between Morton codes / indexes). This is only a trend, of course.

[0075] However, there are exceptions where two nodes at a certain depth may be close in 3D but very far away in the 1D Morton ordering, for example when the nodes are close but opposite to the center of a larger node. In one example, the first voxel belongs to the root node Figure 1 The first voxel belongs to the volume corresponding to b0 in the octree, and this first voxel is close to the center of the root node; the second voxel belongs to the volume corresponding to b7, and is also close to the center of the parent node. In the extreme case, both voxels may be so close that they will share an edge, but after a few depths in the octree traversal, the Morton codes associated with the refinement nodes containing these voxels become increasingly far apart in the 1D Morton ordering.

[0076] For example, there are 2 points (x0, y0, z0) and (x1, y1, z1), and point (x0, y0, z0) = (255, 255, 255), point (x1, y1, z1) = (256, 256, 255), then they can be represented in binary as:

[0077] (x0, y0, z0) = (255, 255, 255) = (011111111b, 011111111b, 011111111b), (x1, y1, z1) = (256, 256, 255) = (100000000b, 100000000b, 011111111b).

[0078] By combining the first bit values ​​of their three coordinates, namely 000b for point (x0, y0, z0) and 110b for point (x1, y1, z1), we can get that point (x0, y0, z0) belongs to node b0 and point (x1, y1, z1) belongs to node b6 in the root node. Figure 1 As shown, they share the same edges.

[0079] And its Morton code can be:

[0080] m0=000111111111111111111111111b=16 777 275

[0081] m1=110001001001001001001001001b=103 060 041

[0082] Then the Euclidean distance in Cartesian space can be

[0083]

[0084] And the Euclidean distance in Morton code space can be

[0085]

[0086] Therefore, the Morton codes associated with the thinning nodes containing 2 points become farther apart in the 1D Morton ordering. Summary of the invention

[0087] Therefore, the object of the present application is to provide a method for encoding a 3D point cloud into a bitstream, and a method for decoding the geometry of a 3D point cloud from a bitstream, with higher coding efficiency.

[0088] The problem is solved by a decoding method according to claim 1 , an encoding method according to claim 2 , an encoder according to claim 12 , a decoder according to claim 13 , a bitstream according to claim 14 , and a software according to claim 15 .

[0089] A first aspect of the present application provides a method for decoding a 3D point cloud from a bitstream, preferably applied to a decoder. The geometry of the point cloud is defined by an octree structure, the octree structure includes multiple nodes, the multiple nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is divided into sub-volumes step by step and contains points of the point cloud, wherein the volume is divided into a plurality of sub-volumes, each sub-volume is associated with one of the multiple nodes of the octree structure, wherein the occupancy information associated with each of the sub-volumes indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, and the method comprises:

[0090] Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order;

[0091] Occupancy information of child nodes of each node is entropy decoded from the bit stream based on the lexicographical ordering.

[0092] On the other hand, a method for encoding a 3D point cloud into a bitstream is provided, preferably applied to an encoder. The geometry of the point cloud is defined by an octree structure, the octree structure includes a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is divided into sub-volumes step by step and contains points of the point cloud, wherein the volume is divided into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each sub-volume indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, and the method comprises:

[0093] Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order;

[0094] Based on the lexicographic sorting, the occupancy information of the child nodes of each node is entropy encoded into a bit stream.

[0095] A point cloud is a set of points in a three-dimensional coordinate system. These points are generally intended to represent the outer surface of one or more objects. Each point has a location (position) in a three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be a Cartesian or any other coordinate system. Therefore, according to the present application, for a given depth d in the octree, the nodes are obtained in a lexicographic order, and the occupancy of the child nodes is encoded at full resolution according to the lexicographic order. Preferably, the nodes are stored in a linked data structure (e.g., a linked list). Therein, the linked data structure has elements connected to each other so that the elements are arranged in a sorted manner. Preferably, a list L can be used. d to store the nodes, and the nodes can be iteratively obtained from the list. With the lexicographic sorting (regardless of the order of the coordinate axes in the lexicographic sorting), the occupancy of any subvolume with a lower z position, a lower y position, and a lower x position can be used for context analysis. Therefore, compared with the prior art (e.g., Morton sorting), the prediction of different subvolumes in the lexicographic sorting is more stable. Therefore, the coding efficiency is improved and the processing delay is reduced.

[0096] Preferably, the method further comprises storing the occupied child nodes at depth d+1 in lexicographic order. Thus, if the node at depth d is not a leaf node, the octree can continue to be traversed in lexicographic order. The child nodes at the next depth d+1 are already stored in a preferred order, thereby further facilitating the next iteration.

[0097] Preferably, the lexicographic ordering of nodes at depth d and / or the lexicographic ordering of nodes at depth d+1 is a raster scan ordering.

[0098] Preferably, the occupancy information of the children of each node at depth d is entropy encoded into a bitstream or entropy decoded from a bitstream based on a lexicographic ordering of the children at depth d+1. Thus, for any child node lower than the encoded child node z, y, x, its occupancy can be used as contextualization. Then, a predetermined pattern of neighboring child nodes can be defined and used to select the coding context for any coded occupied bits.

[0099] Preferably, the occupancy information of the child nodes of each node is entropy encoded into a bitstream or entropy decoded from the bitstream based on the same adjacent child node occupancy pattern indicating the context of each child node. Therefore, the adjacent occupancy pattern used to construct the context for each child node can be the same and predefined, and by using the same adjacent child node occupancy pattern, the construction of the context information can be simplified, thereby improving the coding efficiency.

[0100] Preferably, the context of each child node is selected based on the child node occupancy pattern and / or the occupancy pattern of the adjacent parent node, and if the adjacent node occupancy pattern is used for the selection of the entropy coding context, the context selection may also depend on the child occupancy index within the node, where the index indicates the position of the child node in the node, since the information provided by the occupancy bits of the 6 adjacent nodes of the parent node will not have the same importance when selecting a suitable entropy coding context for encoding the subvolume / node. Therefore, by considering the child occupancy index, i.e. the position of the child node in the parent node, the importance of each occupancy bit of the adjacent node can be distinguished. Preferably, each index is associated with a weight indicating the importance of the adjacent node.

[0101] Preferably, the occupied child nodes at depth d+1 are stored in lexicographic order by traversing each node 4 times. Therein, 4 pipelines may be defined for each node at a certain depth. Each pipeline is formed by the corresponding pipelines of all nodes having fixed coordinate positions in the plane formed by two of the three coordinate axes. For example, referring to Fig. 9 and Fig.10 The pipeline is described in detail. To encode all the children of a node, each node is processed 4 times, and each time one of the 4 pipelines of the node is traversed.

[0102] Preferably, entropy encoding the occupancy information of the child nodes of each node into the bitstream, or entropy decoding the occupancy information of the child nodes of each node from the bitstream, is performed based on a lexicographic ordering of nodes of depth d and a fixed ordering of the child nodes of a given node. Thus, the occupied nodes are processed in a raster scan ordering, but the occupied bits of all sub-volumes of the node are encoded consecutively in a fixed ordering. Compared with the prior art of encoding nodes in Morton ordering, due to the use of lexicographic ordering for scanning, neighboring information is more reliable and the encoding performance is better.

[0103] Preferably, storing the occupied child nodes of depth d+1 is performed by processing each of all nodes obtained at one time, preferably by using an intermediate storage to sort the child nodes lexicographically, preferably by raster scanning. For example, the intermediate list can be used to sort the child nodes within a node in a single pass, so that each node does not have to be processed 4 times to encode all child nodes. The complexity of the encoding is reduced and the performance of the encoding is not affected.

[0104] Preferably, the entropy encoding or decoding of each child node applies a dynamic optimal binary encoder OBUF with immediate update. Therefore, due to the additional beneficial effect of the dynamic OBUF, it is more efficient when a sufficient number of occupied bits have been encoded.

[0105] Another aspect of the present application provides an encoder for encoding a 3D point cloud into a bit stream. The encoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned encoding method are performed.

[0106] Another aspect of the present application provides a decoder for decoding a 3D point cloud from a bit stream. The decoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned decoding method are performed.

[0107] Another aspect of the present application provides a bit stream, which is encoded by the steps of the aforementioned encoding method.

[0108] Another aspect of the present application provides a computer-readable storage medium comprising instructions for executing the steps of the method for encoding a 3D point cloud into a bitstream as described above.

[0109] Another aspect of the present application provides a computer-readable storage medium comprising instructions for executing the steps of the method for decoding a 3D point cloud from a bitstream as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] Hereinafter, the present application is described in more detail with reference to the accompanying drawings.

[0111] The accompanying drawings show:

[0112] Figure 1 The Morton ordering of child nodes and occupied bits is shown.

[0113] Figure 2 The neighborhood of nodes at the same depth or greater than the current node is shown.

[0114] Figure 3An example of occupied nodes in a neighborhood is shown, involving neighboring nodes with a depth greater than the current depth.

[0115] Figure 4 Shown is the occupancy map (white cubes) known from previously encoded and decoded occupied bits when the occupied bits of a child node (striped cubes) are encoded and decoded using Morton ordering.

[0116] Figure 5 A raster scan of voxels is shown.

[0117] Figure 6 The tree-based dynamic reduction function DR is shown n .

[0118] Figure 7 The tree-based dynamic reduction function DR is shown. n Evolution to DR n+1 .

[0119] Figure 8 Shown are the neighboring child nodes of the current node (solid grey) when using Morton scan sorting, where (a) are always available, (b) are never available, and (c) are sometimes available.

[0120] Fig. 9 The raster scanning process of a parent node is shown to raster scan the occupancy of its child nodes.

[0121] Fig.10 The encoding ordering of slice 0 and slice 1 of the occupied nodes of the same fixed x at the current depth is shown.

[0122] Fig.11 An embodiment of a decoder employing the proposed method is shown.

[0123] Fig.12 An embodiment of an encoder using the proposed method is shown.

[0124] Fig.13 Shown is the occupancy map (white cubes) known from previously encoded occupied bits when the occupied bits of a child node (striped cubes) are encoded using raster scan ordering.

[0125] Fig.14 Examples of neighboring sub-occupancy patterns are shown, including 3D (left) and 2D layers (right).

[0126] Fig.15 An example of neighbor node occupancy pattern when the coding sub-occupancy bits (slice volume) are shown.

[0127] Fig.16A raster scan ordered parent node is shown with 8 encoded child occupation bits and a raster scan ordering of child nodes.

[0128] Fig.17 An example is shown (a decoding method for a parent node using raster scan ordering, where the 8 child occupied bits are encoded together).

[0129] Fig.18 An example of an encoder of the second embodiment (encoding method using raster scan ordering) is shown.

[0130] Fig.19 Shown is the occupancy map (white cubes) known from previously coded occupied bits when the occupied bits of child nodes (striped cubes) are coded using raster scan ordering of the nodes but all bits of occupancy of child nodes are encoded using Morton ordering.

[0131] Fig. 20 An encoder block diagram of one embodiment is shown in which child nodes are ordered into a raster scan order by processing the nodes one at a time.

[0132] Fig.21 A decoder block diagram is shown for one embodiment in which child nodes are ordered into a raster scan order by processing the nodes one at a time.

[0133] Fig. 22 The node N is shown in a raster scan order. k The occupied child nodes of L d+1 process.

[0134] Fig.23 shows a buffer L of depth d d+1 , and buffer L d+1 The composition of buffer L0.

[0135] Fig.24 A schematic flow chart of a method for encoding is shown.

[0136] Fig.25 A schematic flow chart of a method for decoding is shown.

[0137] Fig.26 An encoder according to the present application is shown.

[0138] Fig. 27 A decoder according to the present application is shown. DETAILED DESCRIPTION

[0139] In the occupancy tree of G-PCC, each node of the tree represents an occupied cubic volume of a certain size / volume, which can be determined based on its depth in the tree. The occupancy in the tree is encoded depth by depth, starting from the root node that represents the complete volume of the point cloud. In other words, the tree is traversed breadth-first to encode and decode the occupancy tree. For simplicity, our description is limited to an octree representation (without enabling direct encoding mode), where the precision of each of the three dimensions increases by two times for each level of depth of the octree. At a given depth (d) of the tree, the coordinates of all nodes of the tree are processed according to the Morton ordering. For each node, the occupancy of the 8 sub-volumes at the next level of depth is encoded, representing 2×2×2 sub-volumes, and the child nodes in the next depth of the tree are indicated to be provided for the occupied sub-volumes.

[0140] In the case of using quadtree partitioning at a given depth level (d), the occupancy of the sub-volume at the next depth becomes a 2×2×1, 2×1×2, or 1×2×2 sub-volume (i.e., 2D); and the processing order of the child nodes at the next depth (d+1) then becomes the encoding order for their parent nodes (i.e., the nodes at depth d), which are refined by their Morton ordering (i.e., Z-scan ordering along both partitioning directions) in this 2D space. In the case of using binary tree partitioning, the occupancy becomes a 2x1x1, 1x2x1, or 1x1x2 sub-volume (i.e., 1D); and the processing order of the child nodes at the next depth then becomes the encoding order for their parent nodes (i.e., the nodes at depth d) refined by their Morton ordering in this 1D space (i.e., lexicographic ordering along the partitioning directions).

[0141] Basically, in order to process nodes following Morton ordering, a queue structure of nodes can be used. The process starts with a queue containing a single node (the root node corresponding to the complete point cloud volume). Then, when the queue is not empty, the node at the beginning (front) of the queue is removed, the occupancy of its subvolumes is encoded, and if these subvolumes are not leaf nodes, the child nodes of the occupied subvolumes are appended to the end (back) of the queue, following their respective Morton ordering. Therefore, the queue structure acts as a First In First Out (FIFO) buffer: when processing the next depth, the nodes pushed in first are encoded first.

[0142] Figure 1 The 8 possible child nodes of a parent node in Morton order are shown. The occupied bits b0 to b7 indicate the presence or absence of a point in each of the corresponding subvolumes. These bits are encoded when processing a parent node, and the nodes corresponding to the occupied child nodes are appended to the processing queue.

[0143] In the octree representation of the point cloud, if the nodes are processed in depth-first order as in MPEG (Moving Picture Experts Group) GPCC (Geometry-based Point Cloud Compression), it may benefit from knowing the occupancy of nodes at deeper depths, such as Figure 2 shown.

[0144] The node with a depth greater than the current node is used to obtain geometric information in the area that has not been encoded at the current depth (where the y value is greater than the y value of the current node).

[0145] In MPEG G-PCC, the combination of the current depth and the neighboring nodes at the current depth plus 1 is used to define a neighborhood. However, in order to limit the possible number of neighborhood configurations, the neighborhood has been restricted to a subset of the set of nodes that are neighbors to the current node.

[0146] Figure 3 An example of a neighborhood is shown, where only occupied nodes are drawn, also involving neighboring nodes with depth greater than the current depth.

[0147] When using breadth-first scan ordering, it is understood that when encoding a sub-occupancy, the occupancy of all neighboring nodes at the same depth as the current node is always known, since these nodes were encoded at the previous depth.

[0148] Figure 4 It is shown that the occupied volume (consisting of white cubes) at a given depth is known when the encoding of the occupancy bits of the current child node (striped cube) is performed in the case where the nodes are processed in Morton order and the occupancy bits are also encoded in Morton order. From this figure, the nature of the different sub-volumes that can be mapped within the occupancy map can also be understood.

[0149] Raster scan ordering is often used to encode properties of dense content (e.g., grayscale or 3 color channels) that does not require geometry to be present: the geometry covers the entire surface of the image (in the 2D case) or the entire volume (in the case of 3D, biomedical images, or video content).

[0150] In some embodiments, a mask is provided. The mask defines a region of interest to which the attribute encoding is restricted. The encoding of the mask can be assimilated to the encoding of the geometric occupancy in the point cloud. Preferably, the mask is simply encoded by encoding a bit successively for each possible voxel position in the volume under consideration using raster scan ordering and context entropy coding, each bit indicating whether the corresponding voxel is occupied (and therefore whether the attribute is encoded for that voxel position).

[0151] In 2D, raster scan ordering of pixels is simply a lexicographic ordering of the pixels according to their 2D coordinates x and y. Lexicographic ordering in (y, x) is commonly used on images: pixels are processed sorted first by x and then by y; pixels are sorted by increasing x coordinate for a given y coordinate and by increasing y coordinate. 3D raster scan ordering is similar, but in three dimensions (i.e., 3D raster scan ordering is a lexicographic ordering on the arrangement of 3D coordinates). Raster scan ordering of voxels follows only lexicographic ordering, but in three dimensions x, y, and z, e.g., lexicographic ordering in (z, y, x) can be used for volumetric images (or t, y, x for time series of images, e.g., in video).

[0152] like Figure 5 As shown, in a voxel representation of the geometry of a point cloud, voxels (= 3D volumes associated with a single point) can be scanned in a raster scan order, as Figure 5 shown.

[0153] exist Figure 5 The ordering shown follows a raster scan first in y, then in x, and finally in z. This is a lexicographic ordering of xyz. And Figure 5 The causal neighborhood (consisting of white cubes of already encoded nodes / voxels) of the current node / voxel (hash cube) to be encoded is shown. The occupancy of the already encoded neighboring nodes is known and can be used to predict and encode the occupancy of the current node / voxel. An already encoded node / voxel is a node / voxel whose lexicographic rank is lower than that of the current node / voxel.

[0154] The proposed method encodes / decodes subvolume occupancy bits based on raster scan ordered nodes at a given depth in the occupancy tree. In one embodiment of the proposed method, it uses the raster scan ordering of the subvolumes to encode / decode their corresponding occupancy bits; in another embodiment of the proposed method, it uses the raster scan ordering of the nodes to encode / decode the occupancy bits of all their corresponding subvolumes (together).

[0155] In one embodiment, the proposed method Fig. 9 For better understanding, the entire occupied volume is represented in the figure to better see the raster scan ordering of the processing. However, it should be clear that if the ancestor node has been marked as unoccupied, part of the volume will not be encoded / processed.

[0156] like Fig. 9As shown, a lexicographic ordering in (x, y, z) is used (i.e., a cooperative lexicographic ordering in (z, y, x)). For a given depth in the octree, the occupancy of the child nodes is encoded according to the raster scan ordering of the child nodes. In order to achieve the raster scan ordering of the child nodes, 4 tubes (tube 0, tube 1, tube 2, tube 3) are defined for each node at the current depth, each tube being made of each node with the same x and y. The encoding of the occupancy bits of the child nodes of tube 0 is performed by encoding the occupancy bits b0 and b1 of all occupied nodes with the same fixed x and y, and the nodes are sorted by increasing the z of all occupied nodes. The encoding of the occupancy of tube 1 is then performed by encoding the occupancy bits b2 and b3 of the same node and in the same order as tube 0.

[0157] The encoding process for encoding occupied pipe 0 for all occupied nodes with the same fixed x and y and occupied pipe 1 for all occupied nodes with the same fixed x and y is repeated for each y in increasing y order. This provides a raster scan encoding of slice 0 consisting of each bit b0, b1, b2, b3 of all occupied nodes with the same x. An illustration of the raster scan ordering of slice 0 for these nodes is shown in Fig.10 (a) is shown, where each child node is represented in 2D for better illustration. After encoding slice 0 of the nodes at a fixed x, slice 1 of these nodes is then encoded at the same x, and the encoding process of slice 0 is reproduced for slice 1 by continuously encoding in the same order, the occupied pipe 2 then 3 consists of the occupied bits b4, b5 then b6, b7 of the same node. And the raster scan ordering of slice 1 of these nodes is shown in the encoding process of the diagram in Fig.10 (b) where each child node is represented in 2D for better illustration. The encoding process of slice 0 followed by slice 1 is repeated in increasing order for every x nodes at the current depth of all occupied nodes.

[0158] An embodiment of a decoder using the proposed method is Fig.11 is shown as a block diagram, where the list L d Indicates that the node N with a depth of d in the octree is included k (x k ,y k , z k ), with nodes sorted from N0 to N in raster scan order last Sort, and k is the list L d The index of the node in . Fig.12 Both of the following are shown:

[0159] The process of decoding the occupancy of the child node (sIdx, tIdx, i) at depth d+1 from a bitstream following raster scan ordering;

[0160] And put the child nodes into list L d+1 The process of raster scanning ordered nodes to generate the next depth.

[0161] The variable sIdx represents the index of the slice of all occupied nodes with the same x; the variable tIdx represents the index of the pipe of all occupied nodes with the same x and y; and the variable i represents the index of the child node within each pipe.

[0162] In other words, the integer value obtained from the binary word formed by (sIdx, tIdx, i) is the child index of the node (decimal value from 0 to 7).

[0163] The variable kSStart is used to store the sorted index of the first node of the slice of all occupied nodes with the same x; and the variable kTStart is used to store the sorted index of the first node of the pipe of all occupied nodes with the same x and y.

[0164] The depth dMax is the maximum depth in the tree and can be determined from the bitstream.

[0165] The detailed process in the decoder can be described as follows:

[0166] To initialize the algorithm, set the list L0 to contain only the root nodes of the occupied trees, and

[0167] Set the initial depth d to 0;

[0168] Tag 0:

[0169] Get a (first in first out, fifo) list L storing nodes at depth d in raster scan order d ;

[0170] Get an empty list L d+1 to generate a child node in a raster scan order for decoding the next depth;

[0171] Set the node index k to 0 (initialize the variable k with the value 0).

[0172] Label 1:

[0173] Set the slice index sIdx to 0 (initialize the variable sIdx with the value 0).

[0174] Set the first node index in the slice to k (initialize the variable kSStart with the value k);

[0175] Label 2:

[0176] Set the pipe index tIdx to 0 (initialize the variable tIdx with the value 0);

[0177] Set the first node index in the pipeline to k (initialize the variable kTStart with the value of k);

[0178] Label 3:

[0179] From list L d Get node N k (x k ,y k , z k );

[0180] Set the sub-index in the pipeline to 0 (initialize the variable i with the value 0).

[0181] Process 0: Decode and generate L d Middle Node N k the ordered children of the children of the pipe tIdx of the slice sIdx;

[0182] The occupied bits of the i-th child of the pipeline tIdx that decodes the slice sIdx from the bitstream (which is N k Child nodes of (sIdx, tIdx, i));

[0183] If the decoded bit indicates that the child node is occupied, then in list L d+1 At the end of the pipeline tIdx, a new node (which is N) of the i-th child node of the slice sIdx is appended k The new node has coordinates (2×x k +sIdx,2×y k +tIdx,2×z k +i);

[0184] If i is not equal to 1, increase i and repeat process 0, otherwise go to the next step;

[0185] ·Judge N k Is it the last node of depth d (i.e., the list L d The last node of

[0186] If N k If it is not the last node of depth d, then remove it from list L d Get the next occupied node N k+1 (x k+1 ,y k+1 , z k+1). Then determine whether it is x k ≠x k+1 or k ≠y k+1 ;

[0187] If x k ≠x k+1 or k ≠y k+1 , it means that the next node does not belong to the same pipeline as the occupied node with the same x and the same y. (Then determine whether

[0188] tIdx==1;

[0189] If tIdx == 1, it means that the processing of both pipelines (in the pipelines of all occupied nodes with the same x and y) of the child nodes of the current slice of the child node has been completed. Then determine whether x k ≠x k+1 ;

[0190] If x k ≠x k+1 , it means that the next node does not belong to the same slice of the occupied node with the same x. Then determine whether sIdx == 1;

[0191] ●If sIdx == 1, it means processing of two slices of the child node

[0192] (in the slice of all occupied nodes with the same x) is already done.

[0193] Therefore, N k+1 is the next slice of the occupied node and the starting point of the next pipeline of occupied nodes to be processed;

[0194] ● Increase k by 1 and loop to Label1 to start processing the first child node pipeline of the first child node slice of the next occupied node slice;

[0195] ● Otherwise sIdx == 0. This means we can start processing the second slice of the occupied node;

[0196] ●Tag 4:

[0197] ● Increase the slice index sIdx by 1 and reset k to the index of the first node of the slice for all occupied nodes with the same x:

[0198] k = kSStart;

[0199] Loop to label 2 to start the third pipeline processing the child nodes;

[0200] ●Otherwise x k = = xk+1 . This means that the next node belongs to the same slice of the occupied node, but belongs to the next pipeline of the occupied node. Therefore,

[0201] N k+1 is the starting point of the next pipeline to be processed;

[0202] Increase k by 1 and loop to Label2 to start processing the first pipeline of the child node of the next pipeline of the occupied node (belonging to the current slice sIdx of the child node);

[0203] ● Otherwise tIdx == 0. This means we can start processing the second pipeline of child nodes of the current slice of child nodes;

[0204] ●Tag 5:

[0205] Increment the pipeline index tIdx by 1 and reset k to the index of the first node of the pipeline of all occupied nodes with the same x and y: k = kTStart;

[0206] ● loop to label 3 to start the second pipeline for processing child nodes;

[0207] Otherwise x k = = x k+1 And y k = =y k+1 . This means that the next node belongs to the same pipeline of the occupied node. Increase k by 1 and loop to Label3 to continue processing the same pipeline of child nodes;

[0208] Otherwise N k = = N last , which is the last node at depth d. Then determine whether

[0209] tIdx==1;

[0210] If tIdx == 1, it means that the processing of the last pipeline of the child node of the current slice of the child node has been completed, and the next slice of the child node (if there is one) can be processed.

[0211] Then determine whether tIdx==1;

[0212] If sIdx==1, it means that the processing of the last slice of the child node of the last slice of the occupied node has been completed. And the next depth in the occupied tree (if there is one) can be processed. Then determine whether d==dMax;

[0213] • If d == dMax, it means that the last depth of the occupancy tree has been decoded.

[0214] Then complete the occupancy tree;

[0215] ● Otherwise, d < dMax. This means that the next depth of the occupancy tree can be processed.

[0216] Increment the depth d by 1 and loop to Label0 to process the next depth in the occupancy tree;

[0217] · Otherwise sIdx == 0. This means that the processing of the second slice of the children of the last slice of the occupied node can be started. Go to label 4 to start;

[0218] · Otherwise tIdx == 0. This means that the processing of the last pipe of the children of the current slice of the node can be started. Go to label 5 to start.

[0219] In a preferred embodiment, the list L can be omitted d+1 , and when d == dMax, the children are not attached to the list because the list will not be used as there are no more depths to process.

[0220] The coordinates of the children are the coordinates of their parent node refined to one-bit precision. In other words, some of the physical space coordinates of the node center can be obtained by multiplying the coordinates of the node center by the physical dimensions of the node volume scaled according to the depth: ((x k , y k , z k ) + (0.5, 0.5, 0.5)) × (width, length, height) / 2 深度 .

[0221] In a preferred embodiment, when d == dMax, if the child is occupied, output a point (e.g., in a buffer) with coordinates ((2 × x k + sIdx, 2 × y k + tIdx, 2 × z k + i) + (0.5, 0.5, 0.5)) × (width, length, height) / 2 dMax . Generate raster-scan ordered point geometry.

[0222] According to the above encoding process of the children, it is observed that each node N k needs to be accessed / processed 4 times: the first time it is accessed / processed for encoding the occupied pipe 0, the second time for encoding the occupied pipe 1, the third time for encoding the occupied pipe 2, and the last time for encoding the occupied pipe 3.

[0223] This encoding process constructs the nodes required for raster-scan ordering of the occupancy of the children The mentioned ordering is obtained from the raster scan ordering of the nodes being processed. And the raster scan ordering of the nodes is directly obtained from the raster scan ordering of the child node occupancy encoding performed on the parent node (i.e., during the previous depth of the octree occupancy encoding).

[0224] Similarly, an embodiment of an encoder using the proposed method is Fig.12 is shown as a block diagram.

[0225] The effect of the method provided is Fig.13 , which shows the occupancy map coverage of the known sub-volumes when encoding the occupancy bits of the striped sub-volumes when the occupancy bits are encoded according to the raster scan ordering of the sub-volumes of the nodes. Fig.13 Also shown are neighboring positions that can be used to construct a context for entropy coding of occupied bits. With raster scan ordering (regardless of the ordering of the coordinate axes in the lexicographic ordering), the occupancy of any subvolume with a lower z position, lower y position, lower x position can be used for context analysis. Thus, a predetermined pattern of neighboring subvolumes can be defined and used to select a coding context for any coding occupied bit.

[0226] In some embodiments, when each child node is encoded by entropy coding in raster scan order, the neighboring occupancy pattern used to construct the context for each child node can be the same. As an example of a neighboring occupancy pattern, Fig.14 An example of a sub-neighborhood occupancy pattern (occupancy of a white subvolume) is provided which can be used to determine the entropy coding context for encoding the occupancy bits of a striped subvolume when the sub-occupancy is encoded in a raster scan order. On the left, the neighborhood is represented in 3D, while on the right, the neighborhood is represented in 2D layers. In one example, each number on the right can be used to sort the occupancy bits from the most significant / valid one (position 0) to the least significant / valid one (position 15) for use with the dynamic OBUF that has been previously described. This will result in 65536 entropy contexts. In some embodiments where a dynamic OBUF is not used, the number of neighboring nodes can be limited to, for example, positions 0 to 11 to reduce the number of contexts to 4096.

[0227] Fig.15 The neighboring pattern formed by the 6 direct neighboring nodes (nodes sharing a face) of a node is shown when encoding different occupancy bits of the subvolume of the current node. As in the prior art, using Morton sorting, the occupancy of neighboring parent nodes can provide useful information for improving compression efficiency.

[0228] Even for raster scan encoding of subvolumes, an occupancy neighborhood of the same sub-occupancy pattern can be used to encode any of the subvolume occupancy bits. As can be appreciated, depending on the location of the subvolume, the information provided by the occupancy bits of the 6 neighboring volumes / nodes of the parent node will not have the same importance in selecting a suitable entropy coding context for encoding the subvolume. Therefore, if neighbor node occupancy is used for the selection of entropy coding context, the context selection can also depend on the sub-occupancy index. This also applies to the use of other neighboring nodes, such as 36 neighboring nodes, consisting of any neighboring nodes / volumes that have a shared face, edge or wedge with the current node in the modeling of the entropy coding context.

[0229] In some embodiments, the proposed method uses a raster scan ordering of nodes to (together) encode / decode the occupancy bits of all its corresponding subvolumes at a given depth in the occupancy tree. Fig.16 As shown, occupied nodes are processed in raster scan ordering, but the occupancy bits of all subvolumes of the node are encoded consecutively on the node by ordering in a single channel, for example using Morton scan ordering of the subnode occupancy bits (which means that each subvolume of the node is encoded using Morton ordering in a single channel on the node); and in order to order the subnodes by raster scan ordering for the next depth, the same subnode ordering method as in the embodiment is used, which uses 4 channels on each node at the current depth and Figure 7 Shown in.

[0230] Fig.17 shows an example of a child node occupation decoding process of a decoder in some embodiments, Fig.18 shows examples of encoders of some embodiments, and Fig.17 and Fig.18 The sorting process of child nodes is not shown in the figure.

[0231] The detailed process in the encoder / decoder can be described as follows:

[0232] Obtain the fifo list L of nodes at storage depth d by raster scan sorting d ,

[0233] · In the list L from the fifo d Get node N at depth d k (x k， y k , z k) , and initially at each depth from the first node N0(x 0， y 0， z 0) start,

[0234] Encode the bits occupied by the child nodes into a bit stream according to a certain order (such as Morton order), or decode them from the bit stream,

[0235] If N k Not a fifo list L d The last node in the fifo list L d The next node in (N k+1 (x k+1 ,y k+1 , z k+1 ) to encode / decode until L is reached d The last node in

[0236] Encode the occupancy of the child nodes at the next depth until the final depth is reached.

[0237] Fig.19 Shows Fig.16 The described alternative coding ordering encodes the occupancy bits of the striped subvolumes with a known subnode occupancy map. Using the raster scan ordering of the nodes (regardless of the axis ordering used in the lexicographic ordering), the occupancy of any subvolume with a node having a lower z, lower y, or lower x position than the current node (the parent of the striped subvolume) can be used to contextualize the entropy encoding of the striped subvolume occupancy bits as well as any subvolume occupancy bits previously encoded for the current node. Using this approach, it can be clearly distinguished that there is a different pattern of available neighboring nodes for each position of the coded subvolume occupancy. Therefore, in the case of the octree occupancy, there are 8 different patterns. Therefore, the entropy encoding of the occupancy bits needs to be tuned according to each pattern (i.e., for each occupancy bit index) to be more optimal.

[0238] Relative to Fig.11 This complicates the bits to be coded, but is still better than the prior art pattern in an octree with nodes coded with Morton ordering. In the prior art, for Morton ordering of nodes, the pattern occupied by "unknown" (i.e., not yet coded) neighboring nodes varies more with the position in the ancestor node (and therefore with the Morton index), so some neighboring information is not always reliable when constructing the entropy context, and it may introduce bias in the entropy coding probabilities constructed in the adaptive entropy coding context, thereby reducing coding performance.

[0239]

[0116] It should be understood that in point cloud encoding, occupied nodes and subnodes will rarely (and preferably never) differ from example Figure 4 , Fig. 9 , Fig.13 and Fig.16These graphs are densely represented to provide a better view of the available causal sub-occupancy information and the node / sub-node processing ordering. In a usual point cloud, several volumes will be unoccupied, therefore, the nodes of these volumes will not exist and will be processed. The processing ordering of occupied nodes is the same as in the dense content case, and for a given occupied node, the occupancy of all sub-volumes is always encoded, but unoccupied nodes / volumes are not processed at all since they do not exist in the tree structure. Compared to the processing ordering of dense content, the processing of these unoccupied nodes / volumes is skipped.

[0240] For example, in Fig.16 In the illustrated embodiment, the raster scan ordering of the child nodes at depth d+1 is obtained by processing the parent node at depth d four times, which increases the runtime complexity.

[0241] In order to reduce runtime complexity, preferably, the method sorts the child nodes by raster scan sorting, rather than traversing each node 4 times. In some preferred embodiments, such as Fig. 20 As shown. In the raster scan order, N k The occupied child nodes are placed in L d+1 In the step, the intermediate list is used to sort the child nodes at the node once. The occupied pipelines of child nodes 0 to 3 are maintained in 4 separate lists, namely L t0 , L t1 , L t2 and L t3 These lists will be used to construct the occupancy slices for child nodes 0 and 1, which are maintained in 2 separate lists, namely L s0 and L s1 Finally, the list L d+1 From list L s0 and L s1 Build.

[0242] In detail, for N with depth d (k starts from 0) k (x k ,y k , z k ), in which:

[0243] Put (append at the end) a new node for each occupied child of pipe 0 into list L t0 middle,

[0244] Put a new node for each occupied child of pipeline 1 into list L t1 middle,

[0245] Put a new node for each occupied child of pipeline 2 into list L t2 middle,

[0246] Put a new node for each occupied child of pipeline 3 into list L t3 middle,

[0247] The occupied child nodes of each pipeline are processed from bottom to top.

[0248] After iterating over all children of the pipeline of the same x and same y (parent) node at depth d, the pipeline list can be appended to the slice list: L t0 Attached to L s0 At the end of L t1 Attached to L s0 The end of L t2 Attached to L s1 At the end of the t3 Attached to L s1 at the end of each slice list L. s0 and L s1 Then, use the empty pipeline list to process the next pipeline of the (parent) node.

[0249] After iterating over all children of slices of the same (parent) node x at depth d, the slice list can be appended to the next depth list L d+1 :L s0 Attached to L d+1 At the end of L s1 Attached to L d+1 The empty slice list is used to process the next slice of the (parent) node. This provides d+1 The child nodes of the list are sorted in raster scan order. Fig. 22 The diagram shows the process of raster scanning the nodes N. k The occupied child nodes are placed in L d+1 The detailed process.

[0250] If lists are implemented using memory blocks, appending one list to another can take a lot of runtime: memory needs to be copied, and possibly memory allocations. To avoid this problem, in a preferred embodiment, the list L is implemented using a (dynamically) linked list. t0 , L t1 , L t2 , L t3 , L s0 , L s1 , L d and L d+1 . Thus, appending one list to another can have a complexity of O(1). These intermediate buffers allow for keeping nodes processed in one go.

[0251] exist Fig.23 The obtained buffer L storing the child nodes in raster scan order for the next depth is shown in (a) and (b). d+1 , Fig.23 (a) shows the buffer L d+1 A buffer L containing nodes with the same x x (From L x0 To L x(2^d)-1 ), Fig.23 (a) shows the buffer L x0 The structure consists of L t0 , L t1 , L t2 , L t3 And the buffer L d+1 Organized as:

[0252] Buffer L t0 and L t1 :L t0 Pipeline 0 for occupied nodes with same x and same y

[0253] (exist Fig.23 (b)), and L t1 Pipeline 1 for occupied nodes with same x and same y (in Fig.23 (b) and are sorted by increasing y, they are linked

[0254] Continue appending to the buffer L of slice 0 of occupied nodes with the same x s0 ;

[0255] Buffer L t2 and L t3 :L t2 Pipeline 2 for occupied nodes with same x and same y

[0256] (exist Fig.23 (b)), and L t3 Pipeline 3 for occupied nodes with same x and same y (in Fig.23 (b) and are sorted by increasing y, they are linked

[0257] Continue appending to the buffer L of slice 1 of the occupied nodes with the same x s1 ;

[0258] Buffer L of occupied slice 0 of occupied nodes with the same x s0 Buffer L that is appended to occupied slice 1 of the occupied node with the same x s1 , to form a buffer zone L x, which is Fig.23 (a) is represented by, for example, L x0 (for x=x1) and L x1 (for x=x0);

[0259] Buffer L x Append to buffer L x-1 At depth d, buffers L are appended one by one by increasing x x To obtain the buffer L d+1 The raster scan ordered child nodes in (built for the next depth occupancy encoding).

[0260] Reference now Fig.24 , which shows a schematic flow chart of a method for encoding a 3D point cloud into a bitstream according to the present application.

[0261] The method includes:

[0262] Step S10, along each axis X, Y, Z of the coordinate system, obtain a node with a depth of d that contains at least a portion of the point cloud by lexicographic sorting;

[0263] Step S11, entropy encoding the occupancy information of the child nodes of each node into a bit stream based on the lexicographic sorting.

[0264] Reference now Fig.25 , which shows a schematic flow chart of a method for decoding a 3D point cloud from a bitstream according to the present application.

[0265] The method includes:

[0266] Step S20, along each axis X, Y, Z of the coordinate system, obtain a node with a depth of d that contains at least a portion of the point cloud in a lexicographical order;

[0267] Step S21 : entropy decode the occupancy information of the child nodes of each node from the bit stream based on the lexicographic sorting.

[0268] Reference now Fig.26, which shows a simplified block diagram of an example embodiment of an encoder 1100. The encoder 1100 includes a processor 1102 and a storage device 1104. The storage device 1104 can store a computer program or application containing instructions that, when executed, cause the processor 1102 to perform several operations such as those described herein. For example, the instructions can encode and output a bit stream encoded according to the method described herein. It should be understood that the instructions can be stored on a non-transitory computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the described process. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system".

[0269] refer to Fig. 27 , which shows a simplified block diagram of an example embodiment of a decoder 1200. The decoder 1200 includes a processor 1202 and a storage device 1204. The storage device 1204 may include a computer program or application containing instructions that, when executed, cause the processor 1202 to perform several operations such as those described herein. It should be understood that the instructions may be stored on a computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the described processes and methods. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system."

[0270] It should be understood that decoders and / or encoders according to the present application can be implemented in multiple computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer-readable memory, including CD, RAM, ROM, flash memory, etc.

[0271] It should be understood that the decoder and / or encoder described herein and the module, routine, process, thread or other software components implementing the method / process for configuring the encoder or decoder can be implemented using standard computer programming techniques and languages. The application is not limited to specific processors, computer languages, computer programming conventions, data structures, other such implementation details. Those skilled in the art will recognize that the described process can be implemented as a part of a computer executable code stored in a volatile or non-volatile memory, a part of a dedicated integrated chip (Application Specific Integrated Chip, ASIC), etc.

[0272] The present application also provides a computer readable signal encoding data produced by applying the encoding process according to the present application.

[0273] Certain adjustments and modifications may be made to the described embodiments. Therefore, the above embodiments are to be considered as illustrative rather than restrictive. In particular, the embodiments may be freely combined with each other.

Claims

1. A method for decoding a three-dimensional 3D point cloud from a bitstream, preferably, the method is applied to a decoder, the geometry of the point cloud is defined by an octree structure, the octree structure comprises a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is segmented into sub-volumes step by step and contains points of the point cloud, wherein the volume is segmented into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each sub-volume indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, the method comprising: Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order; Occupancy information of child nodes of each node is entropy decoded from the bit stream based on the lexicographical ordering.

2. A method for encoding a three-dimensional 3D point cloud into a bitstream, preferably, the method is applied to an encoder, the geometry of the point cloud is defined by an octree structure, the octree structure comprises a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is segmented into sub-volumes step by step and contains points of the point cloud, wherein the volume is segmented into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each of the sub-volumes indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, the method comprising: Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order; Based on the lexicographic sorting, the occupancy information of the child nodes of each node is entropy encoded into a bit stream.

3. The method according to claim 1 or 2, wherein: The method further includes storing occupied child nodes with a depth of d+1 in lexicographic order.

4. The method according to any one of claims 1 to 3, wherein: The lexicographic ordering of nodes at depth d and / or the lexicographic ordering of child nodes at depth d+1 is a raster scan ordering.

5. The method according to claim 3 or 4, wherein: The occupancy information of the child nodes of each node is entropy encoded into a bit stream or entropy decoded from the bit stream based on the lexicographic ordering of the child nodes with a depth of d+1.

6. The method according to any one of claims 1-5, further entropy encoding the occupancy information of the child nodes of each node into a bit stream or entropy decoding from the bit stream based on the same adjacent child node occupancy pattern indicating the context of each child node.

7. The method according to claim 6, wherein: The context of each child node is selected based on the child node occupancy pattern and / or the occupancy pattern of the adjacent parent node. Preferably, if the occupancy pattern of the adjacent parent node is used for the selection of the entropy coding context, the context selection may also depend on the child occupancy index within the parent node, wherein the index indicates the position of the child node in the parent node.

8. The method according to any one of claims 3 to 7, wherein: The occupied child nodes of depth d+1 are stored in lexicographical order by traversing each node 4 times.

9. The method according to any one of claims 1 to 4, wherein: The occupancy information of the children of each node is entropy encoded into a bitstream or entropy decoded from a bitstream based on a lexicographic ordering of nodes of depth d and a fixed ordering of the children of a given node.

10. The method according to claim 3 or 9, wherein: Storing the occupied child nodes at depth d+1 is performed by processing each of all nodes obtained at once, preferably by lexicographically sorting the child nodes using an intermediate storage, preferably raster-scanning sorting.

11. The method according to any one of claims 1 to 10, wherein: The entropy encoding or decoding of each child node applies a dynamic optimal binary encoder OBUF with real-time updating.

12. An encoder for encoding a three-dimensional 3D point cloud into a bit stream, the encoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to any one of claims 2-11.

13. A decoder for decoding a three-dimensional 3D point cloud from a bitstream, the decoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to any one of claims 1, 3-11.

14. A bitstream encoded by a method according to any one of claims 2-11.

15. A computer-readable storage medium comprising instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1-11.

Images