Method, encoder, decoder, bitstream, and program for encoding and decoding 3D point clouds

Abstract

A method, encoder, and decoder for encoding and decoding a 3D point cloud into a bitstream is provided, the method comprising the steps of: obtaining a node that includes at least a portion of the point cloud at a depth d along each of axes X, Y, and Z of the coordinate system in lexicographic order; and selecting each node N based on a subset of neighboring nodes. The method further comprises the steps of: obtaining a node that includes at least a portion of the point cloud at a depth d along each of axes X, Y, and Z of the coordinate system in lexicographic order; and selecting each node N based on a subset of neighboring nodes. k and entropy coding the occupancy information of each node into a bitstream based on the neighboring pattern.

Description

[Technical Field]

[0001] The present invention relates to a method for encoding a 3D point cloud into a bitstream. It is further an object of the present invention to provide a method for decoding a 3D point cloud from a bitstream. It is also an object of the present invention to provide an encoder and decoder, a bitstream encoded according to the present invention, and software. In particular, it is an object of the present invention to provide a method for searching the occupancy of adjacent nodes and / or adjacent child nodes while encoding or decoding an occupancy tree without sacrificing memory and algorithmic complexity. [Background technology]

[0002] Point clouds as a format for representing 3D data have recently gained attention due to their versatility in their ability to represent all types of 3D objects and scenes. Point clouds can therefore address many use cases, some of which are: Film post-production, Real-time 3D immersive telepresence or VR / AR applications, Free viewpoint video (e.g. for sports viewing), Geographic Information Systems (aka cartography), Cultural heritage (scans of rare objects stored in digital format), Autonomous driving involves 3D mapping of the environment and collecting real-time lidar data.

[0003] A point cloud is a set of points located in 3D space, each of which can have additional values attached to it. These values are commonly called point attributes. A point cloud is thus a combination of geometry (the 3D position of each point) and attributes.

[0004] These attributes may be, for example, three-component color, material properties such as reflectance, and / or two-component normal vectors of the surface associated with the point.

[0005] Point clouds are captured by various types of devices such as camera arrays, depth sensors, lidar, scanners, or are computer-generated (e.g., in film post-production). Depending on the use case, point clouds can have thousands to billions of points for mapping applications.

[0006] In the original representation of a point cloud, each point requires a very large number of bits, at least a dozen bits per spatial component X, Y, or Z, and optionally attributes require many more bits, e.g., color requires three times 10 bits. Practical deployment of point cloud-based applications requires compression techniques that allow point clouds to be stored and distributed over reasonable storage and transmission infrastructures.

[0007] Compression may be lossy (as in the case of lossy video compression) for delivery to and visualization by the user terminal, e.g., on AR / VR glasses or other 3D-enabled devices. Other use cases, such as medical applications or autonomous driving, require lossless compression to avoid affecting decisions made by analyzing the compressed and transmitted point cloud.

[0008] Until recently, point cloud compression (also known as PCC) was not available in the mass market and no standardized point cloud codecs were available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, initiated a work item on point cloud compression. This item involved two standards: MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC), and MPEG-I Part 9 (ISO / IEC23090-9) or Geometry-based Point Cloud Compression (G-PCC).

[0009] The V-PCC and G-PCC standards will have their first versions completed at the end of 2020 and will soon be available on the market.

[0010] The V-PCC coding method compresses point clouds by performing multiple projections of 3D objects to obtain 2D patches that are packed into an image (or video, in the case of moving point clouds). The obtained images or videos are then compressed using existing image / video codecs, allowing for the utilization of already deployed image and video solutions. By its nature, V-PCC is only effective for dense and continuous point clouds, as image / video codecs are unable to compress non-smooth patches resulting from projections of sparse geometric data acquired, for example, by lidar.

[0011] The G-PCC coding method has two methods for compressing geometry.

[0012] The first method is based on an octree (octree / quadtree / binary tree) representation of the point cloud geometry. Occupied nodes are branched until a certain size is reached, and occupied leaf nodes provide the point locations, usually the centers of these nodes. High levels of compression can be obtained for dense point clouds by using a neighbor-based prediction technique. Sparse point clouds are also solved by directly encoding the positions of points in nodes with non-minimal size, stopping the tree construction if only isolated points exist in the node; this technique is known as direct coding mode (DCM).

[0013] The second method is based on prediction trees, where each node represents the 3D location of one point and the relationship between nodes is a spatial prediction from parent to child. This method can only resolve sparse point clouds and offers the advantages of lower latency and easier decoding than occupancy trees. However, compared to the first occupancy-based method, it only offers slightly better compression performance, is more complex to encode, and requires an intensive search for the best predictor (among a long list of predictors for potential subvolumes) when constructing the prediction tree.

[0014] In both schemes, attribute encoding (decoding) is performed after geometry encoding (decoding) is completed, resulting in two-pass encoding. Low latency is therefore achieved by using slices that decompose the 3D space into subvolumes that are coded independently, without prediction between the subvolumes. This can have a significant impact on compression performance when many slices are used.

[0015] An important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. Furthermore, AR / VR point clouds are typically locally 2D, since they represent the surface of an object most of the time. AR / VR point clouds are therefore highly correlated (or said to be dense), in the sense that points are rarely isolated but instead have many neighbors. Summary of the Invention [Problem to be solved by the invention]

[0016] The object of the present invention is to provide a method for encoding a 3D point cloud into a bitstream and a method for decoding the geometry of the 3D point cloud from the bitstream, which reduces memory usage when searching for the occupancy of adjacent nodes and / or adjacent child nodes. [Means for solving the problem]

[0017] This problem is solved by a method for encoding according to claim 1, a method for decoding according to claim 2, an encoder according to claim 10, a decoder according to claim 11, a bitstream according to claim 12 and software according to claim 13.

[0018] In a first aspect, there is provided a method for encoding a 3D point cloud into a bitstream, applied in an encoder, wherein the geometry of the point cloud is defined by an octree structure having a plurality of nodes, the plurality of nodes having a parent-child relationship and representing three-dimensional positions of objects, the point cloud is located in a volume space of a three-dimensional coordinate system that is recursively divided into sub-volumes and contains points of the point cloud, a volume is divided into a set of sub-volumes, each child sub-volume is associated with a respective node of the octree structure, and occupancy information associated with each child sub-volume indicates whether the child sub-volume contains at least one of the points of the point cloud, the method comprising:

[0019] obtaining a node that includes at least a portion of the points at a depth d in lexicographic order along each of the axes X, Y, and Z of the coordinate system;

[0020] Each node N based on a subset of its neighbors k determining a neighborhood pattern of

[0021] and entropy coding the occupancy information of each node into a bitstream based on the neighboring pattern.

[0022] A point cloud is a set of points in a three-dimensional coordinate system. These points are often intended to represent the exterior surfaces of one or more objects. Each point has a position (location) in the three-dimensional coordinate system. The position is represented by three coordinates (X, Y, Z), which may be in a Cartesian coordinate system or any other coordinate system. Thus, according to the present invention, for a given depth d in the octree, the nodes are obtained in lexicographic order. With lexicographic ordering (regardless of the ordering of the coordinate axes used in the lexicographic ordering), the occupation of any node with a lower z-position, lower y-position, or lower x-position can be used for context analysis. Preferably, the nodes are stored in a link data structure (e.g., a linked list), where the link data structure has elements that connect to each other so that the elements are arranged sequentially. Preferably, the list L d The nodes can be stored using [mathematical formula - see original document], and the nodes can be iteratively retrieved from the list. Furthermore, the neighbor pattern of each node is determined based on a subset of neighboring nodes. According to the technical solution of the present invention, when determining the neighbor pattern, the lexicographic ordering of the nodes is taken into account, and only a subset of neighboring nodes is considered, which can avoid wasting too much memory in the occupancy atlas.

[0023] Preferably, the method further comprises the step of storing the occupied child nodes at depth d+1 in lexicographic order. Thus, if the node at depth d is not a leaf node, the octree can be continued to be traversed in lexicographic order. The child nodes at the next depth d+1 are already stored in priority order, further facilitating the next iteration.

[0024] Preferably, the lexicographic order of the nodes at depth d and / or the lexicographic order of the nodes at depth d+1 is a raster scan order.

[0025] Preferably, the occupancy information of each of the nodes is entropy coded into or decoded from the bitstream based on a lexicographical order of nodes at depth d and a fixed order of child nodes of a given node. Thus, occupied nodes are processed in a raster scan order, while the occupancy bits of all child volumes of a node are coded consecutively in a fixed order. Compared with the conventional method of coding nodes in Morton order, the lexicographical order used for scanning makes neighboring information more reliable and the coding performance is better. Preferably, the coding or decoding of child nodes is based on a raster scan ordering.

[0026] Preferably, storing the occupied child nodes at depth d+1 is performed by processing each node in a single pass for all nodes of the obtained node, and preferably by using an intermediate storage to order the child nodes in lexicographic order, preferably in raster scan order. For example, an intermediate list can be used to order the child nodes in a single pass, thereby eliminating the need to process each node four times to encode all child nodes. The encoding complexity is reduced and the encoding performance is not affected.

[0027] Preferably, N k A subset of the neighboring nodes of the node N is stored using a tracking table, which limits the number of neighboring nodes that are kept, and preferably k The tracking table of node N k-1 The tracking table is obtained by updating the corresponding tracking table of the current node N. Therefore, by providing the tracking table, it is not necessary to search all related neighboring nodes every time for the contextualization of the node. The tracking table may be a dynamic table that keeps updating for the next forward node. Preferably, the tracking table is k We only store the closest neighbors of node N. kcan share faces, edges, or vertices with other nodes, which minimizes memory usage and the complexity of searching for adjacent nodes.

[0028] Preferably, the tracking table is a first neighbor offset table (NOT), and the first NOT is a table of neighbor offsets for the current node N. k , where the relative positions of neighboring nodes are defined in NOT. NOT may or may not include the current node (i.e., relative position {0,0,0}). Furthermore, NOT can be easily scaled according to the stated radius (e.g., the radius can be 1, 2, 3, etc., indicating how many nearest neighboring nodes are considered in determining the adjacency pattern).

[0029] It should be understood that all the technical solutions described above regarding the tracking of neighboring nodes and / or the use of raster scan ordering are also applicable to the entropy encoding and / or decoding of child nodes.

[0030] Preferably, the nodes obtained at depth d are stored in a link data structure, and the subset of neighboring nodes is obtained by iterating over the nodes in the link data structure, the link data structure having the elements connecting to each other such that the elements are arranged sequentially, wherein the link data structure for storing the nodes is a linked list L d Preferably, the index or pointer is a pointer to the list L d may be used to find neighboring nodes for a given neighboring position of the current node by iterating over the nodes in

[0031] Preferably, the tracking table is updated by grouping a subset of the retrieved neighboring nodes using vertical tubes of neighboring nodes, each vertical tube including nodes that share the same coordinates in two axes of the coordinate system. Here, each vertical tube consists of all nodes with the same x and y coordinates (or the same x and z, or the same y and z). Tubes are further described in relation to FIG. 5. Because nodes share the same two first coordinates and have a bounded offset difference on the third coordinate, each successive position in the tube corresponds to a node index that increases by a bounded amount in the raster-scan ordered list of nodes. Therefore, applying grouping further reduces coding complexity.

[0032] Preferably, the neighbor pattern is a binary word indicating neighbor occupancy, the binary word being obtained based on the first occupancy and the second occupancy of neighboring nodes, the first occupancy of the neighboring node having a lower coordinate value than the current node in at least one axis, the first occupancy of each neighboring node including an 8-bit child occupancy, the second occupancy of the neighboring node having a higher coordinate value than the current node in at least one axis, and the second occupancy of each neighboring node including a flag, preferably 1 bit, indicating the occupancy of the corresponding neighboring node. Preferably, the binary word is obtained from the tracked neighboring nodes from the above-mentioned tracking table.

[0033] Preferably, the child nodes of all occupied nodes are encoded or decoded in the raster scan order of the child nodes.

[0034] Preferably, said child nodes are stored in a second link data structure, the current child node CN k A subset of adjacent child nodes for a given adjacent child node position of is obtained by iterating over the child nodes in the second link data structure, the second link data structure having the elements connecting to each other such that the elements are arranged sequentially.

[0035] Preferably, the subset of adjacent child nodes is stored using a second adjacent offset table (NOT), the second NOT storing the relative positions of the adjacent child nodes with respect to the current child node.

[0036] Preferably, the occupancy of an adjacent child node having a lower coordinate value than the current child node in the second link data structure is a flag, preferably one bit, indicating the corresponding occupancy of the adjacent child node.

[0037] Preferably, the subset of adjacent child nodes represents the tracked adjacent child nodes in the second link data structure of the current child node, and is updated for the adjacent child node search of the next child node.

[0038] In another aspect of the present invention, there is provided a method for decoding a 3D point cloud from a bitstream, applied in a decoder, wherein the geometry of the point cloud is defined by an octree structure having a plurality of nodes, the plurality of nodes having a parent-child relationship and representing three-dimensional positions of objects, the point cloud is located in a volume space of a three-dimensional coordinate system that is recursively divided into sub-volumes and contains points of the point cloud, a volume is divided into a set of sub-volumes, each child sub-volume is associated with a node of the octree structure, and occupancy information associated with each child sub-volume indicates whether the child sub-volume contains at least one of the points of the point cloud, the method comprising:

[0039] obtaining a node that includes at least a portion of the points at a depth d in lexicographic order along each of the axes X, Y, and Z of the coordinate system;

[0040] Each node N based on a subset of its neighbors k determining a neighborhood pattern of

[0041] and entropy decoding the occupancy information of each node from the bitstream based on the neighboring pattern.

[0042] In another aspect of the present invention, there is provided an encoder for encoding a 3D point cloud into a bitstream, the encoder comprising a memory and a processor, the memory storing instructions that, when executed by the processor, perform the steps of the method for encoding described above.

[0043] In another aspect of the present invention, there is provided a decoder for decoding a 3D point cloud from a bitstream, the decoder comprising a memory and a processor, the memory storing instructions that, when executed by the processor, perform the steps of the method for decoding described above.

[0044] In another aspect of the invention, there is provided a bitstream encoded by the steps of the method for encoding described above.

[0045] In another aspect of the invention, there is provided a computer-readable storage medium comprising instructions for performing the steps of the method for encoding a 3D point cloud into a bitstream as described above.

[0046] In another aspect of the invention, there is provided a computer-readable storage medium comprising instructions for performing the steps of the method for decoding a 3D point cloud from a bitstream as described above. [Brief explanation of the drawings]

[0047] The invention will now be described in more detail with reference to the accompanying drawings.

[0048] The drawings show:

[0049] [Figure 1] An example of occupied nodes / voxels in a neighborhood is shown.

[0050] [Figure 2] Indicates the neighborhood consisting of nodes at the same depth or higher than the current node.

[0051] [Figure 3] 1 shows an example of an occupied node in a neighborhood that includes neighboring nodes at a higher depth than the current depth.

[0052] [Figure 4] When encoding or decoding the occupied bits of a child node (striped cube) using Morton order, we show the occupancy map (white cubes) that is known from the previously encoded or decoded occupied bits.

[0053] [Figure 5] 1 illustrates the raster scan process of a parent node to raster scan encode the occupation of its child nodes.

[0054] [Figure 6] Indicates the coding order of slice 0 and slice 1 for occupied nodes with the same fixed x at the current depth.

[0055] [Figure 7] 1 is an example of a decoder.

[0056] [Figure 8] This is an example of an encoder.

[0057] [Figure 9] When encoding the occupied bits of a child node (striped cube) using raster scan order, we show the occupancy map (white cubes) known from previously encoded occupied bits.

[0058] [Figure 10] 1 shows a raster scan ordered parent node with eight coded child occupancy bits and the raster scan ordering of the child nodes.

[0059] [Figure 11] An example of another decoder (decoding using raster-scan-ordered parent nodes where the eight child-occupied bits are coded together) is shown below.

[0060] [Figure 12] An example of another encoder (encoding using raster-scan ordered parent nodes where the 8 child occupied bits are coded together) is shown below.

[0061] [Figure 13] The occupancy map (white cubes) is shown as it is known from the previously coded or decoded occupancy bits when encoding or decoding the occupancy bits of the child nodes (striped cubes) using the raster scan order of the nodes, but encoding all bits of the occupancy of the child nodes using Morton order.

[0062] [Figure 14] Here is an overview of the decoding method using the provided method.

[0063] [Figure 15] An outline of the encoding method using the provided method is given below.

[0064] [Figure 16] It indicates that the 26 neighboring nodes of the current node (the central node) are coded.

[0065] [Figure 17a] 10 is a flowchart of a process for determining the occupancy of neighboring nodes for a node. [Figure 17b] 10 is a flowchart of a process for determining the occupancy of neighboring nodes for a node.

[0066] [Figure 18] The process of determining the child occupancy of the 13 neighboring nodes of node Nk is shown in detail.

[0067] [Figure 19] The process of determining the depthOccupancy of the 13 neighboring nodes of node Nk is shown in detail.

[0068] [Figure 20] We detail the process of determining the childOccupancy of two neighboring nodes in the same tube as NeighP[i].

[0069] [Figure 21] We detail the process of determining the depthOccupancy of two neighboring nodes in the same tube as NeighP[i].

[0070] [Figure 22] Indicates that adjacent child nodes of the current child node are to be searched, and each cube represents one child node.

[0071] [Figure 23] 1 shows a schematic flow chart of a method for encoding.

[0072] [Figure 24] 1 shows a schematic flow chart of a method for decoding.

[0073] [Figure 25] 1 shows an encoder according to the present invention;

[0074] [Figure 26] 1 shows a decoder according to the invention; DETAILED DESCRIPTION OF THE INVENTION

[0075] Figure 1 shows an example of a neighborhood where only occupied nodes / voxels are depicted. The number of possible occupation configurations for a neighborhood is 2N, where N is the number of nodes / voxels contained in the neighborhood.

[0076] In an octree representation of a point cloud, if nodes are processed in depth-first order, as in MPEG (Moving Picture Experts Group) GPCC (Geometry-based Point Cloud Compression), as shown in Figure 2, we can benefit from occupying nodes at known higher depths.

[0077] Nodes with a higher depth than the current node are used to obtain geometry information in areas not yet coded at the current depth (where the y-values are greater than the y-value of the current node).

[0078] In MPEG G-PCC, the combination of neighboring nodes at the current depth and the current depth + 1 is used to define the neighborhood. However, to limit the number of possible neighborhood configurations, the neighborhood is restricted to a subset of the set of nodes that are neighbors of the current node.

[0079] FIG. 3 shows an example of a neighborhood where only occupied nodes are depicted, when neighboring nodes at a higher depth than the current depth are also included.

[0080] By construction, Morton ordering also allows for easy consideration of neighborhoods within a given cubic subvolume, which includes all nodes at a first predetermined depth D0 whose Morton codes share a common prefix (when written in binary format). This prefix corresponds to the Morton code of a common ancestor node at a second predetermined depth D1, whose total volume is the same as the given cubic subvolume. This allows the size of the subvolume to be multiplied by 2 times the number of possible Morton codes (i.e., nodes) at a given depth. Nwhere N is the number of bits lower than the Morton code prefix. In this case, N is equal to three times the first predetermined depth minus the second predetermined depth, or N = 3 × (D0 - D1). In current G-PCC methods, such neighborhood occupancies can be buffered, for example, in a linear memory space containing the occupancies of all possible Morton codes in a 3D volume. This representation of occupancy is sometimes called an occupancy atlas in G-PCC. Therefore, the occupancy of a child node at a given child position in the volume can be retrieved directly from its position in the buffer corresponding to the child node's Morton suffix (consisting of N bits).

[0081] This allows for quick / immediate finding of the occupancy of a given neighbor node within this volume (i.e., complexity is in O(1)). The overall complexity of finding neighbor node occupancies to encode or decode the child occupancies of N nodes is in O(N), however the encoding or decoding efficiency is suboptimal as it is limited by the size of the atlas (neighbor nodes outside the cube / volume represented by the atlas cannot be obtained, which reduces the encoding or decoding efficiency).

[0082] Instead of using an occupancy atlas, which consumes memory but avoids having the same behavior as G-PCC on the boundaries of the atlas, one can search for the occupancies of neighboring nodes inside the data structure and restrict the search to nodes contained inside the same volume as the node considered by the atlas. For example, searching for a given neighboring node in a buffer containing all ordered occupied nodes can have a cost logarithmic in the number of nodes in the same volume.

[0083] The occupancy map and search method can be applied to obtain the occupancy of adjacent nodes at the same tree depth and the occupancy of previously coded adjacent child nodes at the next tree depth. For the occupancy map, one occupancy map / atlas can be used for nodes at the same tree depth and one occupancy map / atlas can be used for nodes at the next depth, or they can be stored together. For the search method, the occupancy of child nodes at the next tree depth can be stored in a memory structure associated with the node at the same tree depth to reduce search complexity.

[0084] Other than G-PCC, if the size of the occupied atlas is not limited at all (so there are no atlas boundaries and occupied neighbors are not lost), it would be preferable not to use an atlas at all, as the atlas could potentially become too large in memory, which would increase the complexity of neighbor searches.

[0085] If the search for neighbors is done in a Morton-ordered list / buffer of nodes, then the overall complexity of finding neighbors to determine the context for encoding the child occupations of N nodes is O(N.log2(N)). One can try to reduce this complexity by using a hash table, but this comes at the cost of increased memory usage (compared to a list / buffer of occupied nodes), and in the best case (with enough memory and a suitable hash function) the complexity is O(N), but in the very worst case the complexity is O(N 2 ) (or potentially O(N.log2(N)) depending on hash collision handling). Furthermore, the hash table needs to be constructed / populated before it can be used.

[0086] Thus, occupancy atlases reduce complexity, but at the cost of increased memory usage. Typically, the size of an atlas is a few megabytes of data for a good tradeoff in terms of coding efficiency.

[0087] Figure 4 shows that when nodes are processed in Morton order and the occupancy bits are also coded in Morton order, the volume of occupancy (consisting of white cubes) at a given depth is known when performing the coding of the occupancy bits for the current child node (striped cube).

[0088] From this figure, the nature of the different sub-volumes that can be mapped within the occupancy atlas can also be understood.

[0089] According to some methods, the child volume occupancy bits are encoded / decoded based on the nodes in raster scan order at a given depth in the occupancy tree. Some use the raster scan order of the child volumes to encode / decode their respective occupancy bits. Some use the raster scan order of the nodes to encode / decode the occupancy bits of all respective child volumes (together) for more optimal coding efficiency.

[0090] An example of the above method is shown in Figure 5 (Example 1). For better understanding, the entire occupied volume is depicted in the figure to better see the raster scan order of the processing. It should be understood that if an ancestor node is shown as unoccupied, then part of the volume will not be coded / processed.

[0091] As shown in Figure 5, lexicographic ordering on (x,y,z) (i.e., colexicographic ordering on (z,y,x)) is used. For a given depth in the octree, the occupancy of child nodes is encoded according to the raster scan order of the child nodes. To enable raster scan ordering of child nodes, four tubes (tube 0, tube 1, tube 2, tube 3) are defined for each node at the current depth, and each tube is made up of nodes with the same x and y. Then, encoding the occupancy bits of child nodes of tube 0 is performed by encoding the occupancy bits b0, b1 of all occupied nodes with the same fixed x and y, and the nodes are ordered by increasing z of all occupied nodes. Next, encoding of occupied tube 1 is performed by encoding the occupancy bits b2, b3 of the same node, in the same order as tube 0.

[0092] The encoding process for encoding occupied tubes 0 of all occupied nodes with the same fixed x and y, and occupied tubes 1 of all occupied nodes with the same fixed x and y, is repeated for each y in ascending y order. This provides a raster scan encoding of slice 0, consisting of bits b0, b1, b2, and b3 of all occupied nodes with the same x. A diagram of the raster scan ordering of slice 0 of these nodes is shown in Figure 6(a), where each child node is represented in 2D for better illustration. After encoding slice 0 of nodes with a fixed x, the encoding process for slice 0 is replicated for slice 1 by encoding slice 1 of these nodes with the same x, and then sequentially encoding occupied bits b4 and b5, followed by occupied tubes 2 and 3, consisting of occupied bits b6 and b7 of the same nodes, in the same order. The encoding process of slice 1 for these nodes in raster scan ordering is then shown in Figure 6(b), where each child node is represented in 2D for better illustration. The encoding process of slice 0, then slice 1, is repeated for each x number of nodes at the current depth among all occupied nodes in ascending order.

[0093] An example of a decoder using the above method (corresponding decoder to Example 1) is shown as a block diagram in FIG. 7, where list L d is the node N at depth d in the octree. k (x k ,y k ,z k ), and the nodes are N0 to N last k is the list L ordered in raster scan order up to d is the ordering index of the node in Figure 7. A process for decoding the occupancy of the child node (sIdx, tIdx, i) at depth d+1 from the bitstream according to the raster scan ordering; and Raster scan of the next depth to generate an ordered node by listing the child nodes d+1 The process of putting

[0094] The variable sIdx represents the index of the slices of all occupied nodes with the same x, the variable tIdx represents the index of the tubes of all occupied nodes with the same x and y, and the variable i represents the index of the child node within each tube.

[0095] In other words, the integer value obtained from the binary word formed by (sIdx, tIdx, i) is the child index of the node (decimal value from 0 to 7). The variable kSStart is used to store the ordering index of the first node of a slice of all occupied nodes with the same x, and the variable kTStart is used to store the ordering index of the first node of a tube of all occupied nodes with the same x and y. The depth dMax is the maximum depth in the tree and can be determined from the bitstream.

[0096] The detailed process in the decoder can be described as follows: To initialize the algorithm, set the list L0 to contain only the root node of the occupancy tree and set the initial depth d to 0. Label 0: A (first in first out, fifo) list L that stores the nodes at depth d in raster scan order. d Get. Empty list L d+1 to generate a child node in the raster scan ordering for decoding the next depth. Set the node index k to 0 (initialize the variable k with the value 0). Label 1: Set the slice index sIdx to 0 (initialize the variable sIdx to the value 0). Set the first node index in the slice to k (initialize the variable kSStart with the value k). Label 2: Set the tube index tIdx to 0 (initialize the variable tIdx with the value 0). Set the first node index in the tube to k (initialize the variable kTStart with the value k). Label 3: List L d to node N k (x k ,y k ,z k ) to get the Set the child index in the tube to 0 (initialize the variable i with the value 0). Process 0:L d Node N in k Decode and generate ordered child nodes of the child nodes in tube tIdx of slice sIdx. The occupied bits of the i-th child node of the tube tIdx of the slice sIdx from the bitstream (N k Decrypt the child node (sIdx, tIdx, i) of If the decoded bits indicate that the child node is occupied, create a new node (N k List L d+1 This new node is added to the end of the coordinates (2×x k +sIdx,2×y k +tIdx,2×z k +i). If i is not equal to 1, increment i and repeat process 0, else go to next step. N k is the last node of depth d (i.e., list L d Determine whether the node is the last node of the N k is not the last node at depth d, then the list L d to the next occupied node N k+1 (x k+1 ,y k+1 ,z k+1 ) and then x k ≠x k+1 or y k ≠y k+1 It is determined whether or not x k ≠x k+1 or y k ≠y k+1 If tIdx==1, it means that the next node does not belong to the same tube of occupied nodes with the same x and the same y (then determine whether tIdx==1). If tIdx==1, this means that the processing of the two tubes of the child nodes of the current slice of the child node (in the tubes of all occupied nodes with the same x and y) has been completed. k ≠x k+1 It is determined whether or not x k ≠x k+1 If so, it means that the next node does not belong to the same slice of the occupied node with the same x. Then, determine whether sIdx==1. If sIdx==1, this means that the processing of two slices of child nodes (in the slices of all occupied nodes with the same x) has finished. Therefore, N k+1 is the start of the next slice of occupied nodes and the next tube of occupied nodes to be processed. Increase k by 1 and loop to label 1 to start processing the tube of the first child node of the slice of the first child node of the slice of the next occupied node. Otherwise, sIdx==0, which means we can start processing the second slice of the occupied node. Label 4: · Increase the slice index sIdx by 1 and reset k to the index of the first node in the slice of all occupied nodes with the same x: k=kSStart. Loop to label 2 and start processing the third tube of child nodes. Otherwise, x k ==x k+1 This means that the next node belongs to the same slice of the occupied node, but to the next tube of the occupied node. k+1 is the start of the next tube to be processed. Increase k by 1, loop to label 2 and start processing the first tube of the child node of the next tube of the occupied node (belonging to the current slice sIdx of the child node). Otherwise, tIdx==0, which means we can start processing the second tube of the child node of the current slice of the child node. Label 5: · Increase the tube index tIdx by 1 and reset k to the index of the first node in the tube of all occupied nodes with the same x and y: k=kTStart. Loop to label 3 and start processing the second tube of child nodes. Otherwise, x k ==xk+1 and y k ==y k+1 This means that the next node belongs to the same tube as the occupied node. Increase k by 1 and loop to label 3 to continue processing the same tube of the child node. · Otherwise, N k ==N last and the node is the last node at depth d. Determine whether tIdx == 1. · If tIdx == 1, this means that the processing of the last tube of the child node of the current slice of the child node has ended. And the next slice (if any) of the child node can be processed. Next, determine whether tIdx == 1. · If sIdx == 1, this means that the processing of the last slice of the child node of the last slice of the occupied node has ended. And the next depth (if any) within the occupancy tree can be processed. Next, determine whether d == dMax. · If d == dMax, it means that the last depth of the occupancy tree has been decoded. After that, the occupancy tree is completed. · Otherwise, d < dMax. This means that the next depth of the occupancy tree can be processed. Increase d by 1 and loop to label 0 to process the next depth within the occupancy tree. · Otherwise, sIdx == 0. This means that the processing of the second slice of the child node of the last slice of the occupied node can be started. Proceed to label 4 to start it. · Otherwise, tIdx == 0. This means that the processing of the last tube of the child node of the current slice of the child node can be started. Proceed to label 5 to start it.

[0097] List L d+1 can be omitted. If d == dMax, no further depth is processed so the list is not used and the child node is not added to the list.

[0098] The coordinates of a child node are the coordinates of its parent node refined to 1 bit of precision. In other words, some physical space coordinates of the center of a node can be obtained by multiplying the coordinates of the center of the node by the physical dimensions of the node's volume scaled according to its depth. ((x k ,y k ,z k )+(0.5,0.5,0.5))×(width,length,height) / 2 深度 .

[0099] Preferably, if d==dMax, then if the child node is occupied, the coordinates are ((2×x k +sIdx,2×y k +tIdx,2×z k +i)+(0.5,0.5,0.5))×(width,length,height) / 2 dMax The points are output (e.g., in a buffer). A raster-scan-order point geometry is generated.

[0100] From the above child node encoding process, each node N k It can be observed that needs to be accessed / processed four times: once to encode occupied tube 0, a second time to encode occupied tube 1, a third time to encode occupied tube 2, and finally a final time to encode occupied tube 3.

[0101] Node N required by this encoding process to construct the raster scan ordering of its child nodes k The mentioned ordering of is already obtained from the raster scan ordering of the processed nodes. Also, the raster scan ordering of the nodes is obtained directly from the raster scan ordering of the child node occupancy coding performed on the parent node (i.e., the depth period before the octree occupancy coding).

[0102] The encoder of Example 1 is shown as a block diagram in FIG.

[0103] The effect of the provided method is illustrated in FIG. 9, which shows the child volume occupancy map coverage already known when encoding the occupancy bits of striped child volumes when the occupancy bits are encoded according to the raster scan order of the child volumes of a node. FIG. 9 also makes it possible to see neighboring positions that can be used to build a context for entropy encoding of the occupancy bits. Using the raster scan ordering (regardless of the axis ordering used in lexicographic ordering), the occupancy of any child volume with a lower z-position, lower y-position, or lower x-position can be used for contextualization. Thus, a predetermined pattern of neighboring child volumes can be defined and used to select the encoding context for any encoded occupancy bit.

[0104] According to another example, the method uses raster scan ordering of the nodes to (jointly) encode / decode the occupied bits of all respective child volumes at a given depth in the occupancy tree. In this example, as shown in FIG. 10, occupied nodes are processed in raster scan order, but the occupied bits of all child volumes of a node are sequentially encoded in a single pass order at the node, which can use, for example, Morton scan ordering of the child node occupied bits (this means encoding each node's child volume using Morton ordering in a single pass at the node). Then, to order the child nodes in raster scan order for the next depth, the same child node ordering method as in the first example, using four passes at each node at the current depth, is used, as shown in FIG. 7.

[0105] FIG. 11 shows an example of a child node occupation decoding process of a decoder in this embodiment, and FIG. 12 shows an example of an encoder in this embodiment, and the child node ordering process is not shown in both FIG. 11 and FIG. 12.

[0106] The detailed process in the encoder / decoder can be described as follows: A fifo list L that stores the nodes at depth d in raster scan ordering. d Get. FIFO list L d From node N at depth d k (x k ,y k ,z k ), starting with the first node N0(x0,y0,z0) at each depth. The bits occupied by its children are decoded from or encoded into the bitstream according to an ordering such as Morton's order. N k is a fifo list L d If it is not the last node in the fifo list L d Next node N in k+1 (x k+1 ,y k+1 ,z k+1 ) and it is L d Encode / decode until the last node in is reached. ·Encode the child node occupation at the next depth until the final depth is reached.

[0107] FIG. 13 shows the child node occupancy map already known when encoding the occupied bits of a striped child volume when using the alternative encoding ordering described in FIG. 5. Using the raster scan ordering of the nodes (regardless of the axis ordering used in lexicographic ordering), the occupancy of any child volume of a node with a lower z, lower y, or lower x position than the current node (parent node of the striped child volume) is used to contextualize the entropy encoding of the striped child volume occupancy bit and any child volume occupancy bit previously encoded with respect to the current node. In this way, it is possible to clearly distinguish that for each position occupied by an encoded child volume, there is one different pattern for available neighboring nodes. Thus, in the case of octree occupancy, there are eight different patterns. Therefore, the entropy encoding of the occupied bits needs to be adjusted according to each pattern (i.e., for each occupied bit index) to be more optimal.

[0108] This complicates the coded bits compared to the first example, but still outperforms the patterns in an octree with nodes coded in Morton order in the prior art. In the prior art, with Morton ordering of nodes, the patterns occupied by "unknown" (i.e., not yet coded) neighboring nodes further vary with position within the ancestor node (and thus with the Morton index), and therefore some neighbor information is not always reliable when constructing the entropy context, which may introduce bias into the entropy coding probabilities constructed in the adaptive entropy coding context, thereby degrading coding performance.

[0109] It should be understood that in point cloud coding, occupied nodes and child nodes are rarely (and preferably never) dense like the examples in Figures 4, 5, 9, and 10. These figures are represented densely to provide a better view of the available causal child occupancy information and node / child node processing order. In a typical point cloud, some volumes are unoccupied, so the nodes for these volumes do not exist and are processed. The processing order for occupied nodes is the same as if the content were dense: for a given occupied node, the occupancies of all child volumes are always coded, but unoccupied nodes / volumes are not processed at all, since they do not exist in the tree structure. Compared to the processing order for dense content, the processing of these unoccupied nodes / volumes is skipped.

[0110] In the prior art, to construct an adjacent pattern for entropy coding of a node, an occupancy atlas is used to store the occupancy information of many nodes in a cube. For example, in the current G-PCC method, a cube of 512*512*512 nodes is used, which causes a large amount of memory usage. Specifically, in the current G-PCC method, the occupancy tree information is encoded using Morton ordering of nodes in the occupancy atlas. When encoding occupancy bits, the entropy coding context is usually constructed based on the occupancy of adjacent nodes and / or adjacent child nodes. However, in the Morton ordering, obtaining the occupancy of an adjacent node or child node requires a large amount of memory usage (e.g., an occupancy atlas or a hash table of occupied nodes) or a search algorithm with a worst-case complexity of at most O(N.log2(N)).

[0111] Therefore, the problem to be solved by the present invention is to search for the occupancies of adjacent nodes and / or adjacent child nodes while encoding or decoding the occupancy tree without sacrificing the memory of the occupancy atlas and without increasing the overall complexity due to an O(N.log2(N)) search algorithm.

[0112] The provided method processes nodes based on raster-scan-ordered nodes and stores only the information of neighboring nodes adjacent to the node to be coded / decoded in order to build a neighboring pattern for entropy coding. Using that order, a buffer of raster-scan-ordered nodes can be used to both process nodes in order and to look up neighboring nodes (which can optionally be associated with already decoded children).

[0113] In this way, the memory increase is small because the buffer already exists for normal processing of the node, and the provided method utilizes the buffer to generate neighbor patterns for entropy coding. Also, the complexity of searching for neighbor nodes can be expressed as O(N).

[0114] As mentioned above, with a Morton ordering of nodes, it is difficult to avoid using an occupancy atlas to obtain neighborhood occupancies (for contextualization of entropy coding) without sacrificing coding performance or increasing search complexity.

[0115] In the raster scan ordering of nodes, when using an occupancy atlas to search for neighbor occupancies, the positioning in the encoding order traverses the entire point cloud volume regularly in the first two axes, while increasing in the third axis, so it is not easy to limit the atlas size to a subvolume like in the Morton order. Therefore, to be used efficiently to provide occupancy neighborhoods using raster scan ordering, the atlas must represent the number of slices encoded before some of the node (a node's slice represents all nodes with a given position on the third axis), and the number of slices limits the distance of the neighbors along the third axis. Therefore, if the point cloud is located at a depth D, the atlas size must be 2. D ×2 D ×2 D If , the size of the child node's buffer is 2 D+1 ×2 D+1Depending on the size of the point cloud, the buffer may require much more memory than is typically used by a Morton-ordered node-occupancy atlas.

[0116] Fortunately, the raster scan ordering of nodes makes it possible to avoid wasting a lot of memory with an occupancy atlas: instead of keeping a buffer of occupied bits in a dense volume (atlas), we can keep several pointers / iterators / indexes on a list of nodes, each representing the position of a neighboring node relative to the current node.

[0117] In the original method, for each node whose child occupancies are coded, the corresponding neighboring nodes must be searched for for each adjacent position to be used in the coding scheme (e.g., to select an entropy context and / or make occupancy predictions). As with Morton ordering, this search occurs in O(log(N)), where N is the number of nodes at the current tree depth; for example, a binary search can be performed on raster-scan-ordered nodes. If the node is found, the volume is occupied, and its child occupancies that may be associated with it as it is coded / decoded can be searched. If the node is not found, the volume is unoccupied, and therefore the subvolumes of its child nodes are also unoccupied.

[0118] This original method is still far from optimal: to encode the child occupancies of N nodes at a given depth, the complexity is O(N*log(N)), and if M is the number of neighboring nodes searched and used for the encoding process on one node (prediction / context), the complexity can even be expressed as O(M*N*log(N)).

[0119] In order to take advantage of the raster scan ordered nodes and reduce memory consumption, the provided method searches for neighboring node occupancy by storing only the occupancy information of neighboring nodes adjacent to the current node to construct a neighboring pattern for entropy coding.

[0120] In some embodiments, the decoder, shown as a block diagram in FIG. 14, is based on the raster scan ordering encoding method described above, in which encoding the occupancy of all child nodes / child volumes of a node at a given depth is performed in a single pass over the nodes according to their raster scan order. Because of the raster scan order, nodes associated with a given adjacent position (the same relative position for all nodes) can also be processed in a single pass over the nodes. For example, the index in the raster scan ordering of an adjacent node at position (-1,-1,-1) relative to a processed node will increase when increasing the index in the raster scan ordering of the processed node. Thus, by holding the index of an adjacent node at position (-1,-1,-1) of a previously processed node, the index of this processed node can be increased based on the index held by the previously processed node until: 1) Finding the neighboring node at the position (-1,-1,-1) of the currently processed node if this neighboring node is occupied and therefore in the raster scan ordered list of the processed node; and 2) Finding a node having a position higher in the raster scan order than the position (-1,-1,-1) relative to the currently processed node, if the neighboring volume at position (-1,-1,-1) relative to the currently processed node is unoccupied.

[0121] Note that if we consider the raster scan order corresponding to the lexicographical order in (x,y,z), then in the raster scan order, the condition for position (x1,y1,z1) to be higher than or equal to position (x0,y0,z0) is if x1>x0, or if x1=x0 and y1>y0, or if x1=x0 and y1=y0 and z1>z0.

[0122] As shown in FIG. 14, the provided decoding process follows the steps below. Determine a neighbor offsets table (NOT) that defines the relative coordinates between each neighbor node position and the current node position, where each neighbor node shares a common face, a common vertex, or a common edge with the current node. · Iterate over the nodes at each depth to decode the occupancy tree. Get the depth d and the list L d stores the nodes at depth d in raster scan order. List L d Iterate over each node in · Determine the adjacent occupation. Each node N k For the defined NOT and its position P k The positions of the 26 neighboring nodes NeighP are determined using NeighP[i]=Pk+NOT[i], By using 26 pointers to track the adjacent nodes at the corresponding positions, and by retaining the index of the adjacent node at the corresponding position of the previously processed node to process the next node, the adjacent nodes obtained by NOT[i] are stored in the list L. d It can be searched from the index maintained in A pointer is defined and a list L d It is used to iterate over the nodes in to determine the occupancy information (occupied or not) of its 26 neighboring nodes. The obtained neighbor node occupancy is then used to construct a neighbor pattern for each node, which can be used for context selection in entropy coding. · Decode the node's child occupied bits from the bitstream using entropy coding.

[0123] In some embodiments, for example, fewer than 26 neighboring nodes may be used to reduce complexity.

[0124] In some embodiments, the node N k The neighboring nodes of node N k (i.e., the neighboring node may be further away from node N k (It does not have to share any faces, edges, or vertices with

[0125] In some embodiments, more than 26 neighboring nodes can be used to improve coding performance, for example, by using more complex neighboring patterns.

[0126] The corresponding encoder for this embodiment is also shown in FIG.

[0127] In some embodiments, as shown in FIG. 16(a), there are 26 neighboring nodes to be searched to obtain the neighbor pattern of the current node being encoded. To determine the relative position of each neighboring node relative to the current node, a relative position table NOT[i]i=0,1,...,26 is defined, where each element in the table represents the position offset between the neighboring node and the current node. The table has a total of 27 elements, including one element for the current node and 26 elements for the neighboring nodes. The index i of each element NOT[i] represents the order of each neighboring node among the 26 neighboring nodes of the current node in raster scan order. Each element NOT[i] has three dimensions, where the first dimension represents the relative position difference along the x-coordinate, the second dimension represents the relative position difference along the y-coordinate, and the third dimension represents the relative position difference along the z-coordinate. Therefore, the neighbor offset table NOT[i]i=0,1,...,26 can be defined as follows: {{-1,-1,-1},{-1,-1,+0},{-1,-1,+1},{-1,+0,-1},{-1,+0,+0},{-1,+0,+1},{-1,+1,-1},{-1,+1,+0},{-1,+1,+1},{+0,-1,-1},{+0,-1,+0},{+0,-1,+1},{+0,+0,-1},{+0,+ 0,+0},{+0,+0,+1},{+0,+1,-1},{+0,+1,+0},{+0,+1,+1},{+1,-1,-1},{+1,-1,+0},{+1,-1,+1},{+1,+0,-1},{+1,+0,+0},{+1,+0,+1},{+1,+1,-1},{+1,+1,+0},{+1,+1,+1}}.

[0128] The neighbor offset table NOT can be used to allow each node to look up its neighbors at each depth and then obtain its neighbor pattern, which can be used for entropy coding.

[0129] In some embodiments, the table NOT[i] i=0,1,...,25 has 26 elements, including 26 elements for neighboring nodes and no element for the current node, and thus the neighbor offset table NOT[i] i=0,1,...,25 can be defined as follows: {{-1,-1,-1},{-1,-1,+0},{-1,-1,+1},{-1,+0,-1},{-1,+0,+0},{-1,+0,+1},{-1,+1,-1},{-1,+1,+0},{-1,+1,+1},{+0,-1,-1},{+0,-1,+0},{+0,-1,+1},{+0,+0,-1}, {+0,+0,+1},{+0,+1,-1},{+0,+1,+0},{+0,+1,+1},{+1,-1,-1},{+1,-1,+0},{+1,-1,+1},{+1,+0,-1},{+1,+0,+0},{+1,+0,+1},{+1,+1,-1},{+1,+1,+0},{+1,+1,+1}}.

[0130] For each node, the determined offset table NOT can be used to search the 26 neighboring nodes at the same depth as the current node to determine their occupancy. As shown in Figure 16(a), the center node is the node whose child occupancy bit is coded, and since the nodes are coded in raster scan order, in Figure 16(b), the node is a node that has already been coded / decoded if it is occupied (the node is in list L). d). Therefore, the child occupancy of the node / volume in Figure 16(b) is already known; if a node exists, this is defined by the coded / decoded child occupancy bit; if a node does not exist, there is no occupied child node. In Figure 16(c), since the nodes have not yet been coded / decoded, the corresponding child occupancy bit has not yet been coded, and therefore these nodes are not yet known. However, if a node exists, this means that at least one child node is occupied; if a node does not exist, this means that there is no occupied child node. This is because the node occupancy bit (a flag indicating whether a node is occupied or not) is already known for each node / volume at the current depth, since it is obtained from the child occupancy bit coding / decoding of the node at the previous depth. Therefore, the node occupancy (1 bit) of the adjacent nodes shown in Figure 16(c) is defined by the child occupancy bit of the list L. d The determination can be made depending on whether or not there is a corresponding node in the

[0131] Therefore, the neighbor node occupation includes two parts. 1. Part consists of the child occupancies of already coded nodes, and their locations are looked up by using the elements in the table NOT for i=0 to i=12. In some embodiments, neighbor node occupancies are implemented as a table childOccupancy[i], i=0,1,...,12, where each element in the table childOccupancy consists of 8 bits for the child occupancy if the corresponding neighbor node exists and is occupied, and is initialized or set to 0 if the node does not exist. 2. The other part consists of the node occupancies of 13 neighboring nodes that have not yet been coded, and their positions are looked up by using the elements in the table NOT from i=14 to i=26 (i.e., i=13 may represent the current node). In some embodiments, the neighboring node occupancies are implemented as a table depthOccupancy[i], i=14,1,...,26, where each element in the table depthOccupancy consists of a flag (1 bit) indicating whether the corresponding neighboring node is occupied or not (i.e., whether the node is present or not), and this flag is set to 0 for initialization.

[0132] In some embodiments, a detailed process for determining the occupancy of a node's neighbors at each depth is shown in Figures 17a and 17b, 18, and 19. Figures 17a and 17b show a flowchart of this process, Figure 18 shows a detailed process for determining the childOccupancy of a node's 13 neighbors, and Figure 19 shows a detailed process for determining the depthOccupancy of a node's 13 neighbors.

[0133] As shown in Figures 17a and 17b, the method for determining the neighbor node occupation of a given node follows the steps below. A list L that stores nodes at depth d and in raster scan order d Get. For each depth, define a pointer table occCtxNP[i], i=0,1,…,26, where each element in the table is a pointer to a list L d The pointer to the neighboring node of the currently processed node points to the address of the node in This table is used to keep track of the neighboring nodes of the processed node and is updated after searching for neighboring nodes for the currently processed node, and node N k The tracked neighbors of node N k-1 At the start of depth d, the tracking table is obtained by updating the list Ld In one variant, each element of the table occCtxNP[i], i=0, 1, ..., 26 is initialized to be the address of the first node in the list L d in the index of the neighboring node of the currently processed node, which corresponds to the index of the node in List L according to raster scan ordering d The kth node N k (for each depth, k starts from 0), and at depth d, we get node N k Encode the following. At depth d, node N k Encode the following. Define a pointer nextNodeP and use it to find the position of each adjacent node in the list L d Iterate over each node in and find the nodes that belong to this position in list L d Find out whether it exists in Node N by using the pointer nextNodeP, the pointer table occCtxNP, and the determined neighbor offset table NOT according to the steps shown in FIG. 18 k The first part of the childOccupancy of the neighboring nodes is determined, and the detailed process is described in section (a) below. Node N by using the pointer nextNodeP, the pointer table occCtxNP, and the determined neighbor offset table NOT according to the steps shown in FIG. 19. k Determine the second part of the neighbor node occupancy depthOccupancy, and the detailed process is described in section (b) below. After obtaining the occupancy of 26 neighboring nodes, node N k Node N used for context selection in entropy coding of k Construct a neighboring pattern of

[0134] (a) Node N k The detailed process for determining the child occupancy of the 13 neighboring nodes is shown below. Based on the defined neighbor offset table NOT, iterate through each neighbor node position in Figure 16(b), where i increases from 0 to 12. Specifically, · i starts from 0, and the pointer nextNodeP is nextNodeP=occCtxNP[i]+1, This is the list L d occCtxNP[i] points to the address of the next node of occCtxNP[i] in the current depth, and occCtxNP[i] holds the index of the adjacent node at the corresponding position of the previously processed node. Therefore, nextNodeP can search for the index of the adjacent node of the currently processed node based on the previous adjacent information of the previously processed node at the current depth. The address of nextNodeP is in list L d If the node is not lower than the current node in the tree, it means that the node pointed to by the pointer nextNodeP has not yet been coded, and therefore cannot provide a child occupancy as the neighbor occupancy of the current node, and then interrupt the iteration and proceed to the step of determining depthOccupancy. Otherwise, determine the neighbor position NeighP[i] based on the position of the current node and NOT[i] by the following formula: NeighP[i]=CurP+NOT[i] where CurP is the position of the current node. Using the predicted neighbor node position NeighP[i], In the loop, the pointers occCtxNP[i] and nextNodeP are iteratively incremented by one until the address of the node pointed to by nextNodeP is greater than the address of the current node, or until the position of the node pointed to by nextNodeP is no longer lower than the expected neighbor node position NeighP[i]. The address of the node pointed to by nextNodeP is in the list L dis lower than the current node in the list L and the position of the node pointed to by nextNodeP is equal to the expected neighbor position NeighP[i]. d (The node pointed by nextNodeP in belongs to the neighbor node position NeighP[i] of the current node, which means the corresponding neighbor node of the current node is occupied.) · The i-th element of the neighbor occupancy childOccupancy[i] is determined as the 8-bit child occupancy of the node pointed to by nextNodeP. Next, determine the childOccupancy of the two neighboring nodes in the same tube as Neigh[i], which are above the node position NeighP[i]. In the preferred embodiment, the third coordinate of NeighP[i] is incremented, and the pointer nextNodeP is also incremented to determine the occupancy of the two neighboring nodes above the neighboring node position NeighP[i], which will be explained in detail later.

[0135] (b) Node N k The detailed process for determining the depthOccupancy of the 13 neighboring nodes is described below. First, to determine the occupancy of the nodes above the current node, i.e., depthOccupancy[0], the pointer nextNodeP is incremented by 1, and the node pointed to by nextNodeP is added to the list L. d Determine whether the node is not the last node in the The node pointed to by nextNodeP is in list L d If it is not the last node in list L, then increase the third coordinate of the current node by 1. If the position of the node pointed to by nextNodeP and the position of the adjacent node above the current node are the same (this means that the adjacent node above the current node is occupied and the node is in list L), d (meaning that the node is stored in the local node), set the neighbor node occupancy depthOccupancy[0] as true. Next, based on the defined neighbor offset table NOT, we iterate through each neighbor node position in Figure 16(c) while increasing i from 15 to 26 to determine the flags of other neighbor nodes for depthOccupancy. ·i starts from 15 and the pointer nextNodeP is defined as follows: nextNodeP=occCtxNP[i]+1 This is the list L d Points to the address of the next node of occCtxNP[i] in The address of nextNodeP is in list L d If it is not the last node in , then determine the neighbor position NeighP[i] based on the current node and the position in the table NOT[i] by the following formula: NeighP[i]=CurP+NOT[i] Using the predicted neighbor node position NeighP[i], In the loop, the address of the node pointed to by nextNodeP is added to the list L. d The pointers occCtxNP[i] and nextNodeP are iteratively incremented by one until the node is no longer smaller than the last node in the tree or the position of the node pointed to by nextNodeP is no longer lower than the expected neighbor node position NeighP[i]. The address of the node pointed to by nextNodeP is in the list L d is lower than the last node in the list L and the position of the node pointed to by nextNodeP is equal to the expected neighbor node position NeighP[i] (this is d (The node pointed by nextNodeP in belongs to the neighbor node position NeighP[i] of the current node, which means the corresponding neighbor node of the current node is occupied.) · The i-th element of the adjacent occupancy depthOccupancy[i-14] is determined to be true. Next, determine the depthOccupancy of the two neighboring nodes in the same tube as Neigh[i]. In the preferred embodiment, the third coordinate of NeighP[i] is incremented, and the pointer nextNodeP is also incremented to determine the occupancy of the two neighboring nodes above the neighboring node position NeighP[i], which will be explained in detail later.

[0136] (c) List provided d The method for searching for neighboring nodes in is described below.

[0137] If the pointers occCtxNP[i] and nextNodeP are used to search each of the 26 adjacent nodes, after all nodes at a given depth have been processed, the search for adjacent nodes is repeated at most once for all nodes at the same depth. Therefore, the overall complexity for finding adjacent nodes can be expressed as O(N) for encoding the child occupancies of N nodes at a given depth. By searching M adjacent nodes, the complexity can be expressed as O(M*N).

[0138] To further reduce the complexity, it is provided to group the search for neighboring nodes by a vertical tube of neighboring nodes, with each successive position in the tube corresponding to a node index incremented by 0 or 1 in the raster scan ordered list of the node (depending on the occupancy of the corresponding volume and the presence or absence of the node in the list of processed nodes). Only the first neighboring node in the tube is searched. As mentioned above, the pointers occCtxNP[i] and nextNodeP point to the first neighboring node in the cubic tube of 26 neighboring nodes (list L d 18 and 19, the index i is incremented by 3, and the expected neighbor node position NeighP[i] in each loop is the bottom node in the tube, which will be explained in more detail later.

[0139] By using such a simple neighbor search method, the complexity added by the neighbor search is practically negligible, and only limited additional memory is required to store the previous position of each required neighbor relative position, or to store only the previous position of the first neighbor node in each tube of neighbor nodes.

[0140] Determine the childOccupancy of two neighboring nodes in the same tube as NeighP[i].

[0141] After obtaining the occupancy of the bottom neighboring node of the tube located at NeighP[i], the next step is to determine the childOccupancy of the two neighboring nodes above the node located at NeighP[i] that are in the same tube as NeighP[i], where the tube is the cube of the 26 neighboring nodes of the currently processed node. The detailed process is shown in Figure 20 and can be described as follows: First, determine the maximum index i_tubeEnd of the neighboring nodes in the same tube as NeighP[i], where i_tubeEnd is defined as follows:

number

[0142] Determine the depthOccupancy of two neighboring nodes in the same tube as NeighP[i].

[0143] After obtaining the occupancy of the bottom neighboring nodes of the tube located at NeighP[i], the next step is to determine the depthOccupancy of the two neighboring nodes above the node located at NeighP[i] that are in the same tube as NeighP[i], where the tube is the cube of the 26 neighboring nodes of the currently processed node. The detailed process is shown in Figure 21 and can be described as follows: First, determine the maximum index i_tubeEnd of the neighboring nodes in the same tube as NeighP[i], where i_tubeEnd is defined as follows:

number

[0144] When full raster scan ordering for encoding child node occupancy is used, as in the first example (Example 1) of the method described above, the child nodes are encoded in raster scan ordering at each depth, and therefore the neighboring information used to entropy code each child node is based on the neighboring child nodes of the current child node to be encoded.

[0145] Since the child nodes of all occupied nodes at depth d are coded in raster scan order, the adjacent child nodes below the current child node in the raster scan order have already been coded, and their 1-bit occupancies are already known, while the adjacent child nodes above the current child node have not yet been coded, and their 1-bit occupancies are not yet known. Therefore, only the adjacent child nodes below the current child node are searched for neighbor search, so there are 13 adjacent child nodes to be searched, as shown in FIG. 22, where each cube represents a child node and the green cube represents the current child node to be coded. And each adjacent child node has a relative position to the current child node, so the corresponding adjacent child node position offset table CNOT[i], i=0,1,...,12 is: {{-1,-1,-1},{-1,-1,+0},{-1,-1,+1},{-1,+0,-1},{-1,+0,+0},{-1,+0,+1},{-1,+1,-1},{-1,+1,+0},{-1,+1,+1},{+0,-1,-1},{+0,-1,+0},{+0,-1,+1},{+0,+0,-1}}, The index i is based on the raster scan order of the adjacent child nodes.

[0146] How to determine child neighbor node occupancy:

[0147] To get the child neighbor occupancy of the current node, For the 13 adjacent child nodes of the current child node, list L d Define a three-dimensional pointer table occCtxNodes that points to the addresses of the nodes in the list L. dstores the child nodes of all occupied nodes at depth d in raster scan order, and the table occCtxNodes stores, for each depth, a list L d It is initialized to be the address of the first child node in List L d A pointer nextNodeP is defined to point to the next node of one adjacent child node in, which is given by the following formula: nextNodeP=occCtxNodes[i]+1 For each neighboring child position, list L d By searching the child nodes in, we determine the occupancy consisting of 1-bit occupancies of 13 child neighbors. The child adjacent node position NeighCP[i], which is the bottom child node of the tube, is obtained by the following formula: NeighCP[i]=CurCNode+CNOT[i] To obtain the occupancy of the node located in NeighCP[i], the pointers nextNodeP and occCtxNodes[i] are used to find the occupancy of the node located in NeighCP[i]. d Iterate over the child nodes in. The detailed process is the same as in the preferred embodiment of the provided method. For the two child neighbors above the node located in NeighCP[i], since the neighbor child nodes are stored in raster scan order, to obtain their occupied bits, we simply increase the third coordinate of NeighCP[i] by 1 or 2, and also increase nextNodeP by 1 or 2. The detailed process is the same as that in the preferred embodiment of the provided method.

[0148] In some embodiments (e.g., Figures 5-8), to encode / decode the 8-bit child occupancy of each node at a depth, the encoding of each node is achieved by processing the node four times, performing a search for neighboring nodes each time the node is processed (i.e., for each tube of child nodes), thus increasing the complexity roughly by a factor of up to four.

[0149] To efficiently process each tube of nodes, rather than using an O(log(N)) search to find the first node of the tube (or slice) and its neighbors, each time a new tube (or slice) is started, they can be stored in memory the first time they are encountered. Two local copies of the node index and its neighbors' indices are sufficient to maintain overall linear complexity. Each time a node is processed for the first time in slice 0 processing, the first copy is reset with the location of the node and its neighbors. When slice 1 processing is starting, this copy is used to reinitialize the search for the node and its neighbors. Each time a node is processed for the first time in Tube 0 or Tube 2 processing, a second copy is reset with the location of the node and its neighbors. When Tube 1 or Tube 3 processing is starting, this copy is used to reinitialize the search for the node and its neighbors.

[0150] In some embodiments, for child raster scan encoding ordering, the pointer for the parent node is restored to the beginning of the tube / slice each time iterating over the child node's tube[1 or 3] / slice[1] (adjacency patterns must be preserved when starting the (parent) node's tube [i.e., child node's tube 0 and tube 2] and when starting the (parent) node's slice [i.e., slice 0]).

[0151] When iterating through the list of nodes, the end of a node's tube or the end of a node's slice can be found by comparing the next node position with the current node position: if at least one of the first coordinate (x) and second coordinate (y) (in the lexicographical order used) is different, this indicates the end of a tube, and if at least the first coordinate is different, this indicates the end of a slice (x).

[0152] Referring now to Figure 23, there is shown a schematic flow chart of a method for encoding a 3D point cloud into a bitstream according to the present invention.

[0153] The method includes the following steps.

[0154] In step S10, a node is obtained that contains at least a portion of the points at depth d in lexicographical order along each of the axes X, Y, and Z of the coordinate system.

[0155] In step S11, each node N k Determine the neighboring pattern of

[0156] In step S12, the occupancy information of each node is entropy coded into a bitstream based on the neighboring pattern.

[0157] Referring now to Figure 24, there is shown a schematic flow chart of a method for decoding a 3D point cloud from a bitstream according to the present invention.

[0158] The method includes the following steps.

[0159] In step S20, a node is obtained that contains at least a portion of the points at depth d in lexicographic order along each of the axes X, Y, and Z of the coordinate system.

[0160] In step S21, each node N k Determine the neighboring pattern of

[0161] In step S22, the occupancy information of each node is entropy decoded from the bitstream based on the neighboring pattern.

[0162] Referring now to FIG. 25 , a simplified block diagram of an exemplary embodiment of an encoder 1100 is shown. The encoder 1100 includes a processor 1102 and a memory storage device 1104. The memory storage device 1104 can store a computer program or application containing instructions that, when executed, cause the processor 1102 to perform operations such as those described herein. For example, the instructions can encode and output a bitstream encoded according to the methods described herein. It should be understood that the instructions can be stored on a non-transitory computer-readable medium, such as a compact disc, a flash memory device, a random access memory, a hard drive, or the like. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions, such that it operates as a dedicated processor implementing the described processes. In some examples, such a processor may be referred to as a “processor circuit” or “processor circuitry.”

[0163] Referring now to FIG. 26 , a simplified block diagram of an exemplary embodiment of a decoder 1200 is shown. The decoder 1200 includes a processor 1202 and a memory storage device 1204. The memory storage device 1204 may include a computer program or application containing instructions that, when executed, cause the processor 1202 to perform operations such as those described herein. It should be understood that the instructions may be stored on a computer-readable medium, such as, for example, a compact disc, a flash memory device, a random access memory, a hard drive, or the like. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions, such that it operates as a special-purpose processor implementing the described processes and methods. In some examples, such a processor may be referred to as a “processor circuit” or “processor circuitry.”

[0164] It should be understood that a decoder and / or encoder according to the present application may be implemented in several computing devices, including, but not limited to, a server, a suitably programmed general-purpose computer, a machine vision system, and a mobile device. The decoder or encoder may be implemented by software that includes instructions for configuring one or more processors to perform the functions described herein. The software instructions may be stored in any suitable non-transitory computer-readable memory, including a CD, RAM, ROM, flash memory, etc.

[0165] It should be understood that the modules, routines, processes, threads, or other software components implementing the decoders and / or encoders described herein, as well as the methods / processes for configuring the encoders or decoders, can be implemented using standard computer programming techniques and languages. The application is not limited to particular processors, computer languages, computer programming conventions, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated chip (ASIC), etc.

[0166] The present application also provides a computer readable signal encoding data produced by application of an encoding process according to the present application.

[0167] Certain adaptations and modifications of the described embodiments can be made. The above-described embodiments are therefore to be considered as illustrative and not restrictive. In particular, the embodiments can be freely combined with one another.

Claims

1. Preferably, a method for encoding a 3D point cloud into a bitstream, applied in an encoder, comprising the steps of: the geometry of the point cloud is defined by an octree structure having a plurality of nodes; the plurality of nodes have a parent-child relationship and represent three-dimensional positions of objects; the point cloud is located within a volume space of a three-dimensional coordinate system that is recursively divided into sub-volumes and includes points of the point cloud, the volume being divided into a set of sub-volumes; each child subvolume is associated with a respective node of the octree structure, and occupancy information associated with each child subvolume indicates whether the child subvolume contains at least one of the points of the point cloud; The method comprises: obtaining a node that includes at least a portion of the points at a depth d in lexicographic order along each of the axes X, Y, and Z of the coordinate system; Each node N based on a subset of its neighbors ｋ determining a neighborhood pattern of and entropy coding the occupancy information of each node into a bitstream based on the neighboring pattern.

2. Preferably, a method for decoding a 3D point cloud from a bitstream, applied in a decoder, comprising the steps of: the geometry of the point cloud is defined by an octree structure having a plurality of nodes, the plurality of nodes having a parent-child relationship and representing three-dimensional positions of objects; the point cloud is located within a volume space of a three-dimensional coordinate system that is recursively divided into sub-volumes and includes points of the point cloud, the volume being divided into a set of sub-volumes; each child subvolume is associated with a respective node of the octree structure, and occupancy information associated with each child subvolume indicates whether the child subvolume contains at least one of the points of the point cloud; The method comprises: obtaining a node that includes at least a portion of the points at a depth d in lexicographic order along each of the axes X, Y, and Z of the coordinate system; Each node N based on a subset of its neighbors ｋ determining a neighborhood pattern of and entropy decoding the occupancy information of each node from the bitstream based on the neighboring pattern.

3. The method of claim 1 or 2, wherein the method further comprises the step of storing the occupied child nodes at depth d+1 in lexicographic order.

4. The method of claim 1 , wherein the lexicographic order of the nodes at depth d and / or the lexicographic order of the child nodes at depth d+1 is a raster scan order.

5. 5. The method of claim 1, wherein the entropy encoding or entropy decoding of the occupancy information of the child nodes of each of the nodes into or from the bitstream is performed based on a lexicographical order of the nodes at depth d and a fixed order of the child nodes of a given node.

6. Storing the occupied child nodes at depth d+1 is 6. A method according to any one of claims 3 to 5, performed by processing each node in a single pass for all nodes of the obtained nodes, preferably by ordering the child nodes in lexicographical order, preferably in raster scan order, using intermediate storage.

7. N ｋ A subset of neighbors of,is stored using a tracking table, The tracking table limits the number of neighbors stored, and preferably the tracking table limits the number of neighbors stored by node N ｋ－１ The method according to claim 1 , wherein the tracking table is obtained by updating the corresponding tracking table of the

8. the tracking table is a first adjacent offset table (NOT); The first NOT is the current node N ｋ 8. The method of claim 7, further comprising storing the relative positions of the neighboring nodes with respect to the neighboring nodes.

9. The node obtained at depth d is stored in a first link data structure; the subset of neighbor nodes is obtained by iterating over the nodes in the first link data structure; The method of claim 1 , wherein the first link data structure has elements that connect to each other such that the elements are arranged sequentially.

10. The tracking table is updated by grouping the subset of neighboring nodes found using vertical tubes of neighboring nodes; 10. The method of claim 7, wherein each vertical tube comprises nodes that share the same coordinates in two axes of the coordinate system.

11. the adjacent pattern is a binary word indicating adjacent occupancy; The binary word is obtained based on a first occupancy and a second occupancy of neighboring nodes; 11. The method of claim 1, wherein the first occupancy of the adjacent node has a lower coordinate value than the current node on at least one axis, the first occupancy of each adjacent node includes an 8-bit child occupancy, the second occupancy of the adjacent node has a higher coordinate value than the current node on at least one axis, and the second occupancy of each adjacent node includes a flag indicating the occupancy of the corresponding adjacent node.

12. The method of claim 1 , wherein the child nodes of all occupied nodes are encoded or decoded in a raster scan order of the child nodes.

13. the child nodes are stored in a second link data structure; Current child node CN ｋ a subset of adjacent child nodes for a given adjacent child node position of is obtained by iterating over the child nodes in the second link data structure; The method of claim 12 , wherein the second link data structure has elements that connect to each other such that the elements are arranged sequentially.

14. 14. The method of claim 13, wherein the subset of adjacent child nodes is stored using a second adjacent offset table (CNOT), the second NOT storing the relative positions of the adjacent child nodes with respect to the current child node.

15. 15. The method of claim 13 or 14, wherein the occupancy of an adjacent child node having a lower coordinate value than the current child node in the second link data structure is a flag indicating the corresponding occupancy of the adjacent child node.

16. 16. The method of claim 15, wherein the subset of adjacent child nodes represents tracked adjacent child nodes in the second link data structure of the current child node and is updated for adjacent child node search of a next child node.

17. 1. An encoder for encoding a 3D point cloud into a bitstream, comprising:

17. An encoder comprising at least one processor and a memory, said memory storing instructions that, when executed by said processor, perform the steps of the method of any one of claims 1 and 3 to 16.

18. 1. A decoder for decoding a 3D point cloud from a bitstream, comprising: A decoder comprising at least one processor and a memory, said memory storing instructions that, when executed by said processor, perform the steps of the method of any one of claims 2 to 16.

19. A bitstream encoded by the method of any one of claims 1 and 3 to 16.

20. A computer-readable storage medium comprising instructions which, when executed by a processor, perform the steps of the method of any one of claims 1 to 16.

Images