Method, encoder and decoder for encoding and decoding 3d point clouds

Abstract

An encoding method and a decoding method, as well as an encoder and a decoder, for encoding a 3D point cloud into a bitstream. The geometry of the point cloud is defined by an octree structure comprising a plurality of nodes having a parent-child relationship and representing the three-dimensional position of an object, the point cloud being located within a volumetric space of a three-dimensional coordinate system, the volumetric space being segmented step by step into sub-volumes and containing points of the point cloud, wherein the volume is divided into a number of sub-volumes, each sub-level sub-volume being associated with one of the plurality of nodes of the octree structure, and wherein occupancy information associated with each of the sub-level sub-volumes indicates whether the respective sub-level sub-volume contains at least one of the points of the point cloud, the method comprises the following steps: along each axis X, Y and Z of a coordinate system, acquiring nodes which contain at least a part of point clouds and have the depth of d in a dictionary type sorting manner; determining an adjacent mode of each node Nk based on a part of adjacent nodes; the occupancy information of each node is entropy coded into a bitstream based on the neighboring pattern.

Description

Technical Field

[0001] The present application relates to a method for encoding a 3D point cloud into a bitstream. In addition, the object of the present application is to provide a method for decoding a 3D point cloud from a bitstream. In addition, the object of the present application is to provide an encoder and a decoder, a bitstream encoded according to the present application, and software. In particular, the object of the present application is to provide a method for retrieving the occupancy of adjacent nodes and / or adjacent subnodes while encoding or decoding an occupancy tree without sacrificing memory and algorithmic complexity. Background Art

[0002] Point cloud, as a format for representing 3D data, has attracted much attention due to its versatile ability to represent all types of 3D objects or scenes. As a result, point cloud can handle a variety of usage scenarios, including:

[0003] Film post-production,

[0004] Real-time 3D immersive telepresence or VR (virtual reality) / AR (augmented reality) applications,

[0005] Free viewpoint video (e.g. for watching sports events),

[0006] Geographic Information Systems (also known as Cartography),

[0007] Cultural heritage (scanning and storing digitally rare objects),

[0008] Autonomous driving, including 3D mapping of the environment and real-time lidar data collection.

[0009] A point cloud is a set of points located in 3D space, each of which can have values ​​attached to it. These values ​​are often called point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0010] These attributes may be, for example, three-component color, material properties (such as reflectivity), and / or a two-component normal vector of the surface associated with the point.

[0011] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, Lidar, scanners, or they can be computer-generated (for example, in movie post-production). Depending on the use case, point clouds in mapping applications may contain thousands to billions of points.

[0012] In the raw representation of point clouds, each point requires a very high number of bits, at least a dozen bits per spatial component X, Y or Z, and optionally more bits for attributes, e.g. color requires three times 10 bits. Practical deployment of point cloud-based applications requires compression techniques that enable the storage and distribution of point clouds within reasonable storage and transmission infrastructure conditions.

[0013] The compression may be lossy (similar to lossy video compression) during the distribution of point clouds to end users and visualization by the user terminal, such as on AR / VR glasses, or any other 3D-enabled device. Other use cases require lossless compression, such as medical applications or autonomous driving, to avoid compromising the decision-making results obtained by analyzing the compressed and transmitted point cloud.

[0014] Until recently, point cloud compression (also known as PCC) has not been addressed by the mass market and no standardized point cloud codec is available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11 (also known as the Moving Picture Experts Group or MPEG) has started a work project on point cloud compression. The project involves two standards, namely:

[0015] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC); and

[0016] MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-based Point Cloud Compression (G-PCC).

[0017] The V-PCC and G-PCC standards have completed their first versions at the end of 2020 and will soon be available in the market.

[0018] The V-PCC coding method compresses point clouds by performing multiple projections of 3D objects to obtain 2D patches that are packed into images (or videos, for mobile point clouds). The obtained images or videos are then compressed using existing image / video codecs, making it possible to use already deployed image and video solutions. By its nature, V-PCC is only effective on dense and continuous point clouds, as image / video codecs cannot compress non-smooth patches obtained from the projection of sparse geometric data collected, such as Lidar.

[0019] The G-PCC coding method has two schemes for compression geometry.

[0020] The first scheme is based on an occupancy tree (octree / quadtree / binarytree) representation of the point cloud geometry. Occupied nodes are forked until a certain size is reached, at which point the occupied leaf nodes provide the positions of the points, usually the centers of these nodes. A high degree of compression can be obtained for dense point clouds by using neighborhood-based prediction techniques. Sparse point clouds are also addressed by directly encoding the positions of points within nodes with non-minimum sizes, by stopping the construction of the tree when there are only isolated points in the node; this technique is called Direct Coding Mode (DCM).

[0021] The second scheme is based on a prediction tree, where each node represents the 3D position of a point and the relationship between nodes is the spatial prediction from parent to child nodes. This method can only solve sparse point clouds and has the advantages of lower latency and simpler decoding than occupancy trees. However, the compression performance is only slightly better than the first occupancy-based scheme and has complex encoding, intensively searching for the best predictor (in a long list of potential subvolume predictors) when building the prediction tree.

[0022] In both schemes, attribute encoding (de)coding is performed after geometry encoding (de)coding is completed, resulting in two-pass coding. Therefore, low latency is achieved by using slices that decompose the 3D space into independently coded sub-volumes without prediction between sub-volumes. When many slices are used, compression performance may be severely affected.

[0023] An important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves with respect to time. In addition, AR / VR point clouds are usually locally 2D, as they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly correlated (or dense), in the sense that points in them are rarely isolated, but have many neighbors. Summary of the invention

[0024] The purpose of the present application is to provide a method for encoding a 3D point cloud into a bitstream, and a method for decoding the geometry of a 3D point cloud from a bitstream, reducing the memory usage when retrieving the occupancy of neighboring nodes and / or neighboring subnodes.

[0025] This problem is solved by the encoding method of claim 1 , the decoding method of claim 2 , the encoder of claim 10 , the decoder of claim 11 , the bitstream of claim 12 , and the software of claim 13 .

[0026] In a first aspect, a method for encoding a 3D point cloud into a bitstream is provided, preferably applied to an encoder. The geometry of the point cloud is defined by an octree structure, the octree structure includes a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is divided into sub-volumes step by step and contains points of the point cloud, wherein the volume is divided into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each sub-volume indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, and the method comprises:

[0027] Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order;

[0028] Determine each node N based on a portion of its neighboring nodes k The adjacent mode of

[0029] The occupancy information of each node is entropy encoded into a bit stream based on the neighbor pattern.

[0030] A point cloud is a set of points in a three-dimensional coordinate system. These points are generally intended to represent the outer surface of one or more objects. Each point has a location (position) in a three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be a Cartesian or any other coordinate system. Therefore, according to the present application, for a given depth d in the octree, the nodes are obtained in a lexicographic order. Using lexicographic ordering (regardless of the ordering of the coordinate axes in the lexicographic ordering), the occupancy of any node with a lower z position, a lower y position, and a lower x position can be used for context analysis. Preferably, the nodes are stored in a linked data structure (e.g., a linked list). Among them, the linked data structure has elements connected to each other so that the elements are arranged in a sorted manner. Preferably, a list L can be used. d to store nodes, and nodes can be iteratively obtained from the list. In addition, the adjacent pattern of each node is determined based on a subset of adjacent nodes. According to the technical solution of the present application, when determining the adjacent pattern, the lexicographic ordering of the nodes is considered, and only a part of the adjacent nodes is considered, which can avoid wasting too much memory space in the occupancy map.

[0031] Preferably, the method further comprises storing the occupied child nodes at depth d+1 in lexicographic order. Thus, if the node at depth d is not a leaf node, the octree can continue to be traversed in lexicographic order. The child nodes at the next depth d+1 are already stored in a preferred order, thereby further facilitating the next iteration.

[0032] Preferably, the lexicographic ordering of nodes at depth d and / or the lexicographic ordering of nodes at depth d+1 is in raster scan order.

[0033] Preferably, entropy encoding the occupancy information of each node into the bitstream, or entropy decoding the occupancy information of the child nodes of each node from the bitstream, is based on a lexicographic ordering of nodes with a depth of d and a fixed ordering of the child nodes of a given node. Therefore, the occupied nodes are processed in a raster scan ordering, but the occupied bits of all sub-volumes of the node are encoded continuously in a fixed ordering. Compared with the prior art of encoding nodes in Morton ordering, due to the use of lexicographic ordering for scanning, the neighboring information is more reliable and the encoding performance is better. Preferably, the encoding or decoding of the child nodes is based on a raster scan ordering.

[0034] Preferably, the storage of occupied child nodes of depth d+1 is performed by processing each of all nodes obtained at one time, preferably by using an intermediate storage to sort the child nodes lexicographically, preferably by raster scanning. For example, the child nodes may be sorted by processing them once using an intermediate list, so that each node does not need to be processed 4 times in order to encode all child nodes. The encoding complexity is reduced and the encoding performance is not affected.

[0035] Preferably, N k A portion of the adjacent nodes of is stored in a tracking table, wherein the tracking table limits the number of adjacent nodes to be stored. Preferably, the N k The tracking table is obtained by k-1 Therefore, by providing a tracking table, it is not necessary to retrieve all relevant adjacent nodes every time a context analysis is performed on a node. The tracking table can be a dynamic table that keeps the next node updated. Preferably, the tracking table only stores the current node N. k The nearest neighboring node of node N. That is, the neighboring node can be k Share faces, edges, or vertices. Thus, the memory usage and complexity of searching for neighboring nodes is minimized.

[0036] Preferably, the tracking table is a first adjacent offset table NOT, wherein the first NOT stores adjacent nodes relative to the current node N k The relative positions of the neighboring nodes are defined in NOT. NOT may or may not include the current node (i.e., relative positions {0, 0, 0}). Furthermore, NOT can be easily expanded or reduced according to the defined radius (e.g., the radius can be 1, 2, 3, etc., indicating how many nearest neighboring nodes are considered when determining the neighboring pattern).

[0037] It should be understood that all of the technical solutions previously described with respect to tracking neighboring nodes and / or using raster scan ordering are also applicable to entropy encoding and / or decoding of child nodes.

[0038] Preferably, the obtained nodes with a depth of d are stored in a linked data structure, and the part of adjacent nodes is obtained by iterating the nodes in the linked data structure, wherein the linked data structure includes elements associated with each other so that the elements are arranged in a sorted manner. The linked data structure for storing nodes may be a linked list L d Preferably, an index or pointer can be used to access the list L. d Iterate the nodes in to search for the adjacent nodes at a given adjacent position of the current node.

[0039] Preferably, the tracking table is updated by grouping the portion of adjacent nodes searched using vertical pipelines of adjacent nodes, wherein each vertical pipeline includes nodes having the same coordinates on two axes of the coordinate system. Each vertical pipeline is composed of all nodes having the same x and y coordinates (or the same x and z, or the same y and z). The pipeline is further combined Figure 5 To explain. Since the nodes share the same two first coordinates, and the third coordinates have a finite offset difference, each successive position in the pipeline will correspond to increasing the node index by a finite amount in the raster scan sorted list of nodes. Therefore, by applying grouping, the complexity of the encoding is further reduced.

[0040] Preferably, the neighbor pattern is a binary word indicating neighbor occupancy, the binary word is obtained based on a first occupancy and a second occupancy of a neighboring node, the first occupancy of the neighboring node having a smaller coordinate value than the current node on at least one axis, the first occupancy of each neighboring node including an 8-bit sub-occupancy, the second occupancy of the neighboring node having a larger coordinate value than the current node on at least one axis, and the second occupancy of each neighboring node including a flag bit indicating the occupancy of the corresponding neighboring node, preferably 1 bit. Preferably, the binary word is obtained from the tracked neighboring node of the above-mentioned tracking table.

[0041] Preferably, the child nodes of all occupied nodes are encoded or decoded according to the raster scan order of the child nodes.

[0042] Preferably, the child nodes are stored in a second link data structure, and the current child node CN is obtained by iterating the child nodes in the second link data structure. k A portion of adjacent child nodes of a given adjacent child node position of a second linked data structure, wherein the second linked data structure has elements associated with each other such that the elements are arranged in a sorted manner.

[0043] Preferably, the part of adjacent sub-nodes is stored in a second adjacent offset table NOT, wherein the second NOT stores the relative positions of the adjacent sub-nodes with respect to the current sub-node.

[0044] Preferably, the occupancy of the adjacent child node having a smaller coordinate value than the current child node in the second link data structure is a flag bit indicating the corresponding occupancy of the adjacent child node, preferably 1 bit.

[0045] Preferably, the part of neighboring child nodes represents the neighboring child nodes of the current child node that are tracked in the second link data structure, and are updated for neighboring child node search of the next child node.

[0046] Another aspect of the present application provides a method for decoding a 3D point cloud from a bitstream, preferably applied to a decoder. The geometry of the point cloud is defined by an octree structure, the octree structure includes multiple nodes, the multiple nodes have a parent-child relationship and represent the three-dimensional position of the object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is divided into sub-volumes step by step and contains points of the point cloud, wherein the volume is divided into several sub-volumes, each sub-volume is associated with one of the multiple nodes of the octree structure, wherein the occupancy information associated with each of the sub-volumes indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, and the method comprises:

[0047] Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order;

[0048] Determine each node N based on a portion of its neighboring nodes k The adjacent mode of

[0049] Occupancy information of each node is entropy decoded from the bitstream based on the neighbor pattern.

[0050] Another aspect of the present application provides an encoder for encoding a 3D point cloud into a bit stream. The encoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned encoding method are performed.

[0051] Another aspect of the present application provides a decoder for decoding a 3D point cloud from a bit stream. The decoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned decoding method are performed.

[0052] Another aspect of the present application provides a bit stream, which is encoded by the steps of the aforementioned encoding method.

[0053] Another aspect of the present application provides a computer-readable storage medium comprising instructions for executing the steps of the method for encoding a 3D point cloud into a bitstream as described above.

[0054] Another aspect of the present application provides a computer-readable storage medium comprising instructions for executing the steps of the method for decoding a 3D point cloud from a bitstream as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Hereinafter, the present invention is described in more detail with reference to the accompanying drawings.

[0056] The accompanying drawings show:

[0057] Figure 1 An example of occupied nodes / voxels in the neighborhood is shown.

[0058] Figure 2 The neighborhood of nodes at the same depth or greater than the current node is shown.

[0059] Figure 3 An example of occupied nodes in a neighborhood is shown, involving neighboring nodes with a depth greater than the current depth.

[0060] Figure 4 Shown is the occupancy map (white cubes) known from previously encoded and decoded occupied bits when the occupied bits of a child node (striped cubes) are encoded and decoded using Morton ordering.

[0061] Figure 5 A raster scan process of a parent node for raster scan encoding the occupancy of its child nodes is shown.

[0062] Figure 6 The encoding ordering of slice 0 and slice 1 of the occupied nodes of the same fixed x at the current depth is shown.

[0063] Figure 7 An example of a decoder is shown.

[0064] Figure 8 An example of an encoder is shown.

[0065] Fig. 9 Shown is the occupancy map (white cubes) known from previously encoded occupied bits when the occupied bits of a child node (striped cubes) are encoded using raster scan ordering.

[0066] Fig.10 A raster scan ordered parent node is shown with 8 encoded child occupation bits and a raster scan ordering of child nodes.

[0067] Fig.11Another example of a decoder is shown (decoding method of parent nodes using raster scan ordering, where 8 child occupied bits are encoded together).

[0068] Fig.12 Another example of an encoder is shown (encoding method of parent nodes using raster scan ordering, where 8 child occupied bits are encoded together).

[0069] Fig.13 Shown is the occupancy map (white cubes) known from previously coded occupied bits when the occupied bits of child nodes (striped cubes) are coded and decoded using raster scan ordering of the nodes but all bits of the occupancy of child nodes are encoded using Morton ordering.

[0070] Fig.14 An overview of the decoding process using the provided method is shown.

[0071] Fig.15 An overview of the encoding process using the provided method is shown.

[0072] Fig.16 It shows that 26 neighboring nodes of the current node (central node) are encoded.

[0073] Fig.17a and 17b A flow chart of a process for determining the occupancy of neighboring nodes of a node is shown.

[0074] Fig.18 It shows that the node N is determined k The detailed process of childOccupancy of 13 adjacent nodes.

[0075] Fig.19 It shows that the node N is determined k The detailed process of depthOccupancy of 13 neighboring nodes.

[0076] Fig. 20 The detailed process of determining the childOccupancy of two adjacent nodes in the same pipeline as NeighP[i] is shown.

[0077] Fig.21 The detailed process of determining the depthOccupancy of two adjacent nodes in the same pipeline as NeighP[i] is shown.

[0078] Fig. 22 It shows the retrieval of the adjacent child nodes of the current child node, where each cube represents a child node.

[0079] Fig.23 A schematic flow chart of a method for encoding is shown.

[0080] Fig.24 A schematic flow chart of a method for decoding is shown.

[0081] Fig.25 An encoder according to the present application is shown.

[0082] Fig.26 A decoder according to the present application is shown. DETAILED DESCRIPTION

[0083] Figure 1 An example of a neighborhood is shown, where only occupied nodes / voxels are shown. The number of possible occupancy configurations in a neighborhood is 2N, where N is the number of nodes / voxels involved in the neighborhood.

[0084] In the octree representation of the point cloud, if the nodes are processed in depth-first order as in MPEG (Moving Picture Experts Group) GPCC (Geometry-based Point Cloud Compression), it may benefit from knowing the occupancy of nodes at deeper depths, such as Figure 2 shown.

[0085] The node with a depth greater than the current node is used to obtain the geometric information of the area that has not been encoded at the current depth (where the y value is greater than the y value of the current node).

[0086] In MPEG G-PCC, the combination of the current depth and the neighboring nodes at the current depth plus 1 is used to define a neighborhood. However, in order to limit the possible number of neighborhood configurations, the neighborhood has been restricted to a subset of the set of nodes that are neighbors to the current node.

[0087] Figure 3 An example of a neighborhood is shown, where only occupied nodes are drawn, also involving neighboring nodes with depth greater than the current depth.

[0088] By construction, Morton ordering also allows easy consideration of neighborhoods in a given cubic subvolume that contains all nodes at a first given depth D0 whose Morton codes share a common prefix (when written in binary format). This prefix corresponds to the Morton code of a common ancestor node at a second given depth D1, whose total volume is the same as the given cubic subvolume. Thus, the size of the subvolume in terms of the number of possible Morton codes (i.e., nodes) at a given depth is 2 N, where N is the number of bits below the Morton code prefix. Thus, N will correspond to three times the first given depth minus the second given depth: N=3x(D0-D1). In current G-PCC methods, the occupancy of such a neighborhood can, for example, be buffered in a linear memory space containing the occupancy of all possible Morton codes in the 3D volume. This representation of occupancy is sometimes referred to as an occupancy atlas in G-PCC. Therefore, the occupancy of a child node at a given child position in the volume can be directly retrieved from its position in the buffer, which corresponds to the Morton suffix of the child node (consisting of N bits).

[0089] This allows to quickly / immediately find (i.e., complexity is in O(1)) the occupancy of a given neighboring node within that volume. The overall complexity of finding neighboring node occupancy to encode and decode a sub-occupancy of N nodes will then be in O(N), but the encoding and decoding efficiency will be suboptimal because it is limited by the size of the graph (neighboring nodes outside the cube / volume represented by the graph cannot be obtained, and this reduces the encoding and decoding efficiency).

[0090] Instead of using an occupancy map, in order to avoid consuming this memory but have the same behavior as G-PCC on the boundaries of the map, the occupancy of neighboring nodes inside the data structure can be searched, thereby restricting the search to nodes contained inside the same volume as the node considered by the map. For example, a search for a given neighboring node inside a buffer containing all occupied nodes in order may have a cost logarithmic in the number of nodes inside the same volume.

[0091] Occupancy maps and search methods may be applied to obtain the occupancy of neighboring nodes at the same tree depth and the occupancy of previously encoded neighboring child nodes at the next tree depth. In the case of occupancy maps, one occupancy map / map may be used for nodes at the same tree depth and one occupancy map / map for nodes at the next depth, or they may be stored together. In the case of search methods, the occupancy of child nodes at the next tree depth may be stored in a memory structure associated with nodes at the same tree depth in order to reduce search complexity.

[0092] Outside of G-PCC, if the occupancy graph size is not bounded at all (so occupied neighbors are never lost since there are no graph boundaries), it may be preferable to not use the graph at all, since it may grow too large in memory, but then neighbor search complexity will increase.

[0093] If the search for neighboring nodes is done in a node's Morton ordered list / buffer, the overall complexity of looking for neighboring nodes to determine the context for encoding the sub-occupancy of N nodes will be O(N.log2(N)). One can try to reduce this complexity by using a hash table, but this comes at the cost of increased memory usage (compared to the list / buffer of occupied nodes); and in the general best case (if enough memory and a suitable hash function are used), the complexity will become O(N), but in the worst case, the complexity may fall into O(N2) (or O(N.log2(N)), depending on hash collision handling). In addition, the hash table needs to be built / populated before it can be used.

[0094] Thus, occupancy maps reduce complexity, but this comes at the expense of increased memory usage. Typically, the map size is several megabytes of data to be a good compromise in coding efficiency.

[0095] Figure 4 It is shown that when encoding of the occupancy bits of the current child node (striped cube) is performed, the occupied volume (consisting of white cubes) at a given depth is known, in the case where the nodes are processed in Morton order and the occupancy bits are also encoded in Morton order.

[0096] From this figure, one can also understand the nature of the different sub-volumes that can be mapped within the occupancy map.

[0097] According to some methods, the subvolume occupancy bits are encoded / decoded based on the raster scan ordering of the nodes at a given depth in the occupancy tree. In some cases, the raster scan ordering of the subvolumes is used to encode / decode their respective occupancy bits. In some cases, the raster scan ordering of the nodes is used to encode / decode (together) the occupancy bits of all their corresponding subvolumes for better coding efficiency.

[0098] An example of the method described above is Figure 5 (Example 1). For better understanding, the entire occupied volume is represented in the figure to better see the raster scan order of the processing. However, it should be clear that if the ancestor node has been marked as unoccupied, part of the volume will not be encoded / processed.

[0099] like Figure 5As shown, a lexicographic ordering in (x, y, z) is used (i.e., a cooperative lexicographic ordering in (z, y, x)). For a given depth in the octree, the occupancy of the child nodes is encoded according to the raster scan ordering of the child nodes. In order to achieve the raster scan ordering of the child nodes, 4 tubes (tube 0, tube 1, tube 2, tube 3) are defined for each node at the current depth, each tube being made of each node with the same x and y. The encoding of the occupancy bits of the child nodes of tube 0 is performed by encoding the occupancy bits b0 and b1 of all occupied nodes with the same fixed x and y, and the nodes are sorted by increasing the z of all occupied nodes. The encoding of the occupancy of tube 1 is then performed by encoding the occupancy bits b2 and b3 of the same node and in the same order as tube 0.

[0100] The encoding process for encoding occupied pipe 0 for all occupied nodes with the same fixed x and y and occupied pipe 1 for all occupied nodes with the same fixed x and y is repeated for each y in increasing y order. This provides a raster scan encoding of slice 0 consisting of each bit b0, b1, b2, b3 of all occupied nodes with the same x. An illustration of the raster scan ordering of slice 0 for these nodes is shown in Figure 6 (a) is shown, where each child node is represented in 2D for better illustration. After encoding slice 0 of the nodes at a fixed x, slice 1 of these nodes is then encoded at the same x, and the encoding process of slice 0 is reproduced for slice 1 by continuously encoding in the same order, the occupied pipe 2 then 3 consists of the occupied bits b4, b5 then b6, b7 of the same node. And the raster scan ordering of slice 1 of these nodes is shown in the encoding process of the diagram in Figure 6 (b) where each child node is represented in 2D for better illustration. The decoding process of slice 0 followed by slice 1 is repeated in increasing order for every x nodes at the current depth of all occupied nodes.

[0101] Figure 7 An example of a decoder (corresponding to the decoder of Example 1) using the above method is shown in the form of a block diagram, where List L d Indicates that the node N with a depth of d in the octree is included k (x k ,y k ,z k ), with nodes sorted from N0 to N in raster scan order last Sort, and k is the list L d The sorted indices of the nodes in . Figure 7 Both of the following are shown:

[0102] The process of decoding the occupancy of the child node (sIdx, tIdx, i) at depth d+1 from a bitstream following raster scan ordering;

[0103] And put the child nodes into list L d+1 The process of raster scanning ordered nodes to generate the next depth.

[0104] The variable sIdx represents the index of the slice of all occupied nodes with the same x; the variable tIdx represents the index of the pipe of all occupied nodes with the same x and y; and the variable i represents the index of the child node within each pipe.

[0105] In other words, the integer value obtained from the binary word formed by (sIdx, tIdx, i) is the child index of the node (decimal value from 0 to 7). The variable kSStart is used to store the sorted index of the first node of the slice of all occupied nodes with the same x; and the variable kTStart is used to store the sorted index of the first node of the pipe of all occupied nodes with the same x and y. The depth dMax is the maximum depth in the tree and can be determined from the bitstream.

[0106] The detailed process in the decoder can be described as follows:

[0107] To initialize the algorithm, by setting the list L0 to contain only the root node of the occupancy tree and the initial depth d to 0;

[0108] Tag 0:

[0109] Get a first in first out (fifo) list L storing nodes at depth d in raster scan order d ;

[0110] Get an empty list L d+1 to generate a raster scan ordered child node for decoding the next depth; Set the node index k to 0 (initialize the variable k with the value 0);

[0111] Label 1:

[0112] Set the slice index sIdx to 0 (initialize the variable sIdx with the value 0).

[0113] Set the first node index in the slice to k (initialize the variable kSStart with the value k);

[0114] Label 2:

[0115] Set the pipe index tIdx to 0 (initialize the variable tIdx with the value 0);

[0116] · Set the first node index in the pipeline to k (initialize the variable kTStart with the value of k); · Label 3:

[0117] From list L d Get node N k (x k ,y k ,z k );

[0118] Set the sub-index in the pipeline to 0 (initialize the variable i with the value 0).

[0119] Process 0: Decode and generate L d Middle Node N k the ordered children of the children of the pipe tIdx of the slice sIdx;

[0120] The occupied bits of the i-th child of the pipeline tIdx that decodes the slice sIdx from the bitstream (which is N k Child nodes of (sIdx, tIdx, i));

[0121] If the decoded bit indicates that the child node is occupied, then in list L d+1 At the end of the pipeline tIdx, a new node (which is N) of the i-th child node of the slice sIdx is appended k The new node has coordinates (2×x k +sIdx,2×y k +tIdx,2×z k +i);

[0122] If i is not equal to 1, increase i and repeat process 0, otherwise go to the next step; k Is it the last node of depth d (i.e., the list L d The last node of

[0123] If N k If it is not the last node of depth d, then remove it from list L d Get the next occupied node N k+1 (x k+1 ,y k+1 ,z k+1 ). Then determine whether it is x k ≠x k+1 or

[0124] y k ≠y k+1 ;

[0125] If x k ≠x k+1 ork ≠y k+1 , it means that the next node does not belong to the same pipeline as the occupied node with the same x and the same y. (Then determine whether tIdx == 1;

[0126] If tIdx == 1, it means that both pipelines (in the pipeline of all occupied nodes with the same x and y) of the child node of the current slice of the child node are handled.

[0127] The processing has been completed. Then determine whether x k ≠x k+1 ;

[0128] If x k ≠x k+1 , it means that the next node does not belong to the same slice of the occupied node with the same x. Then determine whether sIdx == 1;

[0129] If sIdx == 1, it means that the processing of both slices of the child node (in the slices of all occupied nodes with the same x) has been completed.

[0130] Therefore, N k+1 is the next slice of the occupied node and the starting point of the next pipeline of occupied nodes to be processed;

[0131] Increase k by 1 and loop to Label1 to start processing the first child node pipeline of the first child node slice of the next occupied node slice;

[0132] Otherwise sIdx == 0. This means we can start processing the second slice of the occupied node;

[0133] Label 4:

[0134] Increase the slice index sIdx by 1 and reset k to the index of the first node of the slice for all occupied nodes with the same x:

[0135] k = kSStart;

[0136] Loop to label 2 to start the third pipeline processing the child nodes;

[0137] Otherwise x k = = x k+1 This means that the next node belongs to the same slice of the occupied node, but belongs to the next pipeline of the occupied node. Therefore, N k+1 is the starting point of the next pipeline to be processed;

[0138] ·Increment k by 1 and loop to Label2 to start processing the first pipe of the next pipe of the occupied node's children (belonging to the current slice sIdx of the children);

[0139] ·Otherwise tIdx == 0. This means we can start processing the second pipe of the children of the current slice of the children;

[0140] ·Label 5:

[0141] ·Increment the pipe index tIdx by 1 and reset k to the index of the first node of the pipes of all occupied nodes with the same x and y: k = kTStart;

[0142] ·Loop to Label 3 to start processing the second pipe of the children;

[0143] ·Otherwise x k == x k+1 and y k == y k+1 . This means the next node belongs to the same pipe of the occupied nodes. Increment k by 1 and loop to Label 3 to continue processing the same pipe of the children;

[0144] ·Otherwise N k == N last , this node is the last node at depth d. Then check if tIdx == 1;

[0145] ·If tIdx == 1, it means the processing of the last pipe of the children of the current slice of the children has been completed. And the next slice of the children (if any) can be processed. Then check if tIdx == 1;

[0146] ·If sIdx == 1, it means the processing of the last slice of the children of the last slice of the occupied nodes has been completed. And the next depth in the occupancy tree (if any) can be processed. Then check if d == dMax;

[0147] ·If d == dMax, it means the last depth of the occupancy tree has been decoded. Then complete the occupancy tree;

[0148] ·Otherwise, d < dMax. This means the next depth of the occupancy tree can be processed. Increment the depth d by 1 and loop to Label 0 to process the next depth in the occupancy tree;

[0149] ·Otherwise sIdx == 0. This means the processing of the second slice of the children of the last slice of the occupied nodes can be started. Go to Label 4 to start;

[0150] Otherwise tIdx == 0. This means that the processing of the last pipeline of the child node of the current slice of the child node can start. Go to label 5 to start.

[0151] List L can be omitted d+1 , and do not append the child node to the list when d == dMax, because the list will not be used because there is no more depth to process.

[0152] The coordinates of a child node are the coordinates of its parent node refined by one bit of precision. In other words, the coordinates of the node center can be obtained in some physical space by multiplying the coordinates of the node center by the physical dimensions of the node volume scaled according to the depth:

[0153] ((x k ,y k , z k )+(0.5, 0.5, 0.5))×(width, length, height) / 2 深度 Preferably, when d == dMax, if the child node is occupied, the output coordinates are ((2×x k +sIdx,2×y k +tIdx,2×z k +i)+(0.5, 0.5, 0.5))×(width, length, height) / 2 dMax Generates raster scan-ordered point geometry.

[0154] According to the above encoding process of the child nodes, it is observed that for each node N k Needs to be accessed / processed 4 times: the first time to encode occupied pipe 0, the second time to encode occupied pipe 1, the third time to encode occupied pipe 2, and the last time to encode occupied pipe 3.

[0155] This encoding process builds the nodes required for the raster scan ordering of the child nodes occupied The mentioned ordering is obtained from the raster scan ordering of the nodes being processed. And the raster scan ordering of the nodes is directly obtained from the raster scan ordering of the child node occupancy codec performed on the parent node (i.e., during the previous depth of the octree occupancy codec).

[0156] Figure 8 The encoder in Example 1 is shown in the form of a block diagram.

[0157] The effect of the method provided is Fig. 9 , which shows the occupancy map coverage of the known sub-volumes when encoding the occupancy bits of the striped sub-volumes when the occupancy bits are encoded according to the raster scan ordering of the sub-volumes of the nodes. Fig. 9Also shown are neighboring positions that can be used to construct a context for entropy coding of occupied bits. With raster scan ordering (regardless of the ordering of the coordinate axes in the lexicographic ordering), the occupancy of any subvolume with a lower z position, lower y position, lower x position can be used for context analysis. Thus, a predetermined pattern of neighboring subvolumes can be defined and used to select a coding context for any coding occupied bit.

[0158] According to other examples, the method uses a raster scan ordering of nodes to (together) encode / decode the occupancy bits of all its corresponding subvolumes at a given depth in the occupancy tree. Fig.10 As shown, occupied nodes are processed in raster scan ordering, but the occupancy bits of all subvolumes of the node are encoded consecutively at the node by ordering in a single channel, for example using Morton scan ordering of the subnode occupancy bits (which means that each subvolume of the node is encoded using Morton ordering in a single channel at the node); and in order to order the subnodes by raster scan ordering for the next depth, the same subnode ordering method as in the first example is used, which uses 4 channels on each node at the current depth and Figure 7 Shown in.

[0159] Fig.11 An example of the sub-node occupation decoding process of the decoder in this embodiment is shown, and Fig.12 An example of the encoder of this embodiment is shown, and the sorting process of the child nodes is not in Fig.11 and Fig.12 Shown in.

[0160] The detailed process in the encoder / decoder can be described as follows:

[0161] Obtain the fifo (first-in-first-out) list L of nodes at storage depth d by raster scan sorting d ;

[0162] · In the list L from the fifo d Get node N at depth d k (x k， y k , z k) , and initially at each depth from the first node N0(x 0， y 0， z0) start;

[0163] Encode the bits occupied by the child nodes into a bit stream according to a certain order (such as Morton order), or decode them from the bit stream;

[0164] If N k Not a fifo list Ld The last node in the fifo list L d The next node N in k+1 (x k+1 ,y k+1 , z k+1 ) to encode / decode until L is reached d The last node in ;

[0165] Encode the occupancy of the child nodes at the next depth until the final depth is reached.

[0166] Fig.13 Shows Figure 5 The described alternative coding ordering encodes the occupancy bits of the striped subvolumes with a known subnode occupancy map. Using the raster scan ordering of the nodes (regardless of the axis ordering used in the lexicographic ordering), the occupancy of any subvolume with a node having a lower z, lower y, or lower x position than the current node (the parent of the striped subvolume) can be used to contextualize the entropy encoding of the striped subvolume occupancy bits as well as any subvolume occupancy bits previously encoded for the current node. Using this approach, we can clearly distinguish that there is a different pattern of available neighboring nodes for each position of the coded subvolume occupancy. Therefore, in the case of the octree occupancy, there are 8 different patterns. Therefore, the entropy encoding of the occupancy bits needs to be tuned according to each pattern (i.e., for each occupancy bit index) in order to be more optimal.

[0167] This complicates the bits of coding relative to the first example, but is still better than the pattern in the octree with nodes coded in Morton ordering in the prior art. In the prior art, for Morton ordering of nodes, the pattern occupied by "unknown" (i.e., not yet coded) neighboring nodes varies more with the position in the ancestor node (and therefore with the Morton index), so some neighboring information is not always reliable when constructing the entropy context, and it may introduce bias in the entropy coding probability constructed in the adaptive entropy coding context, thereby reducing coding performance.

[0168] It should be understood that in point cloud encoding, occupied nodes and subnodes will rarely (and preferably never) differ from example Figure 4 , Figure 5 , Fig. 9 and Fig.10These graphs are densely represented to provide a better view of the available causal sub-occupancy information and the node / sub-node processing ordering. In a usual point cloud, several volumes will be unoccupied, therefore, the nodes of these volumes will not exist and will be processed. The processing ordering of occupied nodes is the same as in the dense content case, and for a given occupied node, the occupancy of all sub-volumes is always encoded, but unoccupied nodes / volumes are not processed at all since they do not exist in the tree structure. Compared to the processing ordering of dense content, the processing of these unoccupied nodes / volumes is skipped.

[0169] In the prior art, in order to construct a neighboring pattern for entropy coding of a node, an occupancy map is used to store the occupancy information of many nodes of a cube, for example, a cube of 512*512*512 nodes is used in the current G-PCC method, which results in a large amount of memory usage. In detail, in the current G-PCC method, the octree geometry coding method uses the Morton sorting of nodes to encode and decode the information of the occupancy tree. When encoding the occupancy bits, an entropy coding context is usually constructed based on the occupancy of adjacent nodes and / or adjacent child nodes. However, using Morton sorting, obtaining the occupancy of adjacent nodes or child nodes requires a large amount of memory usage (e.g., a hash table for an occupancy map or occupied nodes), or a search algorithm with a worst-case complexity of at most O(N.log2(N)).

[0170] Therefore, the problem to be solved in this application is how to retrieve the occupancy of adjacent nodes and / or adjacent child nodes while encoding or decoding the occupancy tree without sacrificing the memory of the occupancy graph and increasing the overall complexity of the O(N.log2(N)) search algorithm.

[0171] The proposed method processes nodes based on raster scan sorted nodes and only stores information of neighboring nodes adjacent to the node to be encoded / decoded to build neighboring patterns for entropy coding. Using this sorting, a buffer of raster scan sorted nodes can be used to process nodes in sorted order and search for neighboring nodes (which can also be associated with already decoded sub-occupied nodes if necessary).

[0172] In this way, the increase in memory is negligible, since the buffer already exists for the usual processing of nodes, and the proposed method utilizes the buffer to generate neighboring patterns for entropy coding. In addition, the complexity of neighbor search can be expressed as O(N).

[0173] As mentioned above, for the Morton ordering of nodes, when using the occupancy map to obtain the neighborhood occupancy (for contextualization of entropy coding), it is inevitable to sacrifice the coding performance or increase the search complexity.

[0174] With raster scan ordering of nodes, if one wants to use an occupancy map in order to retrieve neighborhood occupancy, it is not as easy to limit the map size to a subvolume as with the Morton ordering, since positioning with the codec ordering regularly traverses the entire point cloud volume on the two first axes while increasing the third axis. Therefore, with raster scan ordering, in order to efficiently provide occupied neighborhoods, the map will have to represent the number of previously encoded slices of a node (a slice of a node represents all nodes with a given position on the third axis), which limits the distance of the neighborhood along the third axis. Therefore, if the point cloud is 2 D x2 D x2 D , then the size of the buffer used for the child nodes will have to be 2 D+1 x2 D+1 Depending on the size of the point cloud, the memory required for the buffer may be much larger than the memory normally used by the Morton sorting node occupancy graph.

[0175] Fortunately, using an occupancy map can avoid wasting a lot of memory in the case of raster scan ordering of nodes. Instead of keeping a buffer of occupancy bits for a dense volume (the map), you can keep a few pointers / iterators / indexes on the node list, each of which represents the position of a neighboring node relative to the current node.

[0176] With the original approach, for each node whose sub-occupancy is being encoded, for each neighboring position to be used in the encoding scheme (e.g., to select an entropy context and / or make an occupancy prediction), the corresponding neighboring node must be found. Given that raster scan ordering is O(log(N)), where N is the number of nodes at the current tree depth, this search can, for example, be a binary search of the nodes of the raster scan ordering. If the node is found, the volume is occupied and its sub-occupancy can be retrieved, which may have been associated with it when it was encoded / decoded. If the node is not found, the volume is not occupied, and therefore the sub-volumes of its child nodes are also not occupied.

[0177] This naive approach is not optimal at all. To encode the sub-occupancy of N nodes at a given depth, the complexity would be O(N*log(N)). And for the number of neighboring nodes M on a node (prediction / context) to look for and use in the encoding process, the complexity can even be O(M*N*log(N)).

[0178] To exploit the advantages of raster scanning to sort nodes and also to reduce memory consumption, the proposed method retrieves neighboring node occupancy by storing only the occupancy information of neighboring nodes adjacent to the current node to construct neighboring patterns for entropy coding.

[0179] In some embodiments, the decoder Fig.14 , which is shown as a block diagram, is based on the previously described raster scan ordering encoding method, where the encoding of the occupancy of the child nodes / subvolumes of all nodes at a given depth is done at once on all nodes, following their raster scan ordering. Due to the raster scan ordering, nodes associated with a given neighboring position (the same relative position of each node) can also be processed at once on the node. For example, when increasing the index in the raster scan ordering of the processed node, the index in the raster scan ordering of the neighboring node at the position (-1, -1, -1) relative to the visited node will be increased. Therefore, by keeping the index of the neighboring node at the position (-1, -1, -1) of the previously processed node, the index of the processed node can be increased based on the index maintained by the previously processed node until:

[0180] 1) find a neighboring node at the position (-1, -1, -1) of the currently processed node, where the neighboring node is occupied and therefore is in the raster scan sorted list of processed nodes, or

[0181] 2) Find a node at a higher position in the raster scan order than the relative position (-1, -1, -1) of the current processing node, where the adjacent volume at the relative position (-1, -1, -1) of the current processing node is not occupied.

[0182] As a reminder, consider the raster scan ordering that corresponds to the lexicographic ordering of (x, y, z), in which position (x1, y1, z1) is greater than or equal to position (x0, y0, z0) if: x1>x0; or x1=x0 and y1>y0; or x1=x0, y1=y0 and z1>z0.

[0183] like Fig.14 As shown, the proposed decoding process follows the following steps:

[0184] Determine a neighbor offsets table (NOT) that defines relative coordinates between the position of each neighbor node and the position of the current node, wherein each neighbor node shares a common face, common vertex, or common edge with the current node;

[0185] Iterate over the nodes at each depth to decode the occupancy tree;

[0186] Get depth d, and list L d Store nodes at depth d in raster scan order;

[0187] Iterate over the list L d Each node in

[0188] Determine the adjacent occupation situation;

[0189] For each node N k , using the definition of NOT and its position P k To determine the location of the 26 neighboring nodes NeighP

[0190] NeighP[i]=Pk+NOT[i],

[0191] And 26 pointers are used to track the adjacent nodes at the corresponding positions by keeping the index of the adjacent nodes at the corresponding positions of the previously processed nodes for processing the next node, which can be obtained from the list L d Search the adjacent nodes obtained by NOT[i] in the held index;

[0192] Pointers are defined and used to iterate over the list L d The nodes in the to determine the occupancy information of the 26 adjacent nodes (whether they are occupied);

[0193] The obtained neighboring node occupancies are then used to construct a neighboring pattern for each node, which can be used for context selection in entropy coding;

[0194] Decoding the sub-occupancy bits of the node from the bit stream by entropy coding method.

[0195] In some embodiments, fewer than 26 neighbor nodes may be used, for example to reduce complexity.

[0196] In some embodiments, node N k The adjacent nodes can be N k Further (i.e., the neighboring nodes may not be k shared faces, edges or vertices), for example to improve encoding performance by using more complex adjacency patterns.

[0197] In some embodiments, more than 26 neighbor nodes may be used, for example, to improve encoding performance by using more complex neighbor patterns.

[0198] And the corresponding encoder of this embodiment is Fig.15 Shown in.

[0199] In some embodiments, there are 26 neighboring nodes to be retrieved to obtain the neighboring pattern of the current node being encoded, such as Fig.16(a). In order to determine the relative position of each adjacent node relative to the current node, a table indicating relative positions NOT[i]i=0,1,…,26 is defined, wherein each element in the table represents a position offset between an adjacent node and the current node. There are a total of 27 elements in the table, including one element for the current node and 26 elements for adjacent nodes. And the index i of each element NOT[i] indicates the order of each adjacent node among the 26 adjacent nodes of the current node sorted according to a raster scan. Each element NOT[i] has three dimensions, the first dimension represents the relative position difference along the x-coordinate, the second dimension represents the relative position difference along the y-coordinate, and the third dimension represents the relative position difference along the z-coordinate. Therefore, the adjacent offset table NOT[i]i=0,1,…,26 can be defined as:

[0200] {{-1,-1,-1},{-1,-1,+0},{-1,-1,+1},{-1,+0,-1},{-1,+0,+0},{-1,+0,+1},{-1,+1,-1},{-1,+1,+0},{-1,+1,+1},{+0,-1,-1},{+0,-1,+0},{+0,-1,+1},{+0,-1,+0},{+0,-1,+1},{+0,+0,-1},{+0,+ 0, +0}, {+0, +0, +1}, {+0, +1, -1}, {+0, +1, +0}, {+0, +1, +1}, {+1, -1, -1}, {+1, -1, +0}, {+1, -1, +1}, {+1, +0, -1}, {+1, +0, +0}, {+1, +0, +1}, {+1, +1, -1}, {+1, +1, +0}, {+1, +1, +1}}.

[0201] A neighbor offset table NOT can be used for each node at each depth to retrieve its neighboring nodes and then obtain its neighboring pattern, which can be used for entropy coding.

[0202] In some embodiments, the table NOT[i]i=0,1,...,25 has 26 elements, including 26 elements for neighboring nodes and no element for the current node, so the neighboring offset table NOT[i]i=0,1,...,25 can be defined as

[0203] {{-1,-1,-1},{-1,-1,+0},{-1,-1,+1},{-1,+0,-1},{-1,+0,+0},{-1,+0,+1},{-1,+1,-1},{-1,+1,+0},{-1,+1,+1},{+0,-1,-1},{+0,-1,+0},{+0,-1,+1},{+0,-1,+0},{+0,-1,+1},{+0,+0,-1}, {+0, +0, +1}, {+0, +1, -1}, {+0, +1, +0}, {+0, +1, +1}, {+1, -1, -1}, {+1, -1, +0}, {+1, -1, +1}, {+1, +0, -1}, {+1, +0, +0}, {+1, +0, +1}, {+1, +1, -1}, {+1, +1, +0}, {+1, +1, +1}}.

[0204] For each node, using the determined offset table NOT, the 26 neighboring nodes at the same depth as the current node can be retrieved to determine their occupancy. Fig.16 As shown in (a), the node in the center is the node whose child occupied bits are encoded, and since the nodes are encoded in raster scan order, Fig.16 (b) If the node is occupied, then the node has been encoded / decoded (if the node exists in the list L d Therefore, Fig.16 The sub-occupancy of the node / volume in (b) is known: if the node exists, it is defined by the encoded / decoded sub-occupancy bits, and if the node does not exist, there are no occupied sub-nodes. Fig.16 In (c), the nodes have not been encoded / decoded yet, so the corresponding sub-occupancy bits have not been encoded yet, so these nodes are not known yet. But if the node exists, it means that at least one sub-node will be occupied, and if the node does not exist, it means that there are no occupied sub-nodes. This is because the node occupancy bit (the flag bit indicating whether the node is occupied) is known for each node / volume at the current depth, because it is obtained from the sub-occupancy bit encoding / decoding of the node at the previous depth. Therefore, according to the list L d The presence or absence of the corresponding node in Fig.16 Node occupancy of neighboring nodes shown in (c) (1 bit).

[0205] Therefore, the adjacent node occupancy includes two parts:

[0206] 1. A part includes the child occupancy of the nodes that have been encoded, and their positions are retrieved by using the elements in the table NOT from i = 0 to i = 12. In some embodiments, the neighbor node occupancy is implemented as a table childOccupancy[i], i = 0, 1, ..., 12, where each element in the table childOccupancy consists of 8 bits for the child occupancy if the corresponding neighbor node exists and is therefore occupied, and is set to 0 in the case of initialization or the node does not exist.

[0207] 2. The other part consists of the node occupancy of 13 neighboring nodes that have not yet been encoded, and their positions are retrieved from i=14 to i=26 using the elements in the table NOT (i.e., i=13 may indicate the current node). In some embodiments, the neighbor node occupancy is implemented as a table depthOccupancy[i], i=14,1,…,26, where each element in the table depthOccupancy consists of a flag bit (1 bit) indicating whether the corresponding neighboring node is occupied (i.e., whether the node exists), which is set to 0 for initialization.

[0208] In some embodiments, Fig.17a and 17b , Fig.18 and Fig.19 The detailed process of determining the occupancy of neighboring nodes of a node at each depth is shown in FIG. Fig.17a and Fig.17b A flowchart of the process is shown, and Fig.18 shows the detailed process of determining the childOccupancy of the 13 neighboring nodes of a node, and Fig.19 The detailed process of determining the depthOccpp of 13 neighboring nodes of a node is shown.

[0209] like Fig.17a and 17b As shown, the neighboring node occupancy of a provided node is determined by following the steps below:

[0210] Get the depth d and the list L of nodes stored in raster scan order d ;

[0211] For each depth, define a pointer table occCtxNP[i], i=0,1,…,26, where each element in the table is a pointer to the neighboring node of the currently processed node, which points to the list L d The address of the node in;

[0212] This table is used to track nodes adjacent to the processed node and is updated after searching the adjacent nodes for the currently processed node and represents the node N by updating k-1 The tracking table of the adjacent nodes is used to obtain the node N k At the beginning of depth d, the tracking table is initialized to the list L d In one variant, each element of the table occCtxNP[i], i=0,1,…,26 is in the index of the neighboring node of the currently processed node, which corresponds to the list L. d The index of the node in ;

[0213] · Sort by raster scan from list L d Get the kth node N k (for each depth k, starting at 0), and at depth d for node N k Encoding;

[0214] ·N nodes with encoding depth d k ,

[0215] Define a pointer nextNodeP that is used to iterate the list L for each adjacent node position d For each node in the list L d Whether there is a node belonging to this position in the

[0216] ·follow Fig.18 The steps shown in FIG. 1 determine the node N by using the pointer nextNodeP, the pointer table occCtxNP and the determined adjacent offset table NOT. k The adjacent nodes of occupies the first part of childOccupancy, and the detailed process is described in the following part (a);

[0217] ·follow Fig.19 The steps shown in FIG. 1 determine the node N by using the pointer nextNodeP, the pointer table occCtxNP and the determined adjacent offset table NOT. k The neighboring nodes of occupies the second part of depthOccupancy, and the detailed process is described in the following part (b);

[0218] After obtaining the occupancy of the 26 neighboring nodes, use them to build node N k The adjacent pattern for node N k Context selection in entropy coding of .

[0219] (a) The following shows the determination of node N kThe detailed process of childOccupancy of 13 adjacent nodes:

[0220] NOT iterate based on the defined adjacent offset table Fig.16 (b) where i increases from 0 to 12. Specifically,

[0221] · i starts at 0, and the pointer nextNodeP is defined as

[0222] nextNodeP=occCtxNP[i]+1,

[0223] It points to the list L d The address of the next node of occCtxNP[i] in , and occCtxNP[i] keeps the index of the adjacent node at the corresponding position of the previously processed node, so nextNodeP can search the index of the adjacent node for the currently processed node based on the previous adjacent information of the previously processed node at the current depth;

[0224] If the address of nextNodeP is not lower than list L d If the current node in the nextNodeP is not encoded, it means that the node pointed to by the pointer nextNodeP has not been encoded yet, so it cannot provide child occupancy as the neighbor occupancy of the current node, then the iteration is interrupted and go to the step of determining depthOccupancy; otherwise, the neighbor position NeighP[i] is determined based on the position of the current node and NOT[i] by the following equation:

[0225] NeighP[i]=CurP+NOT[i],

[0226] Where CurP is the position of the current node;

[0227] The expected location of the neighboring node NeighP[i];

[0228] In a loop, iteratively increase the pointers occCtxNP[i] and nextNodeP by 1 until the address of the node pointed to by nextNodeP is greater than the current node or the position of the node pointed to by nextNodeP is not lower than the expected neighboring node position NeighP[i];

[0229] If the address of the node pointed to by nextNodeP is lower than the address of the list L d The current node in the list L, and the position of the node pointed to by nextNodeP is also equal to the expected neighboring node position NeighP[i], (which means that the list L dThe node pointed to by nextNodeP belongs to the adjacent node position NeighP[i] of the current node, and the corresponding adjacent node of the current node is occupied).

[0230] Then, the i-th element of the neighbor occupancy childOccupancy[i] is determined to be the 8-bit child occupancy of the node pointed to by nextNodeP;

[0231] The childOccupancy of two adjacent nodes in the same pipeline is then determined to be Neigh[i], which is above NeighP[i]. In a preferred embodiment, the third coordinate of NeighP[i] is increased, and the pointer nextNodeP is also increased to determine the occupancy of the two adjacent nodes above the adjacent node position NeighP[i].

[0232] This will be described in detail later.

[0233] (b) The following describes how to determine the node N k The detailed process of depthOccupancy of 13 neighboring nodes.

[0234] First, to determine the occupancy of the nodes above the current node, that is, depthOccupancy[0], increase the pointer nextNodeP by 1 and determine whether the node pointed to by nextNodeP is not in list L. d The last node in ;

[0235] If the node pointed to by nextNodeP is not in list L d If the position of the node pointed to by nextNodeP is the same as the position of the adjacent node above the current node (which means that the adjacent node above the current node is occupied and the node is stored in list L), the third coordinate of the current node is increased by 1, and if the position of the node pointed to by nextNodeP is the same as the position of the adjacent node above the current node (which means that the adjacent node above the current node is occupied and the node is stored in list L), the next node is pointed to by nextNodeP. d In the example above, if the adjacent node occupancy is greater than 0, depthOccupancy[0] is set to true;

[0236] Secondly, NOT iterate based on the defined adjacent offset table Fig.16 Each adjacent node position in (c), where i increases from 15 to 26, to determine the flag bits of other adjacent nodes of depthOccupancy;

[0237] i starts at 15, and the pointer nextNodeP is defined as:

[0238] nextNodeP=occCtxNP[i]+1,

[0239] It points to the list Ld The address of the next node of occCtxNP[i] in;

[0240] If the address of nextNodeP is not in list L d The last node in the table, the neighboring position NeighP[i] is determined based on the current node and the position of the table NOT[i] by the following equation:

[0241] NeighP[i]=CurP+NOT[i];

[0242] A location with expected neighboring node locations NeighP[i];

[0243] In a loop, iteratively increase the pointers occCtxNP[i] and nextNodeP by 1 until the address of the node pointed to by nextNodeP is not less than the address of the list L. d The position of the last node in or the node pointed to by nextNodeP is not lower than the expected adjacent node position NeighP[i];

[0244] If the address of the node pointed to by nextNodeP is lower than the address of the list L d The last node in the list L, and the position of the node pointed to by nextNodeP is also equal to the expected neighboring node position NeighP[i] (which means that the list L d The node pointed to by nextNodeP belongs to the adjacent node position NeighP[i] of the current node, and the corresponding adjacent node of the current node is occupied).

[0245] ·Then the i-th element of neighboring occupancy depthccupancy[i-14] is determined to be true;

[0246] Then, the depthOccupancy of two adjacent nodes in the same pipeline is determined as Neigh[i]. In a preferred embodiment, the third coordinate of NeighP[i] is increased, and the pointer nextNodeP is also increased to determine the occupancy of the two adjacent nodes above the adjacent node position NeighP[i], which will be described in detail later.

[0247] (c) provided in List L d The method for searching adjacent nodes in is described as follows:

[0248] If the pointers occCtxNP[i] and nextNodeP are used to search each of the 26 neighboring nodes, then after processing all nodes at a given depth, the neighboring node search will iterate at most once on all nodes at the same depth. Therefore, the overall complexity for finding neighboring nodes can be expressed as O(N) for encoding the sub-occupancy of N nodes at a given depth. By searching M neighboring nodes, the complexity can be expressed as O(M*N).

[0249] To further reduce the complexity, it is proposed to group the neighbor node searches by adjacent vertical pipes: each consecutive position in the pipe will correspond to a node index that is incremented by zero or one in the raster scan sorted list of nodes (depending on the occupancy of the corresponding volume and therefore on the presence of the node in the list of processed nodes). Only the first neighbor node in the pipe is searched. As mentioned above, the pointers occCtxNP[i] and nextNodeP are only used (within the list Ld) to search the first neighbor node in the pipe of the cube of 26 neighbor nodes, so in Fig.18 and Fig.19 In each loop in , the index i is increased by 3, and the expected neighbor node position NeighP[i] in each loop is the bottom node in the pipeline, which will be described in detail later.

[0250] By using this simple neighbor search method, the complexity added by the neighbor search is practically negligible, and only limited additional memory is required to store the previous position of each desired neighbor relative position, or even just the previous position of the first neighbor node in each neighbor pipeline.

[0251] The childOccupancy of two adjacent nodes in the same pipeline is determined as NeighP[i].

[0252] After obtaining the occupancy of the bottom neighbor node of the pipeline located at NeighP[i], the next step is to determine the childOccupancy of the two neighbor nodes located above the node location NeighP[i], where the two neighbor nodes are in the same pipeline as NeighP[i], where the pipeline is the cube of the 26 neighbor nodes of the currently processed node. The detailed process is in Fig. 20 is shown in and can be described as:

[0253] First, the maximum index i_tubeEnd of the neighboring nodes in the same tube is determined as NeighP[i], where i_tubeEnd is defined as:

[0254]

[0255] Define index j and j = i + 1. Then iterate each neighboring position above the bottom node of the tube until j reaches i_tubeEnd.

[0256] At each iteration, first determine whether the node pointed to by pointer nextNodeP is not greater than the list L d The address of the current node (CurP) in NeighP[i] is obtained by increasing the third coordinate of NeighP[i] by 1 to view the adjacent nodes above NeighP[i].

[0257] If the position of the node pointed to by pointer nextNodeP is the same as the incremented position NeighP[i], then set the corresponding neighbor node occupancy childoccupancy[i] to the child occupancy bit of the node pointed to by pointer nextNodeP. And also let the pointer point to the list L d The next node in (increase the address of the node pointed to by pointer nextNodeP by 1).

[0258] The depthOccupancy of two adjacent nodes in the same pipeline is determined as NeighP[i].

[0259] After obtaining the occupancy of the bottom neighboring node of the pipeline located at NeighP[i], the next step is to determine the depthOccupancy of the two neighboring nodes located above the node location NeighP[i], which are in the same pipeline as NeighP[i], where the pipeline is the cube of the 26 neighboring nodes of the currently processed node. The detailed process is in Fig.21 is shown in and can be described as:

[0260] First, the maximum index i_tubeEnd of the neighboring nodes in the same tube is determined as NeighP[i], where i_tubeEnd is defined as:

[0261]

[0262] Define index j and j = i + 1. Then iterate over each adjacent position above the bottom node of the tube until j reaches i_tubeEnd;

[0263] At each iteration, first determine whether the node pointed to by pointer nextNodeP is in list L d The last node of NeighP[i] increases the third coordinate of NeighP[i] by 1 to view the adjacent nodes above NeighP[i].

[0264] If the position of the node pointed to by pointer nextNodeP is the same as the added position NeighP[i], set the corresponding neighbor node occupancy depthccupancy[j-14] to true. And make the pointer point to list L d The next node in (increase the address of the node pointed to by pointer nextNodeP by 1).

[0265] In the case of using a full raster scan ordering for encoding child node occupancy, as in the first example of the previously described method (Example 1), the child nodes are encoded in a raster scan ordering at each depth, so that the neighboring information used for entropy encoding each child node is based on the neighboring child nodes of the current child node to be encoded.

[0266] Since the children of all occupied nodes at depth d are encoded in raster scan order, the neighboring child nodes below the current child node in the raster scan order have been encoded and their 1-bit occupancy is known, while the neighboring child nodes above the current child node have not yet been encoded and their 1-bit occupancy is unknown. Therefore, only the neighboring child nodes below the current child node are retrieved for neighbor search, so there are 13 neighboring child nodes to retrieve, such as Fig. 22 As shown, each cube represents a child node, and the green cube represents the current child node to be encoded. And each adjacent child node has a relative position to the current child node, so the corresponding adjacent child node position offset table CNOT[i], i = 0, 1, ..., 12 is

[0267] {{-1,-1,-1},{-1,-1,+0},{-1,-1,+1},{-1,+0,-1},{-1,+0,+0},{-1,+0,+1},{-1,+1,-1},{-1,+1,+0},{-1,+1,+1},{+0,-1,-1},{+0,-1,+0},{+0,-1,+1},{+0,-1,+0},{+0,-1,+1},{+0,+0,-1}},

[0268] And the index i is based on the raster scan ordering of adjacent child nodes.

[0269] Methods for determining the occupancy of child adjacent nodes:

[0270] To obtain the child neighbor occupancy of the current node:

[0271] Define a 13-dimensional pointer table occCtxNodes for the 13 adjacent child nodes of the current child node, which points to the address of the node in the list Ld. Here, the list Ld stores the child nodes of all occupied nodes with a depth of d in raster scan order. And initialize the table occCtxNodes to the address of the first child node in the list Ld for each depth;

[0272] Define the pointer nextNodeP to point to the list L d The next node of an adjacent child node in , and it can be obtained by

[0273] nextNodeP=occCtxNodes[i]+1

[0274] · By in list L d Search each adjacent sub-position in the sub-nodes in to determine the occupancy consisting of the 1-bit occupancy of the 13 sub-adjacent nodes;

[0275] For the child neighbor node position NeighCP[i] of the bottom child node of the pipeline, it can be obtained by the following formula

[0276] NeighCP[i]=CurCNode+CNOT[i],

[0277] Use pointers nextNodeP and occCtxNodes[i] to iterate over the list L d The child nodes in the , to obtain the occupancy of the node located at NeighCP[i]. The detailed process is the same as the process in the preferred embodiment of the proposed method;

[0278] For the two adjacent child nodes above the node at NeighCP[i], since the adjacent child nodes are stored in raster scan order, only the third coordinate of NeighCP[i] is increased by 1 or 2, and nextNodeP is also increased by 1 or 2 to obtain its occupied bits. The detailed process is the same as that in the preferred embodiment of the proposed method.

[0279] In some embodiments (e.g., Figure 5-8 ), in order to encode / decode the 8-bit sub-occupancy of each node at a certain depth, the encoding of each node is achieved by processing the node 4 times, and a search for neighboring nodes is performed each time a node is being processed (i.e., for each pipeline of child nodes), so the complexity is roughly increased by up to 4 times.

[0280] To efficiently process each pipeline of nodes, instead of using an O(log(N)) search to find the first node of a pipeline (or slice) and its neighbors, each time a new pipeline (or slice) is started, they can be kept in memory the first time they are encountered. Two local copies of the node index and its neighbors are sufficient to keep the overall linear complexity.

[0281] Whenever a node is processed for the first time in the processing of slice 0, the first replica is reset with the location of the node and its neighbors. When processing of slice 1 begins, this replica is used to reinitialize the search for the node and its neighbors;

[0282] The second copy is reset with the location of the node and its neighbors each time a node is processed for the first time in processing of pipeline 0 or pipeline 2. This copy is used to reinitialize the search for the node and its neighbors when processing of pipeline 1 or pipeline 3 begins.

[0283] In some embodiments, for sub-raster scan encoding ordering, each time we iterate over the pipeline [1 or 3] / slice [1] of a child node, restore the pointer of the parent node at the beginning of the pipeline / slice (the adjacent pattern must be saved when starting the pipeline of the (parent) node [i.e., pipeline 0 and pipeline 2 of the child node], and when starting the slice of the (parent) node [i.e., slice 0]).

[0284] When iterating over the node list, the end of a node pipeline or the end of a node slice can be detected by comparing the next node position with the current node position: if at least one of the first coordinate (x) and the second coordinate (y) (according to the lexicographic ordering used) differs, this indicates the end of the pipeline; if at least the first coordinate differs, this indicates the end of a slice (x).

[0285] Reference now Fig.23 , which shows a schematic flow chart of a method for encoding a 3D point cloud into a bitstream according to the present application.

[0286] The method includes:

[0287] Step S10, along each axis X, Y, Z of the coordinate system, obtain a node with a depth of d that contains at least a portion of the point cloud by lexicographic sorting;

[0288] Step S11, determining each node N based on a portion of adjacent nodes k The adjacent mode of

[0289] Step S12: entropy encode the occupancy information of each node into a bit stream based on the neighboring pattern.

[0290] Reference now Fig.24 , which shows a schematic flow chart of a method for decoding a 3D point cloud from a bitstream according to the present application.

[0291] The method includes:

[0292] Step S20, along each axis X, Y, Z of the coordinate system, obtain a node with a depth of d that contains at least a portion of the point cloud in a lexicographical order;

[0293] Step S21, determining each node N based on a portion of adjacent nodes k The adjacent mode of

[0294] Step S22: entropy decoding the occupancy information of each node from the bit stream based on the neighboring pattern.

[0295] Reference now Fig.25 , which shows a simplified block diagram of an example embodiment of an encoder 1100. The encoder 1100 includes a processor 1102 and a storage device 1104. The storage device 1104 can store a computer program or application containing instructions that, when executed, cause the processor 1102 to perform several operations such as those described herein. For example, the instructions can encode and output a bit stream encoded according to the method described herein. It should be understood that the instructions can be stored on a non-transitory computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the described process. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system".

[0296] refer to Fig.26 , which shows a simplified block diagram of an example embodiment of a decoder 1200. The decoder 1200 includes a processor 1202 and a storage device 1204. The storage device 1204 may include a computer program or application containing instructions that, when executed, cause the processor 1202 to perform several operations such as those described herein. It should be understood that the instructions may be stored on a computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the described processes and methods. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system."

[0297] It should be understood that decoders and / or encoders according to the present application can be implemented in multiple computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer-readable memory, including CD, RAM, ROM, flash memory, etc.

[0298] It should be understood that the decoder and / or encoder described herein and the module, routine, process, thread or other software components implementing the method / process for configuring the encoder or decoder can be implemented using standard computer programming techniques and languages. The application is not limited to specific processors, computer languages, computer programming conventions, data structures, other such implementation details. Those skilled in the art will recognize that the described process can be implemented as a part of a computer executable code stored in a volatile or non-volatile memory, a part of a dedicated integrated chip (Application Specific Integrated Chip, ASIC), etc.

[0299] The present application also provides a computer readable signal encoding data produced by applying the encoding process according to the present application.

[0300] Certain adjustments and modifications may be made to the described embodiments. Therefore, the above embodiments are to be considered as illustrative rather than restrictive. In particular, the embodiments may be freely combined with each other.

Claims

1. A method for encoding a three-dimensional 3D point cloud into a bitstream, preferably, the method is applied to an encoder, the geometry of the point cloud is defined by an octree structure, the octree structure comprises a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is segmented into sub-volumes step by step and contains points of the point cloud, wherein the volume is segmented into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each sub-volume indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, the method comprising: Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order; Determine each node N based on a portion of its neighboring nodes k The adjacent mode of The occupancy information of each node is entropy encoded into a bit stream based on the neighbor pattern.

2. A method for decoding a three-dimensional 3D point cloud from a bitstream, preferably, the method is applied to a decoder, the geometry of the point cloud is defined by an octree structure, the octree structure comprises a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is segmented into sub-volumes step by step and contains points of the point cloud, wherein the volume is segmented into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each of the sub-volumes indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, the method comprising: Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order; Determine each node N based on a portion of its neighboring nodes k The adjacent mode of Occupancy information of each node is entropy decoded from the bitstream based on the neighbor pattern.

3. The method according to claim 1 or 2, wherein: The method further includes storing occupied child nodes with a depth of d+1 in lexicographic order.

4. The method according to any one of claims 1 to 3, wherein: The lexicographic ordering of nodes at depth d and / or the lexicographic ordering of child nodes at depth d+1 is a raster scan ordering.

5. The method according to any one of claims 1 to 4, wherein: Entropy encoding the occupancy information of the child nodes of each node into the bitstream, or entropy decoding the occupancy information of the child nodes of each node from the bitstream is performed based on a lexicographic ordering of nodes with a depth of d and a fixed ordering of the child nodes of a given node.

6. The method according to any one of claims 3 to 5, wherein: Storing the occupied child nodes at depth d+1 is performed by processing each of all nodes obtained at once, preferably by lexicographically sorting the child nodes using an intermediate storage, preferably raster-scanning sorting.

7. The method according to any one of claims 1 to 6, wherein: N k A portion of the adjacent nodes of the node N is stored in the tracking table, wherein the tracking table limits the number of adjacent nodes to be stored. Preferably, the tracking table is stored by k-1 The corresponding tracking table is updated to obtain.

8. The method according to claim 7, wherein: The tracking table is a first adjacent offset table NOT, wherein the first NOT stores adjacent nodes relative to the current node N. k relative position.

9. The method according to any one of claims 1 to 8, wherein: The obtained node with a depth of d is stored in a first link data structure, and the portion of adjacent nodes is obtained by iterating the nodes in the first link data structure, wherein the first link data structure includes elements associated with each other such that the elements are arranged in a sorted manner.

10. The method according to any one of claims 7 to 9, wherein: The tracking table is updated by grouping the portion of the neighboring nodes searched using vertical pipes of the neighboring nodes, wherein each vertical pipe includes nodes having the same coordinates on two axes of the coordinate system.

11. The method according to any one of claims 1 to 10, wherein: The adjacent pattern is a binary word indicating adjacent occupancy, and the binary word is obtained based on the first occupancy and the second occupancy of the adjacent node, the first occupancy of the adjacent node has a smaller coordinate value than the current node on at least one axis, the first occupancy of each adjacent node includes an 8-bit sub-occupancy, the second occupancy of the adjacent node has a larger coordinate value than the current node on at least one axis, and the second occupancy of each adjacent node includes a flag bit indicating the occupancy of the corresponding adjacent node.

12. The method according to any one of claims 1 to 6, wherein: The children of all occupied nodes are encoded or decoded in raster scan order of the children.

13. The method according to claim 12, wherein: The child nodes are stored in a second link data structure, and the current child node CN is obtained by iterating the child nodes in the second link data structure. k A portion of adjacent child nodes of a given adjacent child node position of a second linked data structure, wherein the second linked data structure has elements associated with each other such that the elements are arranged in a sorted manner.

14. The method according to claim 13, wherein: The part of adjacent child nodes is stored in a second adjacent offset table CNOT, wherein the second NOT stores relative positions of the adjacent child nodes with respect to the current child node.

15. According to the method of claim 13 or 14, the occupancy of the adjacent child node having a smaller coordinate value than the current child node in the second link data structure is a flag bit indicating the corresponding occupancy of the adjacent child node.

16. The method according to claim 15, wherein: The portion of neighboring child nodes represents the neighboring child nodes of the current child node that are tracked in the second link data structure and are updated for neighboring child node search of a next child node.

17. An encoder for encoding a three-dimensional 3D point cloud into a bit stream, the encoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to any one of claims 1, 3-16.

18. A decoder for decoding a three-dimensional 3D point cloud from a bitstream, the decoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to any one of claims 2-16.

19. A bit stream encoded by the method according to any one of claims 1, 3-16.

20. A computer-readable storage medium comprising instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1-16.

Images