Method, encoder and decoder for encoding and decoding 3d point clouds

Abstract

An encoding method and a decoding method, as well as an encoder and a decoder, for encoding a 3D point cloud into a bitstream. The geometry of the point cloud is defined in an octree structure having a plurality of nodes having a parent-child relationship and representing a three-dimensional position of an object, the point cloud being located within a volume space of a three-dimensional coordinate system, the three-dimensional coordinate system being segmented step by step into sub-volumes and containing points of the point cloud, wherein the volume is divided into a number of sub-volumes, each sub-volume being associated with a node of the octree-based structure, and wherein occupancy information associated with each respective sub-level sub-volume indicates whether the respective sub-level sub-volume contains at least one of the points, the method comprising: along each axis X, Y, Z of the coordinate system, determining whether the respective sub-level sub-volume contains at least one of the points; obtaining nodes which contain at least one part of the point cloud and have the depth of d in a dictionary type sorting manner; encoding the occupancy information entropy of each node into a bitstream up to a depth k, where kgt; = d; entropy-encoding, into the bitstream, trisound information for each leaf node having a depth of m, wherein the trisound information comprises information on the presence and location of a vertex on each side of a volume associated with the leaf node; and encoding, into the bitstream, trisound information for each leaf node having a depth of m, wherein the trisound information comprises information on the presence and location of a vertex on each side of the volume associated with the leaf node. The entropy coding of the occupancy information and the entropy coding of the trisound information are based on the same sequence, and the depth is mgt; k.

Description

Technical Field

[0001] The present application relates to a method for encoding a 3D point cloud into a bitstream. In addition, the object of the present application is to provide a method for decoding a 3D point cloud from a bitstream. In addition, the object of the present application is to provide an encoder and a decoder, a bitstream encoded according to the present application, and software. In particular, the object of the present application is to provide a method for reducing the delay between occupancy tree encoding and decoding and trisoup encoding and decoding. Background Art

[0002] Point cloud, as a format for representing 3D data, has attracted much attention due to its versatile ability to represent all types of 3D objects or scenes. As a result, point cloud can handle a variety of usage scenarios, including:

[0003] Film post-production,

[0004] Real-time 3D immersive telepresence or VR (virtual reality) / AR

[0005] (augmented reality) applications,

[0006] Free viewpoint video (e.g. for watching sports events),

[0007] Geographic Information Systems (also known as Cartography),

[0008] Cultural heritage (scanning and storing digitally rare objects),

[0009] Autonomous driving, including 3D mapping of the environment and real-time lidar data collection.

[0010] A point cloud is a set of points located in 3D space, each of which can have values ​​attached to it. These values ​​are often called point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0011] These attributes may be, for example, three-component color, material properties (such as reflectivity), and / or a two-component normal vector of the surface associated with the point.

[0012] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, Lidar, scanners, or they can be computer-generated (for example, in movie post-production). Depending on the use case, point clouds in mapping applications may contain thousands to billions of points.

[0013] In the raw representation of point clouds, each point requires a very high number of bits, at least a dozen bits per spatial component X, Y or Z, and optionally more bits for attributes, such as three times 10 bits for color. Practical deployment of point cloud-based applications requires compression techniques that allow point clouds to be stored and distributed within reasonable storage and transmission infrastructure conditions.

[0014] The compression may be lossy (similar to lossy video compression) during the distribution of point clouds to end users and visualization by the user terminal, such as on AR / VR glasses, or any other 3D-enabled device. Other use cases require lossless compression, such as medical applications or autonomous driving, to avoid compromising the decision-making results obtained by analyzing the compressed and transmitted point cloud.

[0015] Until recently, point cloud compression (also known as PCC) has not been addressed by the mass market and no standardized point cloud codec is available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11 (also known as the Moving Picture Experts Group or MPEG) has started a work project on point cloud compression. The project involves two standards, namely:

[0016] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC); and

[0017] MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-based Point Cloud Compression (G-PCC).

[0018] The V-PCC and G-PCC standards have completed their first versions at the end of 2020 and will soon be available in the market.

[0019] The V-PCC coding method compresses point clouds by performing multiple projections of 3D objects to obtain 2D patches that are packed into images (or videos, for mobile point clouds). The obtained images or videos are then compressed using existing image / video codecs, making it possible to use already deployed image and video solutions. By its nature, V-PCC is only effective on dense and continuous point clouds, as image / video codecs cannot compress non-smooth patches obtained from the projection of sparse geometric data collected, such as Lidar.

[0020] The G-PCC coding method has two schemes for compression geometry.

[0021] The first scheme is based on an occupancy tree (octree / quadtree / binarytree) representation of the point cloud geometry. Occupied nodes are forked until a certain size is reached, at which point the occupied leaf nodes provide the positions of the points, usually the centers of these nodes. A high degree of compression can be obtained for dense point clouds by using neighborhood-based prediction techniques. Sparse point clouds are also addressed by directly encoding the positions of points within nodes with non-minimum sizes, by stopping the construction of the tree when there are only isolated points in the node; this technique is called Direct Coding Mode (DCM).

[0022] The second scheme is based on a prediction tree, where each node represents the 3D position of a point and the relationship between nodes is the spatial prediction from parent to child nodes. This method can only solve sparse point clouds and has the advantages of lower latency and simpler decoding than occupancy trees. However, the compression performance is only slightly better than the first occupancy-based scheme and has complex encoding, intensively searching for the best predictor (in a long list of potential predictors) when building the prediction tree.

[0023] In both schemes, attribute encoding (de)coding is performed after geometry encoding (de)coding is completed, resulting in two-pass coding. Therefore, low latency is achieved by using slices that decompose the 3D space into independently coded sub-volumes without prediction between sub-volumes. When many slices are used, compression performance may be severely affected.

[0024] An important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves with respect to time. In addition, AR / VR point clouds are usually locally 2D, as they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly correlated (or dense), in the sense that points in them are rarely isolated, but have many neighbors.

[0025] A dense (or solid) point cloud represents a continuous surface with a resolution such that the volumes associated with the points (small cubes called voxels) touch each other without presenting any visual holes in the surface. Examples of such point clouds are Figure 1 Shown in.

[0026] Such point clouds are typically used in AR / VR environments and viewed by end users through devices such as TVs (television), smartphones or head-mounted devices. These point clouds are sent to the device or stored locally on the device. Many AR / VR applications use moving point clouds that change over time, rather than static point clouds. Therefore, there is a large amount of data that needs to be compressed. Today, lossless compression based on the octree representation of point cloud geometry can be achieved as low as less than 1 bit per point (1 bit per point, 1bpp). Real-time transmission may involve a frame rate of 50 frames per second (fps), with millions of points per frame, resulting in hundreds of megabits of data per second, and the above compression level may not be enough.

[0027] Therefore, lossy compression can be used to meet conventional requirements, while maintaining acceptable visual quality, and compressing sufficiently to fit within the bandwidth provided by the transmission channel and maintain real-time transmission of frames. In many applications, bit rates as low as 0.1bpp (10 times lower than lossless encoding) are sufficient for real-time transmission.

[0028] The VPCC codec based on MPEG-I Part 5 (ISO / IEC23090-5), also known as Video-based Point Cloud Compression (V-PCC) codec, can achieve such low bitrates by using a lossy compression codec that compresses 2D frames obtained from the projection of a point cloud onto a plane. The geometry is represented by a series of projected patches combined into frames, each patch being a small-scale local depth map. However, VPCC is not general and is limited to narrow point cloud types that do not show locally complex geometry (e.g. trees, hair), because the obtained projected depth maps are not smooth enough to be effectively compressed by video codecs.

[0029] Pure 3D compression techniques can handle any type of point cloud. Whether 3D compression techniques are comparable to VPCC (or any projection plus image coding scheme) on dense point clouds remains an open question. Attempts are ongoing to provide an extension (or modification) of GPCC that would have the same good dense point cloud compression performance as VPCC while maintaining the versatility of GPCC, i.e., being able to handle any type of point cloud (dense, LiDAR, 3D maps). This extension may use the so-called TriSoup encoding and decoding scheme, which is applicable to octrees and will be described in detail below. TriSoup is being explored by the ISO / IEC standardization working group JTC1 / SC29 / WG7.

[0030] There are roughly three main ways to perform lossy compression based on octree representation via the GPCC codec.

[0031] 1. Downsampling + lossless + re-upsampling;

[0032] 2. Locally modify the voxels on the encoder side;

[0033] 3. Perform local modeling on the point cloud.

[0034] The first approach roughly involves downsampling the point cloud as a whole to a smaller resolution, losslessly encoding the downsampled point cloud, and re-upsampling after decoding. Many upsampling schemes have been proposed (such as super-resolution, AI or learning-based 3D post-processing, etc.), and can provide good PSNR (Peak Signal-to-Noise Ratio) results when the downsampling is not too aggressive (for example, no more than 2 times in each direction). However, even if these indicators show good PSNR, the visual quality is controversial and cannot be well controlled.

[0035] The second approach is to have the encoder locally "adjust" the point cloud so that the encoding and decoding of the octree requires less bitrate. To do this, the points can be slightly moved so that they have occupancy information that is better predicted by neighboring nodes, resulting in a lossless encoding of the modified octree with a low bitrate. Unfortunately, this approach is less effective in reducing bitrate.

[0036] The third way is the most promising, but still needs some work before it reaches maturity. The basic idea is to encode the geometry by using a tree (octree) with a resolution as low as a certain level, such as a tree of N×N×N blocks (where N can be, for example, 4, 8 or 16). This tree is encoded using a lossless scheme, such as the GPCC scheme. The tree itself does not require much bitrate because it does not go to the deepest depth and has a small number of leaf nodes compared to the number of points of the point cloud. Then, in each NxNxN block, the point cloud is modeled by some local model. For example, the model can be a mean plane. Alternatively, the model can be a set of triangles in the so-called TriSoup scheme, which will be described in detail below. Summary of the invention

[0037] The purpose of the present application is to provide a method for encoding a 3D point cloud into a bitstream, and a method for decoding the geometry of a 3D point cloud from a bitstream, so that the delay between occupancy tree encoding and decoding and trisoup encoding and decoding is reduced.

[0038] The problem is solved by the encoding method of claim 1 , the decoding method of claim 2 , the encoder of claim 9 , the decoder of claim 10 , the bitstream of claim 11 , and the software of claim 12 .

[0039] In a first aspect, a method for encoding a 3D point cloud into a bitstream is provided, preferably applied to an encoder. The geometry of the point cloud is defined by an octree structure, the octree structure includes a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is divided into sub-volumes step by step and contains points of the point cloud, wherein the volume is divided into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each sub-volume indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, and the method comprises:

[0040] Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order;

[0041] Entropy encoding the occupancy information of each node into a bitstream up to a depth k, where k>=d;

[0042] entropy encoding trisoup information for each leaf node at depth m into a bitstream, wherein the trisoup information includes information about the presence and positions of vertices on edges of a volume associated with the leaf node;

[0043] It is characterized in that the entropy coding of the occupancy information and the entropy coding of the trisoup information are based on the same ordering, and the depth m>k.

[0044] Point cloud is a set of points in a three-dimensional coordinate system. These points are generally intended to represent the outer surface of one or more objects. Each point has a location (position) in a three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be Cartesian or any other coordinate system. Therefore, according to the present application, for a given depth d in an octree, nodes are obtained in a lexicographic sort. Preferably, the nodes are stored in a link data structure (e.g., a linked list). Wherein, the link data structure has elements connected to each other so that the elements are arranged in a sorted manner. Preferably, a list Ld can be used to store nodes, and nodes can be iteratively obtained from the list. Using lexicographic sorting (regardless of the sorting of the coordinate axes in the lexicographic sorting), the occupancy of any subvolume with a lower z position, a lower y position, and a lower x position can be used for context analysis. Therefore, compared with the prior art (e.g., Morton sorting), the prediction of different subvolumes of lexicographic sorting is more stable. In addition, since the encoding sorting of the octree information is the same as the encoding sorting of the trisoup information, a reordering process is no longer required. Therefore, the delay between the occupancy tree encoding and the trisoup encoding is reduced.

[0045] Preferably, the method further comprises storing the occupied child nodes at depth d+1 in lexicographic order. Thus, if the node at depth d is not a leaf node, the octree can continue to be traversed in lexicographic order. The child nodes at the next depth d+1 are already stored in a preferred order, thereby further facilitating the next iteration.

[0046] Preferably, the lexicographic ordering of nodes at depth d and / or the lexicographic ordering of nodes at depth d+1 is in raster scan order.

[0047] Preferably, when the occupancy information of the last level of depth is decoded, the trisoup information decoding is performed in parallel with the occupancy information decoding. Preferably, the decoding process is implemented by two separate arithmetic decoders. Since the octree information and the trisoup information can be in different parts of the bitstream, they are independent of each other, and the address of the trisoup information is known to the decoder. Therefore, the decoding process can be completed by two decoders in parallel, and the decoding efficiency is further improved.

[0048] Preferably, when encoding the occupancy information at the last level of depth, the trisoup information encoding is performed in parallel with the occupancy information encoding, and the delay is a portion of the leaf nodes. Since an edge can be shared between several adjacent leaf nodes, in trisoup encoding, if only one node information is known, the vertex information of the edge can be determined. In addition, since the nodes of the octree encoding are raster scan sorted, the trisoup encoding can be started only after encoding and decoding several nodes through the octree encoding and decoding, and there is no need to wait for the octree encoding and decoding of all nodes at the end of the last level of depth as in the prior art. The last level of depth of the octree encoding and decoding and the trisoup encoding and decoding can be implemented in parallel, and there are only a few node delays between the trisoup encoding and decoding and the octree encoding and decoding of the last level of depth.

[0049] Preferably, the occupancy information and the trisoup information are encoded into two separate parts of the bitstream. Preferably, the trisoup information starts at an addressable bit or byte of the bitstream. Therefore, the decoder can know where the trisoup information starts, so that the decoding of the occupancy information and the trisoup information can be done in parallel. Preferably, a marker is used in the bitstream to locate the starting point of the trisoup information. Preferably, the address can be provided by a syntax element.

[0050] Preferably, the occupancy information and the trisoup information are encoded into or decoded from two separate bit streams. Using two separate bit streams simplifies the processing at the encoder side.

[0051] Another aspect of the present application provides a method for decoding a 3D point cloud from a bitstream, preferably applied to a decoder.

[0052] Another aspect of the present application provides an encoder for encoding a 3D point cloud into a bit stream. The encoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned encoding method are performed.

[0053] Another aspect of the present application provides a decoder for decoding a 3D point cloud from a bit stream. The decoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned decoding method are performed.

[0054] Another aspect of the present application provides a bit stream, which is encoded by the steps of the aforementioned encoding method.

[0055] Another aspect of the present application provides a computer-readable storage medium comprising instructions for executing the steps of the method for encoding a 3D point cloud into a bitstream as described above.

[0056] Another aspect of the present application provides a computer-readable storage medium comprising instructions for executing the steps of the method for decoding a 3D point cloud from a bitstream as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Hereinafter, the present invention is described in more detail with reference to the accompanying drawings.

[0058] The accompanying drawings show:

[0059] Figure 1 A long skirt of dense point cloud (provided by 8i) is shown.

[0060] Figure 2 The TriSoup vertices (black dots) along the edges of the volumes associated with the leaf nodes are shown.

[0061] Figure 3 The TriSoup triangles within a leaf node are shown.

[0062] Figure 4 Determining the TriSoup triangles within a leaf node is shown.

[0063] Figure 5 Another triangle configuration is shown when another coordinate axis is used as the principal axis.

[0064] Figure 6 Ray tracing is shown to render TriSoup triangles into a decoded point cloud.

[0065] Figure 7 A raster scan process of a parent node for raster scan encoding the occupancy of its child nodes is shown.

[0066] Figure 8 The encoding ordering of slice 0 and slice 1 of the occupied nodes of the same fixed x at the current depth is shown.

[0067] Fig. 9 An example of a decoder is shown.

[0068] Fig.10 An example of an encoder is shown.

[0069] Fig.11 Shown is the occupancy map (white cubes) known from previously encoded occupied bits when the occupied bits of a child node (striped cubes) are encoded using raster scan ordering.

[0070] Fig.12 A raster scan ordered parent node is shown with 8 encoded child occupation bits and a raster scan ordering of child nodes.

[0071] Fig.13 Another example of a decoder is shown (decoding method of parent nodes using raster scan ordering, where 8 child occupied bits are encoded together).

[0072] Fig.14 Another example of an encoder is shown (encoding method of parent nodes using raster scan ordering, where 8 child occupied bits are encoded together).

[0073] Fig.15 Shown is the occupancy map (white cubes) known from previously coded occupied bits when the occupied bits of child nodes (striped cubes) are coded using raster scan ordering of the nodes but all bits of occupancy of child nodes are encoded using Morton ordering.

[0074] Fig.16 The raw byte sequence payload of a DU (data unit) with Nu bytes is shown.

[0075] Fig.17 An example of determining the vertices of an edge when the child nodes are encoded in raster scan order is shown.

[0076] Fig.18 It shows that when trisoup encoding starts for the first child node, the buffer L tris Example of a child node in .

[0077] Fig.19 Separate Data Units (DU) for octree information and trisoup information, respectively, are shown.

[0078] Fig. 20 A schematic flow chart of a method for encoding is shown.

[0079] Fig.21 A schematic flow chart of a method for decoding is shown.

[0080] Fig. 22 An encoder according to the present application is shown.

[0081] Fig.23 A decoder according to the present application is shown. DETAILED DESCRIPTION

[0082] TriSoup works by locally modeling the point cloud, using a set of triangles without explicitly providing connectivity information, hence the name “Soup of Triangles”.

[0083] like Figure 2As shown, the vertices of the triangles are encoded along the edges of the volume associated with the leaf nodes of the tree. These vertices are shared between leaf nodes with common edges. This means that at most one vertex is encoded per edge belonging to at least one leaf node. In this way, the continuity of the model is ensured through the leaf nodes.

[0084] The encoding of TriSoup vertices requires two pieces of information per edge:

[0085] a vertex flag indicating whether there is a TriSoup vertex on the edge, and

[0086] • The location of a TriSoup vertex along the edge, when present.

[0087] Therefore, the encoded data includes the octree data plus the TriSoup data. As described below, the increase in data size brought by the TriSoup model is more than offset by the improvement in point cloud reconstruction accuracy brought by the TriSoup triangles.

[0088] The vertex flags are encoded by an adaptive binary arithmetic coder that uses a specific context to encode the vertex flags. By putting s bits (bypass / no entropy coding) into the bitstream, the positions of vertices on edges of length N=2s can be encoded with unit precision.

[0089] Inside a leaf node, if there are at least three vertices on the edge of the leaf node, a triangle is constructed from the TriSoup vertices. The constructed triangle is as follows Figure 3 shown.

[0090] Obviously, other combinations of triangles are possible. The selection of triangles is a three-step process.

[0091] 1. Determine the main direction along one of the three axes;

[0092] 2. Sort the TriSoup vertices according to the main direction;

[0093] 3. Construct a triangle based on the ordered list of vertices.

[0094] Figure 4 will be used to explain the process. Each of the three axes is tested and the axis with the largest total triangular surface is taken as the principal axis.

[0095] To simplify the drawings, Figure 4 Only the tests on two axes are shown in . The first test along the vertical axis is performed by projecting the cube and the trisoup vertices vertically onto the 2D plane ( Figure 4The vertices are then ordered in a clockwise order relative to the center of the projection node (square). The triangles are then constructed according to a fixed rule based on ordered vertices. Here, when 4 vertices are involved, triangles 123 and 134 are systematically constructed. When there are 3 vertices, the only possible triangle is 123. When there are 5 vertices, the fixed rule may be to construct triangles 123, 134, and 451. And so on, up to 12 vertices.

[0096] The second test along the horizontal axis is performed by projecting the cube and trisoup vertices horizontally onto the 2D plane ( Figure 4 left side).

[0097] The vertical projection shows that the total 2D surface of the triangle is maximal, so the principal axis is chosen to be vertical, and the constructed TriSoup triangle is obtained by following the order of the vertical projection, as Figure 4 As shown, inside the node.

[0098] It should be noted that using the horizontal axis as the principal axis will produce Figure 5 Another configuration of the triangle shown.

[0099] The sufficient selection of principal axes by maximizing the projection surface allows for continuous reconstruction of the point cloud without holes.

[0100] TriSoup triangles are rendered into points via ray tracing. The set of all points rendered via ray tracing constitutes the decoded point cloud.

[0101] The rays are emitted in three directions parallel to the axis, such as Figure 6 As shown. Its origin is an integer (voxelized) coordinate point with the precision corresponding to the sampling precision required for rendering. The intersection point with one of the trisoup triangles (the dotted point if any) is then voxelized (i.e. rounded to the nearest point of the required sampling precision) and added to the list of rendered points.

[0102] After applying Trisoup to all leaf nodes (i.e., building triangles and obtaining points by ray tracing), discard all copies of the same point in the list of rendered points (i.e., keep only one voxel among all voxels sharing the same position and volume) to obtain a set of decoded (unique) points.

[0103] In the prior art of lossy point cloud encoding and decoding using the trisoup method, the Morton sorting of the nodes is first used for lossless octree encoding and decoding of the nodes until the specified depth in the octree, after the octree geometry encoding and decoding, and then the triangle soup representation is used on the leaf nodes for trisoup information. In order to encode / decode the trisoup information including the vertex presence flag and vertex position of each leaf node, the detailed process follows the following steps:

[0104] Determine the trisoup vertex for each node by:

[0105] Obtain the data stored in list L through Morton-order d The leaf nodes in

[0106] Iterate over the list L d Each leaf node in the to determine the vertex existence flag and vertex position of each edge of each leaf node;

[0107] After getting the vertex information of each node, put all the edges of the leaf nodes (with vertex information) into the list L s middle;

[0108] · List L of lexicographically sorted pairs on the three x, y, and z axes of the edge s Sort by

[0109] By keeping only the leaf nodes that share a common edge and removing them from the sorted list L s Remove other elements that are the same as the current edge to find the unique edge;

[0110] Calculate the vertices (vertex presence flag and vertex position) for each unique edge that intersects the surface;

[0111] Build triangles and perform voxelization of the triangle-based model to reconstruct the geometric information of the point cloud in the encoder and decoder, where a reordering process is also required to obtain the final vertices for each unique edge of the triangle voxelization;

[0112] Encode / decode vertex information of all leaf nodes into / from the bitstream.

[0113] According to some methods, the subvolume occupancy bits are encoded / decoded based on the raster scan ordering of the nodes at a given depth in the occupancy tree. In some cases, the raster scan ordering of the subvolumes is used to encode / decode their respective occupancy bits. In some cases, the raster scan ordering of the nodes is used to encode / decode (together) the occupancy bits of all their corresponding subvolumes for better coding efficiency.

[0114] In 2D, the raster scan order of pixels is simply a lexicographic ordering of the pixels according to their 2D coordinates x and y. A lexicographic ordering in (y, x) is commonly used on images: pixels are processed sorted first by x and then by y; pixels are sorted by increasing x coordinate for a given y coordinate and by increasing y coordinate. 3D raster scan ordering is similar, but in three dimensions (i.e., 3D raster scan ordering is a lexicographic ordering on the arrangement of 3D coordinates). Raster scan ordering of voxels follows only a lexicographic ordering, but in three dimensions x, y, and z, e.g., a lexicographic ordering in (z, y, x) can be used for volumetric images (or t, y, x for time series of images, e.g., in video).

[0115] An example of the method described previously is in Figure 7 For better understanding, the entire occupied volume is represented in the figure to better see the raster scan ordering of the processing. However, it should be clear that if the ancestor node has been marked as unoccupied, part of the volume will not be encoded / processed.

[0116] like Figure 7 As shown, a lexicographic ordering in (x, y, z) is used (i.e., a cooperative lexicographic ordering in (z, y, x)). For a given depth in the octree, the occupancy of the child nodes is encoded according to the raster scan ordering of the child nodes. In order to achieve the raster scan ordering of the child nodes, 4 pipelines (pipeline 0, pipeline 1, pipeline 2, pipeline 3) are defined for each node at the current depth, each pipeline consisting of each node with the same x and y. The encoding of the occupancy bits of the child nodes of pipeline 0 is performed by encoding the occupancy bits b0 and b1 of all occupied nodes with the same fixed x and y, and the nodes are sorted by increasing the z of all occupied nodes. The encoding of the occupancy of pipeline 1 is then performed by encoding the occupancy bits b2 and b3 of the same node and in the same order as pipeline 0.

[0117] The encoding process for encoding occupied pipe 0 for all occupied nodes with the same fixed x and y and occupied pipe 1 for all occupied nodes with the same fixed x and y is repeated for each y in increasing y order. This provides a raster scan encoding of slice 0 consisting of each bit b0, b1, b2, b3 of all occupied nodes with the same x. An illustration of the raster scan ordering of slice 0 for these nodes is shown in Figure 8(a) is shown, where each child node is represented in 2D for better illustration. After encoding slice 0 of the nodes at a fixed x, slice 1 of these nodes is then encoded at the same x, and the encoding process of slice 0 is reproduced for slice 1 by continuously encoding in the same order, the occupied pipe 2 then 3 consists of the occupied bits b4, b5 then b6, b7 of the same node. And the raster scan ordering of slice 1 of these nodes is shown in the encoding process of the diagram in Figure 8 (b) where each child node is represented in 2D for better illustration. The decoding process of slice 0 followed by slice 1 is repeated in increasing order for every x nodes at the current depth of all occupied nodes.

[0118] An example of the decoder described previously is in Fig. 9 is shown as a block diagram, where the list L d Represents the node N at depth d in the octree k (x k ,y k ,z k ), with nodes sorted from N0 to N in raster scan order last Sort, and k is the list L d The sorted indices of the nodes in . Fig. 9 Both of the following are shown:

[0119] The process of decoding the occupancy of the child node (sIdx, tIdx, i) at depth d+1 from a bitstream following raster scan ordering;

[0120] And put the child nodes into list L d+1 The process of raster scanning ordered nodes to generate the next depth.

[0121] The variable sIdx represents the index of the slice of all occupied nodes with the same x; the variable tIdx represents the index of the pipe of all occupied nodes with the same x and y; and the variable i represents the index of the child node within each pipe.

[0122] In other words, the integer value obtained from the binary word formed by (sIdx, tIdx, i) is the child index of the node (decimal value from 0 to 7).

[0123] The variable kSStart is used to store the sorted index of the first node of the slice of all occupied nodes with the same x; and the variable kTStart is used to store the sorted index of the first node of the pipe of all occupied nodes with the same x and y.

[0124] The depth dMax is the maximum depth in the tree and can be determined from the bitstream.

[0125] The detailed process in the decoder can be described as follows:

[0126] To initialize the algorithm, by setting the list L0 to contain only the root node of the occupancy tree, and setting the initial depth d to 0.

[0127] Tag 0:

[0128] Get a (first in first out, fifo) list L storing the nodes at depth d in raster scan order d ;

[0129] Get an empty list L d+1 to generate a child node in a raster scan order for decoding the next depth;

[0130] Set the node index k to 0 (initialize the variable k with the value 0).

[0131] Label 1:

[0132] Set the slice index sIdx to 0 (initialize the variable sIdx with the value 0).

[0133] Set the first node index in the slice to k (initialize the variable kSStart with the value k).

[0134] Label 2:

[0135] Set the pipe index tIdx to 0 (initialize the variable tIdx with the value 0);

[0136] Set the first node index in the pipeline to k (initialize the variable kTStart with the value of k).

[0137] Label 3:

[0138] From list L d Get node N k (x k ,y k ,z k );

[0139] Set the sub-index in the pipeline to 0 (initialize the variable i with the value 0).

[0140] ●Process 0: Decode and generate L d Middle Node N k the ordered children of the children of the pipe tIdx of the slice sIdx;

[0141] Occupied bits of the i-th child of the pipeline tIdx that decodes slice sIdx from the bitstream

[0142] (It is N kChild nodes of (sIdx, tIdx, i));

[0143] If the decoded bit indicates that the child node is occupied, then in list L d+1 At the end of the pipeline tIdx, a new node (which is N) of the i-th child node of the slice sIdx is appended k The new node has coordinates

[0144] (2×x k +sIdx,2×y k +tIdx,2×z k +i);

[0145] If i is not equal to 1, increase i and repeat process 0, otherwise go to the next step. If N k is the last node of depth d (i.e., the list L d The last node of ), then:

[0146] If N k If it is not the last node of depth d, then remove it from list L d Get the next occupied node N k+1 (x k+1 ,y k+1 ,z k+1 ). Then determine whether it is x k ≠x k+1 return

[0147] Yes k ≠y k+1 ;

[0148] If x k ≠x k+1 or k ≠y k+1 , it means that the next node does not belong to the same pipeline as the occupied node with the same x and the same y. (Then determine whether tIdx == 1;

[0149] If tIdx == 1, it means that the processing of both pipelines (in the pipelines of all occupied nodes with the same x and y) of the child nodes of the current slice of the child node has been completed. Then determine whether x k ≠x k+1 ;

[0150] If x k ≠x k+1 , it means that the next node does not belong to the same slice of the occupied node with the same x. Then determine whether sIdx == 1;

[0151] If sIdx == 1, it means that the processing of both slices of the child node (in the slices of all occupied nodes with the same x) has been completed. k+1 is the next slice of the occupied node and the starting point of the next pipeline of occupied nodes to be processed;

[0152] ● Increase k by 1 and loop to Label1 to start processing the first child node pipeline of the first child node slice of the next occupied node slice;

[0153] ● Otherwise sIdx == 0. This means we can start processing the second slice of the occupied node;

[0154] Label 4:

[0155] Increase the slice index sIdx by 1 and reset k to the index of the first node of the slice of all occupied nodes with the same x: k = kSStart;

[0156] Loop to label 2 to start the third pipeline processing the child nodes;

[0157] Otherwise x k = = x k+1 . This means that the next node belongs to the same slice of the occupied node, but belongs to the next pipeline of the occupied node.

[0158] Therefore, N k+1 is the starting point of the next pipeline to be processed;

[0159] Increase k by 1 and loop to Label2 to start processing the first pipeline of the child node of the next pipeline of the occupied node (belonging to the current slice sIdx of the child node);

[0160] Otherwise tIdx == 0. This means we can start processing the second pipeline of the child nodes of the current slice of child nodes;

[0161] Label 5:

[0162] Increase the pipeline index tIdx by 1 and reset k to have the same x

[0163] and y is the index of the first node of the pipeline of all occupied nodes:

[0164] k = kTStart;

[0165] Loop to label 3 to start the second pipeline processing the child nodes;

[0166] Otherwise x k = = x k+1 and k = =y k+1This means that the next node belongs to the same pipeline as the occupied node. Increment k by 1 and loop to Label3 to continue processing the same pipeline for the child nodes;

[0167] · Otherwise N k == N last , this node is the last node at depth d. Then check if tIdx == 1;

[0168] ● If tIdx == 1, it means that the processing of the last pipeline of the child nodes of the current slice of the child nodes has been completed. And the next slice (if any) of the child nodes

[0169] can be processed. Then check if tIdx == 1;

[0170] · If sIdx == 1, it means that the processing of the last slice of the child nodes of the last slice of the occupied node has been completed. And the next depth (if any) in the occupancy tree

[0171] · If d == dMax, it means that the last depth of the occupancy tree has been decoded. Then complete the occupancy tree;

[0172] · Otherwise, d < dMax. This means that the next depth of the occupancy tree can be processed. Increment the depth d by 1 and loop to Label0 to process the next depth in the occupancy tree;

[0173] ● Otherwise sIdx == 0. This means that the processing of the second slice of the child nodes of the last slice of the occupied node can be started. Go to label 4 to start;

[0174] · Otherwise tIdx == 0. This means that the processing of the last pipeline of the child nodes of the current slice of the child nodes can be started. Go to label 5 to start.

[0175] The list L can be omitted d+1 , and the child nodes are not appended to this list when d == dMax because this list will not be used as there are no more depths to process.

[0176] The coordinates of the child nodes are refined from the coordinates of their parent nodes with one-bit precision. In other words, some physical space coordinates of the node center can be obtained by multiplying the coordinates of the node center by the physical dimensions of the node volume scaled according to the depth:

[0177] ((x k , y k , z k )+(0.5, 0.5, 0.5)) × (width, length, height) / 2 深度 .

[0178] Preferably, when d == dMax, if the child node is occupied, the output has coordinates ((2×x k +sIdx,2×y k +tIdx,2×z k +i)+(0.5, 0.5, 0.5))×(width, length, height) / 2 dMax Generates raster scan ordered point geometry.

[0179] According to the above encoding process of the child nodes, it is observed that for each node N k Needs to be accessed / processed 4 times: the first time to encode occupied pipe 0, the second time to encode occupied pipe 1, the third time to encode occupied pipe 2, and the last time to encode occupied pipe 3.

[0180] This encoding process builds the nodes required for the raster scan ordering of the child nodes occupied The mentioned ordering is obtained from the raster scan ordering of the nodes being processed. And the raster scan ordering of the nodes is directly obtained from the raster scan ordering of the child node occupancy codec performed on the parent node (i.e., during the previous depth of the octree occupancy codec).

[0181] An example described previously is Fig.10 is shown as a block diagram.

[0182] The effect of the previously described method is Fig.11 , which shows the occupancy map coverage of the known sub-volumes when encoding the occupancy bits of the striped sub-volumes when the occupancy bits are encoded according to the raster scan ordering of the sub-volumes of the nodes. Fig.11 Also shown are neighboring positions that can be used to construct a context for entropy coding of occupied bits. With raster scan ordering (regardless of the ordering of the coordinate axes in the lexicographic ordering), the occupancy of any subvolume with a lower z position, lower y position, lower x position can be used for context analysis. Thus, a predetermined pattern of neighboring subvolumes can be defined and used to select a coding context for any coding occupied bit.

[0183] According to another example, the method may use a raster scan ordering of nodes to (together) encode / decode the occupancy bits of all its corresponding subvolumes at a given depth in the occupancy tree. Fig.12As shown, occupied nodes are processed in raster scan ordering, but the occupancy bits of all subvolumes of the node are encoded consecutively at the node by ordering in a single channel, for example using Morton scan ordering of the subnode occupancy bits (which means that each subvolume of the node is encoded using Morton ordering in a single channel at the node); and in order to order the subnodes by raster scan ordering for the next depth, the same subnode ordering method as in the first embodiment is used, which uses 4 channels on each node at the current depth and Fig. 9 Shown in.

[0184] Fig.13 An example of the sub-node occupation decoding process of the decoder in this embodiment is shown, and Fig.14 An example of the encoder of this embodiment is shown, and the sorting process of the child nodes is not in Fig.13 and Fig.14 Shown in.

[0185] The detailed process in the encoder / decoder can be described as follows:

[0186] Obtain the fifo list L of nodes at storage depth d by raster scan sorting d

[0187] · In the list L from the fifo d Get node N at depth d k (x k ,y k , z k ), and initially starting from the first node N0(x0, y0, z0) at each depth,

[0188] Encode the bits occupied by the child nodes into a bit stream according to a certain order (such as Morton order), or decode them from the bit stream,

[0189] If N k Not a fifo list L d The last node in the fifo list L d The next node in (N k+1 (x k+1 ,y k+1 , z k+1 ) to encode / decode until L is reached d The last node in

[0190] ●Encode the occupancy of child nodes at the next depth until the final depth is reached.

[0191] Fig.15 Shows Figure 7The described alternative coding ordering encodes the occupancy bits of the striped subvolumes with a known subnode occupancy map. Using the raster scan ordering of the nodes (regardless of the axis ordering used in the lexicographic ordering), the occupancy of any subvolume with a node having a lower z, lower y, or lower x position than the current node (the parent of the striped subvolume) can be used to contextualize the entropy encoding of the striped subvolume occupancy bits as well as any subvolume occupancy bits previously encoded for the current node. Using this approach, we can clearly distinguish that there is a different pattern of available neighboring nodes for each position of the coded subvolume occupancy. Therefore, in the case of the octree occupancy, there are 8 different patterns. Therefore, the entropy encoding of the occupancy bits needs to be tuned according to each pattern (i.e., for each occupancy bit index) in order to be more optimal.

[0192] This complicates the bits of coding relative to the first embodiment, but is still better than the pattern in the octree with nodes coded in Morton order in the prior art. In the prior art, for Morton ordering of nodes, the pattern occupied by "unknown" (i.e., not yet coded) neighboring nodes varies more with the position in the ancestor node (and therefore with the Morton index), so some neighboring information is not always reliable when constructing the entropy context, and it may introduce bias in the entropy coding probability constructed in the adaptive entropy coding context, thereby reducing coding performance.

[0193] It should be understood that in point cloud encoding, occupied nodes and subnodes will rarely (and preferably never) differ from example Figure 7 , Fig.11 , Fig.12 and Fig.15 These graphs are densely represented to provide a better view of the available causal sub-occupancy information and the node / sub-node processing ordering. In a usual point cloud, several volumes will be unoccupied, therefore, the nodes of these volumes will not exist and will be processed. The processing ordering of occupied nodes is the same as in the dense content case, and for a given occupied node, the occupancy of all sub-volumes is always encoded, but unoccupied nodes / volumes are not processed at all since they do not exist in the tree structure. Compared to the processing ordering of dense content, the processing of these unoccupied nodes / volumes is skipped.

[0194] As mentioned above, in the lossy point cloud encoding using the trisoup method, assuming that the leaf node is at depth d, first, the lossless octree geometry codec is based on the Morton sort node at depth d-1, and its generation is stored in the list L dThe Morton sorted leaf nodes at depth d at . Then, the trisoup information is generated using the triangle soup representation on the leaf nodes, during which a process of sorting the edges of all leaf nodes is required because they are generated by using the Morton scan sorted leaf nodes, and the edges used for vertex information should be sorted according to the raster scan. After sorting the edges of all leaf nodes according to the raster scan sort (i.e., the dictionary sorting on the three axes of x, y, and z), trisoup encoding is performed. Therefore, a reordering step is required between the octree encoding and decoding and the trisoup information encoding and decoding to change from Morton sorting to raster scan sorting.

[0195] Disadvantages of current trisoup encoding method:

[0196] This reordering process is expensive in terms of computation and memory access, and has latency: processing of trisoup edges requires that most (or even all, with current designs) of the last depth of the tree be already decoded before the reordered nodes can be obtained, and therefore before processing of the trisoup information to reconstruct the point cloud can begin. Furthermore,

[0197] ● The reordering happens not only at one place during the trisoup encode / decode process, but also in both the vertex determination process and the trisoup voxelization process, so the computational complexity cost is much higher and causes high latency between octree encode / decode and trisoup encode / decode;

[0198] • And especially on the decoder side: since the octree nodes are decoded and stored in Morton ordering, the decoder needs to decode a large portion of the octree information of the last depth before it can start processing the trisoup.

[0199] Furthermore, according to existing solutions, the coded trisoup information enters the bitstream after the occupancy tree information, making it impossible to encode or decode the coded trisoup information without previously completing the encoding or decoding of the occupancy tree information: there is no CABAC (Context-based Adaptive Binary Arithmetic Coding) reset between the two bitstream parts. Therefore, the problem to be solved is how to reduce the delay between occupancy tree encoding and trisoup encoding, and between occupancy tree decoding and trisoup decoding.

[0200] To solve the above problems, the proposed solution is

[0201] Use the same ordering of the nodes for octree and trisoup information encoding and decoding. For example, Morton ordering for both or preferably raster scan ordering for both, as raster scan ordering is more convenient, especially for trisoup encoding;

[0202] • Encode and decode octree information and trisoup information into two independently accessible parts of the bitstream.

[0203] By doing so, the need to wait for reordering is eliminated, and the encoding / decoding of the trisoup information can be started before the occupancy tree encoding / decoding is completed. Therefore, a portion of the encoding / decoding of the trisoup information can be performed in parallel with the octree encoding / decoding (occupancy tree).

[0204] Therefore, according to the present application, the delay between occupancy tree encoding and trisoup encoding and between occupancy tree decoding and trisoup decoding can be reduced.

[0205] In a first embodiment, in addition to using the same ordering of nodes for both the octree and the trisoup, such as a raster scan ordering (a variant Morton ordering may also be used), the trisoup information and the octree information are both in separate parts of the same data unit (DU) of the point cloud slice. Fig.16 An example of DU is shown in FIG. Fig.16 As shown, the payload of a data unit (DU) is a sequence of Nu bytes. The first two bytes contain the DU header, and the remaining bytes are called the raw byte sequence payload (RBSP). The last byte of the RBSP containing the encoded data includes the RBSP stop bit ('1') followed by a '0' bit to achieve byte alignment.

[0206] However, the trisoup information needs to start at an addressable bit or byte in the bitstream (DU): in one example, the cabac encoder is flushed between the octree and trisoup information encoding and decoding (cabac flushing means outputting the remaining bits that may be left in the cabac buffer and outputting any additional bits that the decoder needs in order to correctly decode the last binary symbol encoded before the flushing operation). After the flush, the cabac encoder is reset so that the cabac decoder can start decoding from subsequent bits in the bitstream (in byte-aligned variants, padding bits are inserted after the flushed bits so that the decoder can start decoding from the next byte, and in some other variants, padding bytes may also be inserted to align the first byte to multiple bytes). Therefore, both the encoder and the decoder can start encoding / decoding the trisoup information to / from the bitstream, starting from the starting bit / byte position and using the newly initialized entropy encoder / decoder. In order for the decoder to know where the trisoup information bitstream starts, the addressable bit or byte needs to be signaled / indicated to the decoder. And to signal / indicate to the decoder where the trisoup information bitstream starts in a DU, there are two example methods:

[0207] 1. An example method is to use a marker and put the marker into the bitstream to locate the start of the trisoup information, where the marker refers to a bit sequence that is not allowed in the bitstream (for other content besides the marker), for example, the system does not allow a 32-bit sequence where each bit is '0', so this sequence can be used as a marker to signal the decoder to start another part of the bitstream to locate a specific bitstream in the DU. Typically, when the encoder should have output this bit sequence, another reserved symbol (e.g., a sequence of 31 bits equal to 0 and 1 bit equal to 1) is used as an escape sequence, and when encountered, the next bit in the bitstream replaces the last bit: 31 bits equal to 0, 1 bit equal to 1, and 1 bit equal to 0 indicates to the decoder that it must use a sequence of 32 bits equal to 0; 31 bits equal to 0, 2 bits equal to 1 indicate to the decoder that it must use a sequence of 31 bits equal to 0 and 1 bit equal to 1. Typically, the marker uses byte alignment and is set as a 4-byte sequence, where each byte has 8 bits. And a marker can be placed at the end of the octree information bitstream to start the trisoup information bitstream in the DU at the encoder and decoder, after decoding the marker, it knows that the subsequent bitstream needs to be decoded by the initialized entropy decoder.

[0208] 2. Another example method is in a syntax element (e.g., in the DU header of a point cloud slice)

[0209] The address of the bit or byte is provided to easily locate the bit or byte for the beginning of the first information bit stream. And the address of the bit or byte can be obtained by calculating the offset between the bit or byte and the starting bit or byte of the DU. And the DU header is in Fig.16 Shown in.

[0210] Decoding method in the first embodiment:

[0211] Since the octree information and the trisoup information are located in different parts of the bitstream and the bit or byte address of the trisoup information is known to the decoder, the first embodiment allows the octree and trisoup information to be decoded in parallel by decoding them using two separate arithmetic decoders, one for octree decoding and the other for trisoup decoding. When the entropy encoding of the trisoup information (vertex existence flags and vertex positions) does not depend on the geometry of the nodes, the parallel decoding of the bitstream can be performed independently. After decoding the occupancy information of the octree and trisoup information of the leaf nodes (such as vertex existence flags and vertex positions), voxelization of the trisoup triangles is implemented to reconstruct the point cloud data.

[0212] Decode method variants:

[0213] However, in recent versions of trisoup, the compression of trisoup information uses the geometry of local nodes. Therefore, some dependencies are introduced between occupancy information and trisoup information. In any case, the determination of the positioning of the edges and the voxelization of the trisoup triangles (and therefore the reconstruction of the point cloud data) depend locally on the geometry of the nodes and the trisoup information of the leaf nodes. Therefore, the geometry of the nodes and the trisoup are mutually correlated. And if you want to perform three tasks in parallel (decoding of occupancy information at the last depth, decoding of trisoup information and voxelization), this mutual correlation introduces only a small amount of delay. For example, when the entropy coding context of the vertex information depends on the occupancy of all nodes around the corresponding edge, the delay is roughly one slice of the leaf node for the entropy coding of the trisoup information (i.e., all leaf nodes have, for example, the same x and y); and when voxelization is performed, two slices of the leaf node obtain all the surrounding edges of the node.

[0214] In the example of parallel octree and trisoup decoding in this variant, where child nodes are encoded in raster scan order (e.g., Figure 7-9 Assuming that a leaf node is at depth d in the octree, it uses octree decoding for the node’s occupancy tree up to depth d-1, and uses trisoup decoding to decode the trisoup information of the leaf node at depth d. The proposed decoder follows these steps:

[0215] First, get the octree information bitstream and trisoup information bitstream;

[0216] Octree decoding is then used to decode the node occupancies in raster scan order until depth d-1, and during this process the children of the node at depth d-2 are put into list L d-1 So we can get a list L of nodes stored in raster scan order. d-1 , which can be used for the last level of octree decoding, i.e., depth d-1;

[0217] Then, for the node at depth d-1, the octree decoding of the node is interleaved with the trisoup decoding by taking advantage of the raster scan ordering of the nodes. Specifically, for example,

[0218] Get the list L of nodes at storage depth d-1 in raster scan order d-1 ;

[0219] From List L d-1 Process node N k (k starts at 0), and decodes the occupancy of its child nodes (leaf nodes) from the bitstream in raster scan order;

[0220] Starting from decoding the first child node, the node information of each child node is stored in the buffer L after being decoded by the octree codec. tris It can be used for trisoup decoding of these already decoded child nodes;

[0221] And when the sIdx and tIdx of the current child node meet the conditions (sIdx=1 and tIdx=1) for the first time, after the current child node is decoded, trisoup encoding and decoding starts from the first child node of N0 (sIdx and tIdx are 0), and N0 is also L d Therefore, the buffer L tris Stores the following node information:

[0222] a node slice with the same x and a node pipeline with the same x and y that precedes the current child in raster scan order,

[0223] and the current child node,

[0224] Its Fig.18 As shown in the figure, the transparent green node represents the current child node.

[0225] And the nodes in transparent pink, yellow, and orange are the decoded child nodes.

[0226] After the octree of the current child node is decoded, if its information is occupied, its information is put into buffer L tris Then, we can obtain the information of all adjacent child nodes of the first child node and list L d The vertices of each edge of the first child node (which is the first leaf node) in are determined, so trisoup decoding can be started from this child node;

[0227] Then the decoded pairs are stored in list L through the octree d The next child node in the list L is decoded, and then trisoup iteratively decodes the next child node in the list L d The second child node (leaf node) in is decoded. Therefore, there is only a small delay between octree decoding and trisoup decoding;

[0228] • During the trisoup decoding process, the information of each leaf node is decoded from a dedicated bitstream.

[0229] The encoding method in the first embodiment:

[0230] In the first embodiment, at the last level of the octree encoding of the occupancy tree, trisoup encoding is performed in parallel with octree encoding. Since an edge can be shared between several adjacent leaf nodes, in trisoup encoding, if only one node information is known, the vertex information of the edge may not be determined. Fig.17 As shown, taking the middle (left) edge as an example, the vertex information of the edge can only be obtained when the occupancy information of the 4 nodes (sharing the edge) is known, and then the vertex information of the edge can be encoded. Since the nodes of the octree encoding are in raster scan order, the trisoup encoding can only start after encoding and decoding several nodes through the octree encoding and decoding, and there is no need to wait for the octree encoding and decoding of all nodes at the end of the last level of depth as in the prior art. The last level of depth of the octree encoding and decoding and the trisoup encoding and decoding can be implemented in parallel, and there is only a few node delay between the trisoup encoding and decoding and the octree encoding and decoding of the last level of depth.

[0231] In the example of parallel octree and trisoup encoding in the first embodiment, child nodes are encoded in raster scan order. Assuming that a leaf node is at depth d in the octree, it uses octree encoding for the node's occupancy tree until depth d-1, and uses trisoup encoding to encode the trisoup information of the leaf node at depth d, the proposed encoder follows these steps:

[0232] First, octree encoding is used to encode node occupancies in raster scan order up to depth d-1, and during this process, the children of the node at depth d-2 are put into list L d-1 So we can get a list L of nodes stored in raster scan order. d-1 , which can be used for the last level of depth of octree encoding, that is, at depth d-1;

[0233] Then, for the node at depth d-1, the octree encoding of the node is interleaved with the trisoup encoding by taking advantage of the raster scan ordering of the nodes. Specifically, for example,

[0234] ● Obtain a list L of nodes at storage depth d-1 in raster scan order d-1 ;

[0235] ●Processing List L d-1 N working nodes k (k starts at 0), and encodes its child nodes (leaf nodes) in raster scan order as described in the previously described method, and places the child nodes in list L d At depth d in;

[0236] Starting from the first child node, the node information of each child node is stored in the buffer L after being encoded by the octree codec. tris It can be used for trisoup encoding and decoding of these already encoded and decoded child nodes;

[0237] When the sIdx and tIdx of the current child node meet the conditions (sIdx=1 and tIdx=1) for the first time, after the current child node is encoded, trisoup encoding and decoding starts from the first child node of N0 (sIdx and tIdx are 0), and N0 is also L d Therefore, the buffer L tris Stores the following node information:

[0238] ● a node slice with the same x and a node pipeline with the same x and y that precedes the current child in raster scan order,

[0239] ● and the current child node,

[0240] Its Fig.18 , where the transparent green node represents the current child node, and the nodes in transparent pink, yellow and orange are the already encoded child nodes. After the octree encoding of the current child node, if its information is occupied, its information is put into the buffer L tris Then, we can obtain the information of all adjacent child nodes of the first child node and list Ld The vertices of each edge of the first child node (which is the first leaf node) in are determined, so trisoup encoding can be started from this child node;

[0241] Then the octree encoding pairs are stored in list L d The next child node in the list L is encoded, and then trisoup iteratively encodes the next child node in the list L d The second child node (leaf node) in

[0242] Therefore, there is only a small delay between octree encoding and trisoup encoding;

[0243] During the trisoup encoding process, the trisoup information bitstream of each leaf node is placed into a buffer.

[0244] Finally, once the octree encoding is completed, the bitstream of trisoup information is appended to the end of the octree bitstream, so the octree information bitstream and the trisoup information bitstream are in separate parts of the DU.

[0245] In the second embodiment, the same ordering of nodes is also used for octree encoding and trisoup encoding, such as raster scan ordering, and the parallelism of octree encoding and trisoup encoding is achieved at the encoder and decoder, and the processing on the encoder side is also simplified (and the buffered bitstream does not have to be concatenated), and the octree and trisoup information of the point cloud slice can be encoded into separate data units (DU). The entire encoding process is the same as in the first embodiment, except that each point cloud data slice has two DUs, such as DU1 for the occupancy information byte sequence payload of the data unit with Nu1 bytes, and DU2 for the trisoup information byte sequence payload of the data unit with Nu2 bytes, such as Fig.19 shown.

[0246] refer to Fig. 20 , which shows a schematic flow chart of a method for encoding a 3D point cloud into a bitstream according to the present application.

[0247] The method includes:

[0248] Step S10, along each axis X, Y, Z of the coordinate system, obtain a node with a depth of d that contains at least a portion of the point cloud by lexicographic sorting;

[0249] In step S11, the occupancy information of each node is entropy encoded into a bit stream until a depth k, where k>=d;

[0250] In step S12, the trisoup information of each leaf node with a depth of m is entropy encoded into a bitstream, wherein the trisoup information includes information on the existence and position of vertices on each edge of the volume associated with the leaf node; the entropy encoding of the occupancy information and the entropy encoding of the trisoup information are based on the same ordering, and the depth m>k.

[0251] refer to Fig.21 , which shows a schematic flow chart of a method for decoding a 3D point cloud from a bitstream according to the present application.

[0252] The method includes:

[0253] Step S20, along each axis X, Y, Z of the coordinate system, obtain a node with a depth of d that contains at least a portion of the point cloud in a lexicographical order;

[0254] In step S21, the occupancy information of each node is entropy decoded from the bitstream until a depth k, where k>=d;

[0255] In step S22, the trisoup information of each leaf node with a depth of m is entropy decoded from the bitstream, wherein the trisoup information includes information on the existence and position of vertices on each edge of the volume associated with the leaf node; the entropy decoding of the occupancy information and the entropy decoding of the trisoup information are based on the same ordering, and the depth m>k.

[0256] Reference now Fig. 22 , which shows a simplified block diagram of an example embodiment of an encoder 1100. The encoder 1100 includes a processor 1102 and a storage device 1104. The storage device 1104 can store a computer program or application containing instructions that, when executed, cause the processor 1102 to perform several operations such as those described herein. For example, the instructions can encode and output a bit stream encoded according to the method described herein. It should be understood that the instructions can be stored on a non-transitory computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1102 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the described process. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system."

[0257] refer to Fig.23, which shows a simplified block diagram of an example embodiment of a decoder 1200. The decoder 1200 includes a processor 1202 and a storage device 1204. The storage device 1204 may include a computer program or application containing instructions that, when executed, cause the processor 1202 to perform several operations such as those described herein. It should be understood that the instructions may be stored on a computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 1202 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the described processes and methods. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuit system."

[0258] It should be understood that decoders and / or encoders according to the present application can be implemented in multiple computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer-readable memory, including CD, RAM, ROM, flash memory, etc.

[0259] It should be understood that the decoder and / or encoder described herein and the module, routine, process, thread or other software components implementing the method / process for configuring the encoder or decoder can be implemented using standard computer programming techniques and languages. The application is not limited to specific processors, computer languages, computer programming conventions, data structures, other such implementation details. Those skilled in the art will recognize that the described process can be implemented as a part of a computer executable code stored in a volatile or non-volatile memory, a part of a dedicated integrated chip (Application Specific Integrated Chip, ASIC), etc.

[0260] The present application also provides a computer readable signal encoding data produced by applying the encoding process according to the present application.

[0261] Certain adjustments and modifications may be made to the described embodiments. Therefore, the above embodiments are to be considered as illustrative rather than restrictive. In particular, the embodiments may be freely combined with each other.

Claims

1. A method for encoding a three-dimensional 3D point cloud into a bitstream, preferably, the method is applied to an encoder, the geometry of the point cloud is defined by an octree structure, the octree structure comprises a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is segmented into sub-volumes step by step and contains points of the point cloud, wherein the volume is segmented into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each sub-volume indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, the method comprising: Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order; Entropy encoding the occupancy information of each node into a bitstream up to a depth k, where k>=d; entropy encoding trisoup information for each leaf node at depth m into a bitstream, wherein the trisoup information includes information about the presence and positions of vertices on edges of a volume associated with the leaf node; It is characterized in that the entropy coding of the occupancy information and the entropy coding of the trisoup information are based on the same ordering, and the depth m>k.

2. A method for decoding a three-dimensional 3D point cloud from a bitstream, preferably, the method is applied to a decoder, the geometry of the point cloud is defined by an octree structure, the octree structure comprises a plurality of nodes, the plurality of nodes have a parent-child relationship and represent the three-dimensional position of an object, the point cloud is located in a volume space of a three-dimensional coordinate system, the volume space is segmented into sub-volumes step by step and contains points of the point cloud, wherein the volume is segmented into a plurality of sub-volumes, each sub-volume is associated with one of the plurality of nodes of the octree structure, wherein occupancy information associated with each of the sub-volumes indicates whether the corresponding sub-volume contains at least one of the points of the point cloud, the method comprising: Acquire nodes with a depth of d that contain at least a portion of the point cloud along each axis X, Y, Z of the coordinate system in a lexicographical order; Entropy decoding the occupancy information of each node from the bitstream up to a depth k, where k>=d; entropy decoding trisoup information of each leaf node of depth m from the bitstream, wherein the trisoup information includes information about the existence and position of vertices on each edge of the volume associated with the leaf node; It is characterized in that the entropy decoding of the occupancy information and the entropy decoding of the trisoup information are based on the same ordering, and the depth m>k.

3. The method according to claim 1 or 2, wherein: The method further includes storing occupied child nodes with a depth of d+1 in lexicographic order.

4. The method according to any one of claims 1 to 3, wherein: The lexicographic ordering of nodes at depth d and / or the lexicographic ordering of child nodes at depth d+1 is a raster scan ordering.

5. The method according to any one of claims 2 to 4, wherein: When the occupancy information is decoded at the last level of depth, the trisoup information decoding is performed in parallel with the occupancy information decoding.

6. The method according to any one of claims 1 to 4, wherein: When encoding and decoding the occupancy information at the last level of depth, the trisoup information encoding and decoding are performed in parallel with the occupancy information encoding and decoding, and are delayed for a portion of leaf nodes.

7. The method according to any one of claims 1 to 6, wherein: The occupancy information and the trisoup information are encoded into or decoded from two independent parts of the bitstream, preferably, the trisoup information starts at an addressable bit or byte of the bitstream.

8. The method according to any one of claims 1 to 6, wherein: The occupancy information and the trisoup information are encoded into or decoded from two separate bit streams.

9. An encoder for encoding a three-dimensional 3D point cloud into a bit stream, the encoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to any one of claims 1 , 3 to 8 .

10. A decoder for decoding a three-dimensional 3D point cloud from a bitstream, the decoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to any one of claims 2 to 8.

11. A bit stream encoded by the method according to any one of claims 1, 3 to 8.

12. A computer-readable storage medium comprising instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 8.

Images