Encoding / decoding point positions of point clouds contained in cubic volumes

Abstract

A method and apparatus for encoding / decoding position information of points of a point cloud into / from a bit stream including at least one data unit, the points being contained in cubic volumes associated with leaf nodes of an octree structure, includes the steps of: encoding occupancy information of nodes of the octree structure, the occupancy information representing the presence of points of the point cloud contained in the cubic volume; and, for each current cubic volume containing at least one point of the point cloud, encoding Trisoup information representing the presence of vertices at edges of the current cubic volume and the positions of the vertices along the edges. According to the invention, the occupancy information of a leaf node and the Trisoup information of other leaf nodes are encoded into the data unit in an interleaved manner.

Description

Cross-Citation of Related Applications

[0001] This application claims priority to European Patent Application No. "22306441.1" filed September 28, 2022, the entire contents of which are incorporated herein by reference. [Technical Field]

[0002] This application relates generally to point cloud compression, and more particularly to a method and apparatus for encoding / decoding positions of points of a point cloud into / from a bitstream comprising at least one data unit. [Background technology]

[0003] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one embodiment of the present application, as described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to better understand all aspects of the present application.

[0004] Point clouds have recently gained attention as a format for representing 3D data. They vary in their ability to represent all types of physical objects or scenarios. Point clouds can be used for various purposes, such as cultural heritage / architecture, to scan objects like sculptures or buildings in 3D to share their spatial configuration without transmitting or accessing the objects. They are also a way to ensure the preservation of knowledge about objects that may be destroyed, for example, a temple destroyed in an earthquake. This type of point cloud is usually static, colored, and large.

[0005] Another use case is topography and cartography, where 3D representations allow maps to be not only flat but also include relief. Google Maps is currently a good example of a 3D map, but it uses meshes rather than point clouds. However, point clouds can also be an appropriate data format for 3D maps, and typically this type of point cloud is static, color, and large.

[0006] Virtual reality (VR), augmented reality (AR), and immersive worlds have become a hot topic recently, and many see them as the future of 2D tablet video. The basic idea is to immerse the viewer in the surrounding environment, whereas standard television only allows the viewer to see the virtual world in front of him / herself. There are several levels of immersion depending on the viewer's degree of freedom in the environment. Point clouds are a good candidate format for distributing VR / AR worlds.

[0007] The automotive industry, especially the foreseeable field of autonomous vehicles, is another area where point clouds can be used extensively. Autonomous vehicles should be able to "sense" their environment in order to make better driving decisions based on the detected presence and nature of objects in their nearest vicinity, as well as road layout.

[0008] A point cloud is a set of points located in three-dimensional (3D) space, optionally with additional values ​​attached to each point. These additional values ​​are commonly called attributes. Attributes may be, for example, a three-component color, a material property (e.g., reflectance), and / or a two-component normal vector to a surface associated with the point.

[0009] A point cloud is therefore a combination of geometry data (the positions of points in 3D space, usually expressed as 3D Cartesian coordinates x, y, z) and attributes.

[0010] Point clouds can be sensed by various types of devices, such as camera arrays, depth sensors, lasers (also known as light detection and ranging, or laser radar), radar, or can be computer-generated (e.g., in film post-production). Depending on the use case, for cartography, a point cloud may contain thousands to billions of points. The original representation of a point cloud requires a very large number of bits per point, at least a dozen bits per Cartesian coordinate x, y, or z, and optionally provides more bits for one or more attributes, e.g., three times the 10 bits for color.

[0011] For many applications, it is important to be able to distribute point clouds to end users or store them on servers while maintaining an acceptable (or preferably very good) quality of experience, consuming only a reasonable amount of bitrate or storage space. Efficient compression of these point clouds is a key point for the practical realization of many immersive world distribution chains.

[0012] For distribution and visualization by end users, e.g., in AR / VR glasses or any other 3D-enabled device, the compression may be lossy (e.g., in the case of video compression), but there are also use cases, such as medical applications or autonomous driving, that require lossless compression in order not to alter decisions obtained from subsequent analysis of the compressed and transmitted point cloud.

[0013] Until recently, the mass market had no solution to the problem of point cloud compression (also known as PCC), and there were no standardized point cloud decoders available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Video Image Experts Group or MPEG, started a work project on point cloud compression. This resulted in two standards: MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (also known as V-PCC) MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (also known as G-PCC)

[0014] The V-PCC coding method compresses point clouds by projecting them multiple times onto a 3D object to obtain 2D patches that are packaged into an image (or video in the case of processing dynamic point clouds). It compresses images and videos captured with existing image / video codecs, making the most of image and video solutions already in place. By its very nature, V-PCC is only efficient with dense and continuous point clouds, since image / video codecs cannot compress non-smooth patches, such as those obtained from projecting sparse geometry data sensed by laser radar.

[0015] The G-PCC coding method has two schemes for compressing sensed sparse geometry data.

[0016] The first approach is based on an occupancy tree, which is a tree of any type, such as an octree, quadtree, or binary tree, that represents point cloud geometry. Occupied nodes (i.e., nodes associated with cubic volumes containing at least one point of the point cloud) are split until a certain size is reached, and occupied leaf nodes provide the 3D locations of points, e.g., at the centers of these nodes. Occupancy information is carried by occupancy data (binary data, flags), which signal the occupancy status of each of a node's child nodes. Using neighborhood-based prediction techniques, dense point clouds can achieve high levels of compression for occupied data. Sparse point clouds can also be solved by directly encoding the locations of non-minimal-sized points within a node, halting tree construction if only isolated points exist within the node. This technique is called direct coding mode (DCM).

[0017] The second approach is based on a prediction tree, where each node represents the 3D position of a point and the parent / child relationships between nodes represent parent-to-child spatial prediction. This approach only solves sparse point clouds and offers the advantages of lower latency and easier decoding than occupancy trees. However, it offers only a slight improvement in compression performance over the first occupancy-based approach and increases encoding complexity because the encoder must intensively search for the best predictor (from a long list of potential predictors) to build the prediction tree.

[0018] In these two approaches, attribute encoding (decoding) is performed after geometry encoding (decoding) is completed, effectively resulting in two encodings (decodings). Therefore, low latency for combined geometry / attributes is achieved by using point cloud partitions that decompose the 3D space into independently coded subvolumes without prediction between subvolumes. Using many point cloud partitions severely impacts compression performance.

[0019] One important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. AR / VR point clouds are also typically localized and 2D, as they often represent the surface of an object. As such, AR / VR point clouds are highly connected (or dense), meaning that a single point is rarely isolated and has many neighbors.

[0020] A dense (or solid) point cloud represents a continuous surface with a certain resolution, without presenting any visual cavities in the surface, so that the cubic volumes (small cubes) associated with the points touch each other.

[0021] Such point clouds are typically used in AR / VR environments, where end users view them on devices such as televisions, smartphones, or headphones. They are either transmitted to the device or typically stored locally. Many AR / VR applications use moving point clouds that change over time, rather than static point clouds. As a result, point cloud data can be very large and must be compressed.

[0022] Currently, lossless compression of octree representations of geometry based on point cloud frames (time instances of a point cloud) can be achieved at just under 1 bit per point (1 bpp), which may be insufficient for real-time transmission, which can involve millions of points per point cloud frame with a maximum frame rate of 50 frames per second (fps), generating hundreds of megabits of data per second.

[0023] As such, lossy compression can be used, which typically requires maintaining acceptable visual quality while compressing enough to fit the bandwidth provided by the transmission channel. Most applications require ensuring real-time transmission of point cloud frames, which imposes complexity and latency limits on the compression scheme. In many applications, real-time transmission is possible at bit rates as low as 0.1 bpp (10 times more compression than lossless coding).

[0024] V-PCC achieves such low bit rates through lossy compression using a video codec that compresses two-dimensional frames (pictures) of portions of a point cloud projected onto a plane to obtain projection patches. The geometry of the point cloud frame is represented by these projection patches assembled into a frame. Each patch is a small local depth map. However, V-PCC is not general-purpose and is limited to narrow types of point clouds that do not exhibit locally complex geometry (e.g., trees or hair) because the resulting projection depth map is not smooth and cannot be effectively compressed by video codecs such as HEVC or VVC.

[0025] Pure 3D compression techniques can handle any type of point cloud. For dense point clouds, it remains an open question whether 3D compression techniques can compete with V-PCC (or any projection + image coding scheme). Standardization is still underway to provide an extension (modification) of the current G-PCC codec to provide competitive lossy compression, which will compress dense point clouds in the same way as V-PCC, while maintaining the versatility of G-PCC (being able to handle any type of point cloud: dense point clouds, sparse point clouds (e.g., point clouds captured by laser radar), 3D maps, etc.).

[0026] Essentially, there are three main ways to obtain a lossy proposal based on an octree representation of the point cloud, which is accomplished with the current G-PCC codec.

[0027] The first method involves downsampling the entire point cloud to a smaller resolution, losslessly encoding the downsampled point cloud, decoding the encoded downsampled point cloud, and then upsampling the decoded downsampled point cloud back to the original resolution. Many upsampling methods have been proposed, including super-resolution, artificial intelligence-based upsampling, and learning-based 3D post-processing. If the downsampling is not too extreme—for example, if the downsampling in each spatial direction does not exceed a factor of two—the first method can provide good peak signal-to-noise ratio (PSNR) results. However, even if the first method can provide good PSNR results, the visual quality of the upsampled point cloud is controversial and poorly controlled.

[0028] The second method is to have the encoder "adjust" the point cloud locally to reduce the bitrate required to code the octree: for example, by moving points in the point cloud a little to get occupancy information that is better predicted by neighboring nodes, and thus the modified octree can be losslessly coded at a lower bitrate. Unfortunately, this method only results in a small decrease in bitrate.

[0029] A third approach is the so-called TriSoup coding scheme, which is being considered by the ISO / IEC-based JTC1 / SC29 / WG7 standardization workgroup. The TriSoup coding scheme is the most promising of these three approaches, but needs some work before it can reach maturity.

[0030] The TriSoup coding scheme gets its name from "soup of triangles" because it uses a set of triangles to provide a local model of the geometry of a point cloud without explicitly providing connectivity information.

[0031] The point cloud geometry is represented in a three-dimensional coordinate system by a 3D tree (usually an octree) of lower resolutions down to a certain resolution, such as an NxNxN cubic volume (cube) associated with a leaf node, where n can be, for example, 4, 8, or 16. This tree is encoded using a lossless coding scheme (e.g., G-PCC). The tree itself does not go deep enough, and because the number of leaf nodes (cubic volumes) is small compared to the number of points in the point cloud, it does not require much bitrate. Next, within each NxNxN cubic volume, the point cloud geometry is modeled by a local model consisting of a set of triangles. As shown in Figure 1, the vertices of the triangles are coded along the edges of the cubic volume associated with the tree's leaf nodes. Four vertices (deep black dots) along the cube edges are shown. These four vertices are used to form two triangles, as described below.

[0032] The TriSoup coding scheme is not limited to a specific number of vertices located along the edges of a cubic volume, but a maximum of one vertex per edge is specified. The TriSoup coding scheme is not limited to a specific number of triangles within a specific cubic volume. Vertices are shared between cubic volumes that have a common edge to which the vertex belongs. This means that at most one vertex is coded per edge that belongs to at least one cubic volume. This ensures local model continuity through cubic volumes associated with leaf nodes of the tree.

[0033] Coding vertices on the edge of a cubic volume requires encoding vertex information for each vertex, which includes a vertex flag indicating whether the vertex is on the edge of the cubic volume and the position of the vertex along that edge (if present). The position of the vertex in the 3D coordinate system is then clearly derived from the position of the vertex along the edge and the positions of the start and end vertices of the cubic volume that defines the edge.

[0034] Vertex flags can be coded by a self-adaptive binary arithmetic codec that uses a specific context to code the vertex flags. Length N=2 s The positions of the vertices along the edges of can be coded with a single precision by squeezing s bits into the bitstream (bypassing / not entropy coding).

[0035] That is, coding point cloud geometry using the TriSoup coding scheme requires coding octree data (e.g., by G-PCC), occupancy information, and Trisoup information representing the presence of vertices on and positions along the edges of a cubic volume. The increase in coded data due to encoding Trisoup information is compensated for by an improvement in triangular point cloud reconstruction, as described below.

[0036] In the cubic volume associated with a leaf node of the tree, if there are at least three vertices on an edge of the cubic volume, then at least one triangle is constructed from the vertices. If there are more than three vertices on an edge of the cubic volume, then one or more triangles may be constructed.

[0037] There are various construction processes for constructing such triangles.

[0038] The current TriSoup coding scheme constructs triangles by first determining a dominant direction along one of the three axes of a three-dimensional coordinate system representing the cubic volume, then ordering at least three vertices based on the determined dominant direction, and finally constructing at least one triangle based on the list of at least three ordered vertices.

[0039] The leading direction is determined by testing each of the three directions along the three axes, and the direction that maximizes the total surface of the triangle (viewed along the test direction) is retained as the leading direction.

[0040] 2 and 3 show illustrative examples of the process for determining the leading direction based on the current TriSoup.

[0041] For simplicity of illustration, only two axis tests are shown in Figures 2 and 3.

[0042] FIG. 2 shows an illustrative example of a cubic volume and four vertices (lower right) projected along a vertical axis onto a two-dimensional plane (upper right). These four vertices are then ordered in a clockwise direction relative to the center of the projected cubic volume (square). Based on the ordered vertices, two triangles (upper left) are constructed according to a fixed rule. Here, the fixed rule is that when four vertices are involved, triangles 123 and 134 are constructed. With three vertices, a uniquely possible triangle 123 can be constructed. With five vertices, triangles 123, 134, and 451 can be constructed. This continues up to 12 vertices. Other fixed rules for constructing triangles from vertices may also be used.

[0043] Figure 3 shows an illustrative example of a cubic volume and its four vertices (lower right) projected along the horizontal axis onto a two-dimensional plane (lower left). These four vertices are then ordered in clockwise order relative to the center of the projected cubic volume (square). Based on the ordered vertices, two triangles (upper left) are constructed according to the fixed rules above.

[0044] In the examples of Figures 2 and 3, the vertical projection (Figure 2) shows the (largest) total 2D projected surface area of ​​the triangle in the 2D plane (larger than the total 2D projected surface area of ​​the triangle in the 2D plane by the horizontal projection, Figure 3). Therefore, the vertical direction is chosen as the leading direction, and two triangles are constructed based on the vertices generated by the vertical projection, ordered by their vertex numbers.

[0045] By selecting the leading direction by maximizing the total 2D projection surface of the triangle in the 2D plane, a hole-free point cloud can be continuously reconstructed.

[0046] The current TriSoup coding scheme performs a ray tracing process on each cubic volume of the tree, rendering (modeling) the triangles defined within each cubic volume as rendering points that represent the points of the point cloud contained in that cubic volume. The positions of all rendering points obtained by ray tracing become the positions of decoded points in the point cloud. This process is called "voxelizing" the triangles.

[0047] As shown in Figure 4, ray tracing performed on a triangle of a cubic volume essentially involves emitting rays along three directions parallel to the axes of the 3D coordinate system. The origin of the emitted rays is a point with integer (voxelized) coordinates whose precision corresponds to the sampling precision required for rendering. The intersection of the ray with the triangle (indicated by a dotted point, if any) is then voxelized (= rounded to the nearest point with the required sampling) and added to the list of rendering points.

[0048] After applying the TriSoup coding scheme to the cubic volumes associated with all leaf nodes of the tree representing the point cloud geometry (i.e., after constructing triangles for each cubic volume of the cubic volume and obtaining rendering points by ray tracing), we discard copies of the same rendering point in all rendering point lists (i.e., keep only one voxel (rendering point) among all voxels sharing the same 3D location) to obtain a set of decoded (unique) points of the point cloud.

[0049] When more than four vertices are involved, constructing triangles according to fixed rules based on ordered vertices can lead to visually awkward results. In fact, for a given set of vertices, it is possible to construct other triangles that are different from those constructed by the above fixed rules, resulting in a different set of rendering points (a different visual effect of decoded points in the point cloud). Also, PSNR (Peak Signal-to-Noise Ratio) results show that reconstructing triangles based on the above fixed rules is not optimal.

[0050] The occupancy information and Trisoup information of a point cloud data partition may be carried by the bitstream in multiple independent data units of the bitstream or in two independent parts of the same data unit DU. After encoding the bitstream, the compressed point cloud data is transmitted via a transmission system (e.g., the Internet). If the transmission system has only one transmission channel, the complete bitstream of the entire data unit carrying the octree information of the octree structure (including all information required to reconstruct the octree structure, including occupancy information) is transmitted before transmitting the data unit (or part of the same data unit) carrying the Trisoup information of the leaf nodes of the octree structure. Because the octree information of a point cloud data partition is large, latency is introduced in the transmission of the encoded point cloud geometry data, affecting use cases where it is used in real time (e.g., autonomous vehicles). One simple solution is to use two transmission channels in the transmission system. One channel can be used to transmit occupancy information and the other channel can be used to transmit Trisoup information. This allows occupancy information and Trisoup information to be transmitted simultaneously, avoiding the introduction of transmission latency. However, this can lead to underutilization of the channel capacity and in some cases to wasted capacity. Also, there are many use cases where a single channel is used.

[0051] The problem to be solved is to reduce the bitstream transmission latency for transmitting occupancy information and Trisoup information when the occupancy information and Trisoup information of an octree structure are transmitted over a single transmission channel.

[0052] It is with the above in mind that at least one embodiment of the present application is designed. Summary of the Invention [Problem to be solved by the invention]

[0053] The following section presents a simplified summary of at least one embodiment in order to provide a basic understanding of some aspects of the present application. This summary is not a detailed overview of the embodiment. It is not intended to identify key or critical elements of the embodiment. The following summary presents only some aspects of at least one embodiment in a simplified form as a prelude to the more detailed description provided elsewhere in this document. [Means for solving the problem]

[0054] According to a first aspect of the present application, there is provided a method for encoding position information of points of a point cloud into a bitstream including at least one data unit, the points being contained in cubic volumes associated with leaf nodes of an octree structure having a maximum depth d, at least three vertices being located on each edge of the cubic volume, each edge having at most one vertex, and the position information of the points contained in the cubic volume being represented by a triangle connecting the at least three vertices, the method comprising the steps of: encoding occupancy information of nodes of the octree structure, the occupancy information representing the presence of a point of the point cloud contained in the cubic volume; and encoding trisoup information for each current cubic volume containing at least one point of the point cloud, the trisoup information representing the presence of a vertex at an edge of the current cubic volume and the position of the vertex along the edge, the occupancy information of a leaf node and the trisoup information of other leaf nodes being encoded in the data unit (DU) in an interleaved manner.

[0055] According to a second aspect of the present application, there is provided a method for decoding position information of points of a point cloud from a bitstream including at least one data unit, the points being included in cubic volumes associated with leaf nodes of an octree structure having a maximum depth d, at least three vertices being located on each edge of the cubic volume, each edge having at most one vertex, and the position information of points included in the cubic volume being represented by a triangle connecting the at least three vertices, the method comprising: decoding occupancy information of nodes of the octree structure, the occupancy information representing a presence of a point of the point cloud included in the cubic volume; and decoding Trisoup information for each current cubic volume including at least one point of the point cloud, the Trisoup information representing a presence of a vertex at an edge of the current cubic volume and a position of the vertex along the edge, the occupancy information of a leaf node and the Trisoup information of other leaf nodes being coded in the data unit (DU) in an interleaved manner.

[0056] In one exemplary embodiment, the method further includes a first list storing nodes of depth d-1 ordered according to an encoding order or a decoding order based on the coordinates of the nodes in a three-dimensional system, and encoding / decoding occupancy information of p child nodes of depth d of the nodes in the first list into / from a series of bits BSoct1 of the data unit.

[0057] In an exemplary embodiment, the method further includes a step of encoding / decoding Trisoup information into / from a series of bits BStris1 of the data unit for a first child node of a first node in the first list.

[0058] In one exemplary embodiment, the method further includes the step of alternately encoding / decoding occupancy information into / from a series of bits BSoct2 of the data unit and encoding / decoding Trisoup information into / from a series of bits BStris2 of the data unit for child nodes of nodes in the first list until encoding / decoding occupancy information of the last child node of the node in the first list.

[0059] In one exemplary embodiment, the method further comprises the step of encoding / decoding Trisoup information of the remaining child nodes in a series of bits BStris3 added to the data unit.

[0060] In one exemplary embodiment, the method further includes obtaining a second list, the second list storing child nodes of a node at a depth d-1, the nodes having the same coordinates in the three-dimensional system, the nodes being ordered according to a first encoding order or a first decoding order, and the child nodes being ordered according to a second encoding order or a second decoding order.

[0061] In an exemplary embodiment, the method further includes the steps of encoding / decoding Trisoup information for a first child node of a node of the second list in a series of bits BStris1 of the data unit, and encoding / decoding octree information of a first child node of a node having the same updated coordinates in a series of bits BSoct1 of the data unit, the same updated coordinates being equal to the same coordinates plus 1.

[0062] In one exemplary embodiment, the method further includes the step of encoding / decoding the Trisoup information of the next child node of the node having the same updated coordinates into / from the series of bits BStris2 of the data unit, in an interleaved manner, until the encoding / decoding of the occupancy information of the last child node of the node in the first list, and encoding / decoding the occupancy information of the next child node into / from the series of bits BStris2 of the data unit.

[0063] In an exemplary embodiment, the method further includes a step of encoding / decoding Trisoup information of the remaining child nodes of the second list into / from a series of bits BStris3 of the data unit (DU).

[0064] In an exemplary embodiment, the encoding order and / or the decoding order and / or the first encoding order and / or the first decoding order and / or the second encoding order and / or the second decoding order is a Morton order or a raster scan order.

[0065] According to a third aspect of the present application, there is provided a bitstream formatted to include encoded point cloud geometry data obtained by the method according to the first aspect of the present application.

[0066] According to a fourth aspect of the present application, there is provided an apparatus comprising means for carrying out one of the methods according to the first or second aspect of the present application.

[0067] According to a fifth aspect of the present application, there is provided a computer program product including instructions, which when executed by one or more processors, cause the one or more processors to perform the method of the first or second aspect of the present application.

[0068] According to a sixth aspect of the present application, there is provided a non-transitory storage medium carrying program code instructions for carrying out the method of the first or second aspect of the present application.

[0069] The specific nature of at least one exemplary embodiment and other objects, advantages, features and applications of the at least one exemplary embodiment will become apparent from the following description of the embodiments taken in conjunction with the drawings. [Brief explanation of the drawings]

[0070] Reference will now be made, by way of example, to the drawings illustrating exemplary embodiments of the present application. [Figure 1] Schematic of a TriSoup cubic volume containing four vertices. [Figure 2] FIG. 10 illustrates an illustrative example of a process for determining a leading direction based on a current TriSoup. [Figure 3] FIG. 10 illustrates an illustrative example of a process for determining a leading direction based on a current TriSoup. [Figure 4] FIG. 2 is a diagram illustrating a schematic example of ray tracing for rendering points of a point cloud, according to an example embodiment; [Figure 5] 7 is a diagram illustrating a data unit structure of bitstream B generated by the encoding method of FIG. 6 according to an exemplary embodiment. [Figure 6] 3 is a block diagram that schematically illustrates a method 300 for encoding position information of points of a point cloud contained in a cubic volume associated with a leaf node of an octree structure into a bitstream, according to an example embodiment. [Figure 7]4 is a block diagram that schematically illustrates a method 400 for decoding, from a bitstream, position information of points of a point cloud contained in a cubic volume associated with a leaf node of an octree structure, according to an example embodiment. [Figure 8] 3 is a block diagram that schematically illustrates a variation of the method 300. FIG. [Figure 9] FIG. 4 is a block diagram that schematically illustrates a variation of the method 400. [Figure 10] FIG. 10 illustrates an example of a raster scan order for a complete octree structure within a global cube containing all points of a point cloud. [Figure 11] FIG. 10 illustrates an example of delay introduced by p child nodes according to an example embodiment. [Figure 12] FIG. 10 illustrates an example of delay introduced by p child nodes according to an example embodiment. [Figure 13] 1 is a block diagram illustrating an example of a system for implementing aspects and exemplary embodiments.

[0071] In different figures, similar reference numerals may be used to represent similar components. DETAILED DESCRIPTION OF THE INVENTION

[0072] At least one example of an embodiment will now be described more fully with reference to the drawings, at least one of which is illustrated. However, the embodiments may be implemented in many alternative forms and should not be construed as being limited to the examples set forth herein. Accordingly, it should be understood that the embodiments are not intended to be limited to the particular forms disclosed. Rather, the present application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present application.

[0073] At least on the one hand, it relates generally to point cloud encoding and decoding, on the other hand, it relates generally to the transmission of generated or encoded bitstreams, and on the other hand, it relates generally to the reception / access of decoded bitstreams.

[0074] Additionally, the present aspects are not limited to MPEG standards such as MPEG-I Part 5 or Part 9 related to point cloud compression, etc., but may be applied, for example, to other standards and recommendations, whether existing or developed in the future, and extensions of such standards and recommendations, including MPEG-I Part 5 and Part 9. Unless otherwise indicated or technically excluded, the aspects described herein may be used alone or in combination.

[0075] Briefly, the present invention provides a method for encoding / decoding position information of points of a point cloud into / from a bitstream containing at least one data unit. The points are contained in cubic volumes associated with leaf nodes of an octree structure having a maximum depth d. At least three vertices are located on each edge of the cubic volume, each edge having at most one vertex, and the position information of the points contained in the cubic volume is represented by a triangle connecting the at least three vertices. The method includes the steps of: encoding occupancy information of nodes of the octree structure, the occupancy information representing the presence of a point of the point cloud contained in the cubic volume; and encoding trisoup information for each current cubic volume containing at least one point of the point cloud, the trisoup information representing the presence of a vertex on an edge of the current cubic volume and the position of the vertex along the edge. According to the present invention, the occupancy information of a leaf node and the trisoup information of other leaf nodes are encoded in the data unit in an interleaved manner.

[0076] The present invention is advantageous because, when a single transmission channel is used, the transmission of Trisoup information does not need to wait for the transmission of occupancy information for all nodes in the octree structure before transmitting the Trisoup information for a leaf node. Thus, the present invention reduces the latency between the encoder and decoder compared to the prior art, allowing the bitstream generated by the encoding method of the present invention to be transmitted as quickly as possible.

[0077] FIG. 5 is a diagram illustrating a data unit structure of bitstream B generated by the encoding method of FIG. 6 according to an exemplary embodiment.

[0078] Assume that the octree structure of the point cloud has a depth of (d+1), starting from the depth of layer 0 (the root node of the octree structure) and ending at the depth of layer d (the final depth). The leaf node is at the final depth d of the octree structure, and the parent node of the child node is at depth (d-1).

[0079] The bitstream B includes multiple data units DU, e.g., one data unit DU for each point cloud data partition. Each data unit DU is dedicated to a point cloud data partition, and the point cloud data can be divided into multiple point cloud data partitions.

[0080] The DU structure in FIG. 5 includes a DU header, which is usually formed of two bytes, and a raw byte sequence payload (RBSP), which is formed of a series of bits.

[0081] The DU header provides information about the type of information carried by the RBSP.

[0082] The RBSP includes one series of bits BSoct1 and at least one series of bits BSoct2 representing the occupancy information of nodes of the octree structure, the nodes being located at a depth strictly less than or equal to depth d.

[0083] The RBSP further includes a sequence of bits BStris1, at least one sequence of bits BStris2, and a third sequence of bits BStris3, which represent the Trisoup information of the leaf nodes of the octree structure. According to the present invention, the sequences of bits BSoct2 and BStris2 are interleaved in the data unit DU.

[0084] FIG. 6 is a block diagram that schematically illustrates a method 300 for encoding position information of points of a point cloud contained in a cubic volume associated with a leaf node of an octree structure into a bitstream, according to an example embodiment.

[0085] Essentially, method 300 encodes occupancy information for nodes of an octree structure, said occupancy information representing the presence of points of a point cloud contained in a cubic volume associated with the node of the octree structure. For each current cubic volume that contains at least one point of the point cloud, method 300 includes encoding Trisoup information representing the presence of vertices on edges of the current cubic volume and the positions of the vertices along said edges.

[0086] In step 310, the method 300 encodes the occupancy information of the nodes at the first d layer depths (depth 0 to depth d-1) into a series of bits BSoct1.

[0087] In step 320, the list L d-1 Get the list L d-1 stores the nodes at depth (d-1) ordered according to the coding order based on the coordinates of these nodes in the three-dimensional system.

[0088] In step 330, one first child node index m is initialized to encode occupancy information, and one second child node index k is initialized to encode Trisoup information, for example, m=k=0.

[0089] In step 340, list L d-1The occupancy information of p (integer) child nodes of depth d (i.e., the occupancy information of p leaf nodes) of node is coded into a series of bits BSoct1, starting from the first child node (m=0) and ending with the pth child node (m=p-1). The child nodes are considered in coding order, where k=0, m=p-1.

[0090] Encoding the occupancy information of p child nodes before encoding the Trisoup information of the leaf node introduces a small delay between occupancy information encoding and Trisoup information encoding. This delay is unavoidable because encoding the Trisoup information of the leaf node requires encoding the occupancy information of adjacent nodes (here, p child nodes) in advance. The reason why Trisoup encoding of a node requires occupancy information of adjacent nodes (p child nodes) is due to the characteristics of the Trisoup encoding of each child node.

[0091] In step 350, the list (L d-1 For the first child node (k=0) of the first node in k = 1, encode the Trisoup information into a series of bits BStris1.

[0092] In step 360, list L d-1 For the child nodes of the node in L d-1 The coding of the occupancy information of the series of bits BSoct2 and the coding of the Trisoup information of the series of bits BStris2 are performed in an interleaved manner (i.e., alternately) until the coding of the occupancy of the last child node (m=Nmax) of the last node of the last node, where m=Nmax and k=Nmax-p.

[0093] In sub-step 361 of step 360, the first child index m is incremented by 1 and the second child index m is also incremented by 1 to form the list L d-1 The octree information and Trisoup information of the next child node of the node are encoded.

[0094] Note that the child node index for occupancy information encoding is different from the child node index for Trisoup information encoding, and there is a small latency (delay) between these two child node indexes.

[0095] In substep 362, the occupancy information of the mth child node is encoded into a series of bits BSoct2, and in substep 363, the Trisoup information of the kth child node is encoded into a series of bits BStris2. As shown in Figure 5, the series of bits BSoct2 and BStris2 are alternately added to the data unit, i.e., the occupancy information of the mth child node and the Trisoup information of the kth child node are encoded in an interleaved manner.

[0096] Sub-steps 361-363 are repeated, and in each iteration, a series of bits BSoct2 and a series of bits BStris2 are added to the data unit DU in an interleaved manner.

[0097] In step 370, the Trisoup information of the remaining child nodes (p=Nmax-k) is encoded into a series of bits BStris3.

[0098] A data unit DU ends with a series of bits (eg 10000).

[0099] FIG. 7 is a block diagram that schematically illustrates a method 400 for decoding, from a bitstream, location information of points of a point cloud contained in a cubic volume associated with a leaf node of an octree structure, according to an example embodiment.

[0100] Essentially, method 400 decodes occupancy information of nodes of an octree structure, said occupancy information representing the presence of points of a point cloud contained in a cubic volume associated with the node of the octree structure. For each current cubic volume that contains at least one point of the point cloud, method 400 includes decoding Trisoup information representing the presence of vertices at edges of the current cubic volume and the positions of the vertices along said edges.

[0101] After obtaining the data unit DU of the bitstream B relating to the information of the partition of the point cloud data, in step 410, the method 400 decodes the occupancy information of the nodes at the depth of the first d layers (depth 0 to depth d-1) from the series of bits BSoct1.

[0102] In step 420, list L d-1 Get the list L d-1 stores the nodes at depth (d-1) ordered according to the decoding order based on the coordinates of these nodes in the three-dimensional system.

[0103] In step 430, one first child node index m is initialized for decoding the occupancy information, and one second child node index k is initialized for decoding the Trisoup information, for example, m=k=0.

[0104] In step 440, the series of bits BSoct1 of the data unit DU is extracted from the list L d-1 Decode the occupancy information of p child nodes of depth d (i.e., the occupancy information of p leaf nodes) of node m, starting from the first child node (m=0) and ending with the pth child node (m=p-1), where k=0 and m=p-1.

[0105] In step 450, the list L d-1 For the first child node (k=0) of the first node in, decode the Trisoup information from the series of bits BStris1 of the data unit DU.

[0106] In step 460, the list L d-1 For the child nodes of the node in, the decoding of occupancy information from the sequence of bits BSoct2 of the data unit DU and the decoding of Trisoup information from the sequence of bits BStris2 of the data unit DU are performed in an interleaved manner (i.e., alternately), d-1The above interleaved decoding is repeated until the occupancy information of the last child node (m=Nmax) of the last node is decoded, where m=Nmax and k=Nmax-p.

[0107] In sub-step 461 of step 460, the first child index m is incremented by 1 and the second child index m is also incremented by 1 to form the list L d-1 The octree information and Trisoup information of the next child node of the node are decoded.

[0108] In sub-step 462, the occupancy information of the mth child node is decoded from a series of bits BSoct2 of the data unit DU, and in sub-step 463, the Trisoup information of the kth child node is decoded from a second series of bits BStris2 of the data unit DU. As shown in Figure 5, the series of bits BSoct2 and BStris2 can be decoded alternately from the data unit DU, i.e., the occupancy information of the mth child node and the Trisoup information of the kth child node can be decoded in an interleaved manner.

[0109] Sub-steps 461-463 are repeated, and in each iteration the sequence of bits BSoct2 and the sequence of bits BStris2 are considered in an interleaved manner in the data unit DU.

[0110] In step 470, the Trisoup information of the remaining child nodes (p=Nmax-k) is decoded from the series of bits BStris3 of the data unit DU.

[0111] A data unit DU ends with a series of bits (eg 10000).

[0112] In one exemplary embodiment, by taking advantage of the properties of a linear memory structure (i.e., each element in memory holds information about how to locate the next and previous elements), the iterations of steps 361-363 and 461-463 can directly locate the next child node in a linear memory structure (e.g., a list).

[0113] This example embodiment is advantageous because it avoids using the child node indices m and k to locate and then iterate over each child node.

[0114] In one exemplary embodiment, the encoding / decoding of the occupancy information of a leaf node and the encoding / decoding of the Trisoup information of said leaf node are performed in parallel.

[0115] This embodiment reduces the latency between the encoding / decoding of occupancy information and the encoding / decoding of Trisoup.

[0116] In one exemplary embodiment, the p child nodes in steps 340 and 440 include a child node of one point cloud slice (a slice consists of a cubic volume with the same coordinates), a child node of a tube, and a child node located at the first child node of another tube in another point cloud slice. The number p is: p = N slice +N tube +1, However, N slice represents the number of child nodes in one point cloud slice, and N tube represents the number of child nodes in one tube.

[0117] Figure 11 shows an example of 43 child nodes, including 36 child nodes of the point cloud slice PCS (gray cubic volume), 6 child nodes of the tube TU, and one child node CN located at the first child node of another tube in another point cloud slice.

[0118] In one exemplary embodiment, the p child nodes in steps 540 and 640 include the child nodes of the node, and these nodes are organized in a list L, as shown in FIG. d-1As shown in Fig. 12, the light gray cubes represent the child nodes belonging to the node with x=0, the central gray cubes represent the child nodes belonging to the node with x=1, the cube CU represents the child nodes of the currently performed Trisoup coding, and the dark gray cubes represent the child nodes of the currently performed Octree coding, so the delay is two slices of the child nodes of the nodes with the same x.

[0119] In the exemplary embodiment of the method 300 and the method 400 of FIGS. 6 and 7, the coding of the Trisoup information is performed by using a list L for coding the occupancy information. d-1 The same list as L d-1 Using the list L d-1 Iterate over the child nodes of the node in

[0120] In one variant, the node information of the child nodes whose occupancy information has been coded / decoded is stored in a list (buffer) L tris are stored in the list L tris The Trisoup information of the child nodes can be used later for encoding / decoding by iterating over the child nodes of the node in

[0121] FIG. 8 is a block diagram that schematically illustrates a variation of the method 300.

[0122] In step 510, the method 300 encodes the occupancy information of the nodes at the first d layer depths (depth 0 to depth d-1) into a first series of bits BSoct1.

[0123] In step 520, list L d-1 Get the list L d-1 stores the nodes at depth (d-1) ordered according to the coding order based on the coordinates of these nodes in the three-dimensional system.

[0124] In step 530, initialize one child node index k for encoding Trisoup information, for example, k=0.

[0125] In step 540, nodes at depth (d-1) and with the same coordinate in the three-dimensional system are considered, for example, nodes at depth (d-1) and with the same x-coordinate (e.g., x=0). These nodes are ordered according to a first encoding order, and their child nodes (at depth d) are ordered according to a second encoding order. The occupancy information of these child nodes is coded into a series of bits BSoct1 (Fig. 5). The child nodes of the nodes with the same coordinate are then classified into a list L tris These child nodes are added to the list L tris are ordered according to the second encoding order in

[0126] In step 550, list L tris For the first child node (k=0) of node d, the Trisoup information is encoded as a series of bits BStris1. On the other hand, the octree information of the first child node of the node with the same coordinate x=1 (the coordinate x of the node at depth d-1 is added by 1) is encoded as a series of bits BSoct2.

[0127] In step 560, the Trisoup information of the next child node of the node with the same updated coordinate x=1 is encoded into a second series of bits BStris2, and the list L tris The occupancy information of the next child node (k=k+1) of the list L is encoded into a series of bits BSoct2 in an interleaved manner. d-1 The above interleaved encoding is repeated until the occupancy information of the last child node (Nmax-th child node) of the node in is encoded.

[0128] At each iteration, when the encoding of the occupancy information of the child node of the node having the same coordinate is completed, the coordinate is incremented (updated) by 1, and the occupancy information of the child node of the node having the updated coordinate is encoded.

[0129] In step 570, list L tris The Trisoup information of the remaining child nodes (Nmax-k) of is encoded into a series of bits BStris3.

[0130] The data unit DU ends with a series of bits (10000).

[0131] FIG. 9 is a block diagram that schematically illustrates a variation of the method 400.

[0132] After the data unit DU of the bitstream B relating to the information of the partition of the point cloud data is obtained, in step 610, the method 400 decodes the occupancy information of the nodes at the depth of the first d layers (depth 0 to depth d-1) from the series of bits BSoct1.

[0133] In step 620, list L d-1 Get the list L d-1 stores the nodes at depth (d-1) ordered according to the decoding order based on the coordinates of these nodes in the three-dimensional system.

[0134] In step 630, initialize one child node index k for decoding Trisoup information, for example, k=0.

[0135] In step 640, nodes at depth (d-1) and with the same coordinate in the three-dimensional system are considered, for example, nodes at depth (d-1) and with the same x-coordinate (e.g., x=0). These nodes are ordered according to the first decoding order, and their child nodes (at depth d) are ordered according to the second decoding order. The occupancy information of these child nodes is decoded from the series of bits BSoct1 (Fig. 5). The child nodes of the nodes with the same x-coordinate are listed as L. tris These child nodes are added to the list L tris are ordered according to the second decoding order in

[0136] In step 650, list L trisFor the first child node (k=0) of node x, the Trisoup information is decoded from the set of bits BStris1, while the octree information of the first child node of the node with the same updated coordinate x=1 is decoded from the set of bits BSoct2.

[0137] In step 660, list L d-1 Until the occupancy information of the last child node (Nmax-th) of the node in is decoded, the decoding of the Trisoup information of the next child node of the node with the same updated coordinate x=1 from the series of bits BStris2 of the data unit DU and the decoding of the occupancy information of the next child node (k=k+1) from the series of bits BSoct2 of the data unit DU are performed in an interleaved manner.

[0138] At each iteration, when the decoding of the occupancy information of the node with the same coordinate is completed, the coordinate is incremented (updated) by 1, and the occupancy information of the node with the updated coordinate is decoded.

[0139] In step 670, the set of bits BStris3 is extracted from the list L tris The remaining child nodes (Nmax-k ) Decrypt the Trisoup information.

[0140] A data unit DU ends with a series of bits (eg 10000).

[0141] In one exemplary embodiment, the occupancy information and / or Trisoup information is entropy coded / decoded.

[0142] In one exemplary embodiment, the encoding order and / or decoding order is Morton order or raster scan order.

[0143] In an exemplary embodiment, the first encoding order and / or the first decoding order and / or the second encoding order and / or the second decoding order is a Morton order or a raster scan order.

[0144] The last two illustrative embodiments involve the encoding order and decoding order being the same.

[0145] FIG. 10 shows an example of a raster scan order for a complete octree structure within a global cube that contains all points of the point cloud.

[0146] A cube containing all points of the point cloud is divided into (multiple) cubic volumes. A point cloud slice s is formed by cubic volumes that have the same coordinate (e.g., y-coordinate) in a three-dimensional coordinate system (x, y, z). Each point cloud slice s is divided into tubes, defined as a set of cubic volumes that have the same coordinate (e.g., x). Scanning the nodes of the octree structure according to the raster scan order involves individually scanning each tube of each point cloud slice s. Raster scanning a point cloud slice involves: considering the first cubic volume cs of the first tube T1 of the point cloud slice, and then considering subsequent cubic volumes of tube T1 up to the last cubic volume ce; then considering the first cubic volume of another tube T2 of the point cloud slice, and considering subsequent cubic volumes of tube T2 up to the last cubic volume of tube T2, and so on.

[0147] FIG. 13 illustrates an example block diagram of a system for implementing various aspects and example embodiments.

[0148] System 500 may be implemented as one or more devices and may include various components as described below. In various embodiments, system 500 may be configured to implement one or more aspects described herein.

[0149] Examples of devices that may comprise all or part of system 500 include personal computers, laptop computers, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected cars and their associated processing systems, head-mounted displays (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from video decoders, pre-processors that provide input to video encoders, web servers, set-top boxes, and any other devices for processing point clouds, videos, or images, or other communications devices. Elements of system 500 may be implemented singly or in combination on a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 500 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 500 may be communicatively coupled to other similar systems or other electronic devices, for example, via a communications bus or dedicated input and / or output ports.

[0150] The system 500 includes at least one processor 510, configured to execute instructions loaded therein to implement, for example, aspects described herein. The processor 510 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 500 may include at least one memory 520 (e.g., a volatile memory device and / or a non-volatile memory device). The system 500 may include a storage device 540, including non-volatile and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, magnetic disk drives, and / or optical disk drives. By way of non-limiting example, the storage device 540 may include an internal storage device, an additional storage device, and / or a network-accessible storage device.

[0151] The system 500 may include an encoder / decoder module 530 configured to process data to provide encoded / decoded point cloud geometry shape data, for example, and the encoder / decoder module 530 may include its own processor and memory. The encoder / decoder module 530 may represent a module or modules included in a device to perform encoding and / or decoding functions. As is known, a device may include either one or both of an encoding and decoding module. Additionally, the encoder / decoder module 530 may be implemented as a separate element of the system 500 or may be coupled within the processor 510 as a combination of hardware and software known to those skilled in the art.

[0152] Program code loaded into the processor 510 or the encoder / decoder 530 to perform aspects described herein may be stored in the storage device 540 and subsequently loaded into the memory 520 and executed by the processor 510. According to various embodiments, during execution of the processes described herein, one or more of the processor 510, the memory 520, the storage device 540, and the encoder / decoder module 530 may store one or more of the following items: point cloud frames, encoded / decoded geometry / attribute videos / images or portions of encoded / decoded geometry / attribute videos / images, bitstreams, matrices, variables, and equations, formulas, logic for operations, and intermediate or final results of operations.

[0153] In some embodiments, memory within the processor 510 and / or encoder / decoder module 530 may be used to store instructions and provide working memory for processes performed during encoding or decoding.

[0154] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 510 or the encoder / decoder module 530) is used for one or more of these functions. The external memory may be memory 520 and / or storage device 540, such as dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, fast external dynamic volatile memory, such as RAM, may be utilized as working memory for video encoding / decoding and decoding operations, such as those directed to MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, and also referred to as MPEG-2 Video), HEVC (High Efficiency Video Coding / Decoding), VVC (Versatile Video Coding / Decoding), or MPEG-I Parts 5 or 9.

[0155] As shown in block 590, inputs can be provided to elements of system 500 via various input devices, including, but not limited to, (i) an RF section capable of receiving RF signals transmitted wirelessly, such as from a broadcast station, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0156] In various embodiments, the input devices of block 590 have associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with each of the following required elements: (i) selecting a desired frequency (also called signal selection, or limiting the signal to a frequency band); (ii) downconverting the selected signal; (iii) selecting a signal frequency band by controlling the frequency band back to a narrower frequency band, which in some embodiments is called a channel (for example); (iv) demodulating the downconverted and frequency-band-limited signals; (v) performing error correction; and (vi) demultiplexing to select a desired data packet flow. The RF section of various embodiments purposely includes elements that perform these functions, such as a frequency selector, a signal selector, a frequency band limiter, a channel selector, a filter, a downconverter, a demodulator, an error correction device, and a demultiplexer. The RF section may also include a tuner that performs each of these functions, including, for example, downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband.

[0157] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium, after which the RF section can perform frequency selection by filtering, downconverting, and re-filtering to obtain a desired frequency band.

[0158] In various embodiments, the order of these (and other) elements may be rearranged, some of these elements may be omitted, and / or other elements that perform similar or different functions may be added.

[0159] Adding elements can include inserting elements such as amplifiers and analog-to-digital converters between existing elements. In various embodiments, the RF section can include an antenna.

[0160] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 500 to other electronic devices via USB and / or HDMI connections. It should be understood that, when desired, aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or within processor 510. Thus, it should be understood that, when desired, aspects of USB or HDMI interface processing may be implemented in a separate interface IC or within processor 510. Upon demodulation, the error corrected and demultiplexed stream may be provided to various processing elements, including processor 510 and encoder / decoder 530 operating in conjunction with memory and storage elements, to process the data stream when desired for display on an output device.

[0161] The various elements of system 500 may be provided within a unitary housing, within which a suitable connection layout 590 (e.g., internal buses known in the art, including I2C buses, wires, and printed circuit boards) may be used to connect the elements to one another and transmit data therebetween.

[0162] System 500 may include a communication interface 550 such that it can communicate with other devices over a communication channel 900. Communication interface 550 includes, but is not limited to, a transceiver configured to transmit and receive data over communication channel 900. Communication interface 550 may include, but is not limited to, a modem or a network card, and communication channel 900 may be implemented within a wired and / or wireless medium, for example.

[0163] In various embodiments, a Wi-Fi network, such as IEEE 802.11, can be used to stream data to system 500. The Wi-Fi signal in these embodiments can be received via communication channel 900 suitable for Wi-Fi communication and communication interface 550. Communication channel 900 in these embodiments can typically be connected to an access point or router that provides access to external networks, including the Internet, allowing streaming applications and other over-the-top wireless communications.

[0164] Another embodiment may provide streaming data to system 500 using a set-top box, which carries the data through an HDMI connection in input block 590.

[0165] In some embodiments, the RF connection of input block 590 is used to provide streaming data to system 500 .

[0166] Streaming data can be used as a form of signaling information used by system 500. The signaling information can include information on the number of points, coordinates, and / or sensor configuration parameters, such as bitstream B and / or point clouds.

[0167] It should be noted that signaling can be achieved in various manners, for example, in various embodiments, one or more syntax elements, flags, etc. can be used to send signaling information to corresponding decoders.

[0168] System 500 can provide output signals to a variety of output devices, including a display 600, speakers 700, and other peripheral devices 800. In various example embodiments, other peripheral devices 800 can include one or more of a separate DVR, disc player, stereo system, lighting system, and other devices that provide functionality based on the output of system 500.

[0169] In various embodiments, control signals may be communicated between system 500 and display 600, speaker 700, or other peripheral device 800 using AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocol signaling enabling device-to-device control, with or without a user.

[0170] Output devices can be communicatively connected to the system 500 via dedicated connections by corresponding interfaces 560, 570 and 580.

[0171] Optionally, an output device can be connected to system 500 using communication channel 900 via communication interface 550. Display 600 and speakers 700 can be integrated into a single unit along with other components of system 500 in an electronic device (e.g., a television).

[0172] In various embodiments, the display interface 560 can include a display driver such as a timing controller (T Con) chip.

[0173] For example, if the RF portion of input 390 is part of a separate set-top box, display 600 and speakers 700 are optionally separate from one or more of the other components. In various embodiments where display 600 and speakers 700 may be external components, the output signals may be provided via dedicated output connections (including, for example, an HDMI port, a USB port, or a COMP output terminal).

[0174] 1-13, various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the precise operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0175] While some examples have been described with reference to block diagrams and / or operational flowcharts, each block represents a circuit element, module, or portion containing one or more executable instruction codes for implementing a specified logic function(s). It should be noted that in other embodiments, the function(s) shown in the blocks may not occur in the order shown. For example, depending on the functionality involved, two blocks shown one after the other may in fact be executed essentially in parallel, or the blocks may be executed in the reverse order.

[0176] For example, the embodiments and aspects described herein may be implemented in a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if discussed only in the context of a single type of embodiment (e.g., discussed only as a method), embodiments of the discussed features may be implemented in other forms (e.g., an apparatus or a computer program).

[0177] The methods may be implemented in, for example, a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device, etc. Processors further include communication devices.

[0178] Additionally, methods may be implemented with instructions executed by a processor, and such instructions (and / or data values ​​produced by embodiments) may be stored on a computer-readable storage medium. A computer-readable storage medium may take the form of a computer-readable program product having computer-readable program code embodied in one or more computer-readable media and embodied therein, executable by a computer. Given their inherent ability to store information thereon and retrieve information provided thereby, computer-readable storage media, as used herein, may be considered non-transitory storage media. A computer-readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Although the following provides more specific examples of computer-readable storage media to which the present embodiment can be applied, it should be understood that these are merely illustrative and not an exhaustive list, as would be readily apparent to one skilled in the art: portable computer floppy disks, hard disks, read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0179] The instructions may create an application tangibly embodied on a processor-readable medium.

[0180] For example, instructions may reside in hardware, firmware, software, or a combination thereof. For example, instructions may be found in an operating system, a standalone application, or a combination of both. A processor may therefore be characterized as, for example, a device configured to perform a process or a device that includes a processor-readable medium (e.g., a storage device) having instructions for performing a process. Also, in addition to or in place of instructions, the processor-readable medium may store data values ​​produced by an embodiment.

[0181] The device may be implemented in, for example, appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptop computers, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics products, head-mounted displays (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from video decoders, pre-processors that provide input to video encoders, web servers, set-top boxes, and any other device for processing point clouds, video, or images, or other communications devices. The device may also be mobile and mounted in a moving vehicle.

[0182] The computer software may be implemented in the processor 510, in hardware, or in a combination of hardware and software. By way of non-limiting example, an embodiment may be implemented in one or more integrated circuits. The memory 520 may be of any type compatible with the technology environment and may be implemented in any suitable data storage technology (by way of non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories). By way of non-limiting example, the processor 510 may be of any type compatible with the technology environment and may cover one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0183] As will be apparent to those skilled in the art, embodiments can generate a variety of signals shaped to carry, for example, storable or transmittable information. The information can include, for example, instructions for performing a method or data generated by one of the described embodiments. For example, a signal can be shaped to carry a bit stream of the described embodiments. The signal can be shaped, for example, as an electromagnetic wave (e.g., a radio frequency portion of the frequency spectrum) or a baseband signal. Shaping can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. As is well known, the signal can be transmitted over different wired or wireless links. The signal can be stored on a processor-readable medium.

[0184] The terms used herein are used only to describe particular embodiments and are not intended to be limiting. As used herein, the singular forms "a," "an," and "the" include the plural, unless the context clearly dictates otherwise. Furthermore, as used herein, the terms "include / comprise" and / or "including / comprising" may indicate the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. Furthermore, when an element is referred to as being "responsive to" or "connected to" another element, it may be directly responsive to or connected to the other element, or intermediate elements may be present. Conversely, when an element is referred to as being "directly responsive to" or "directly connected to" another element, intermediate elements are not present.

[0185] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any one of the symbols / terms " / ," "and / or," and "at least one" is intended to cover the selection of the first listed option (A), the selection of the second listed option (B), or the selection of two options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to cover the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of only the third listed option (C), or the selection of only the first and second listed options (A and B), or the selection of the first and third listed options (A and C), or the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). This can be expanded to any number of listed items, as would be apparent to one skilled in the art.

[0186] Various values ​​may be used in this application, and the specific values ​​are exemplary and the described aspects are not limited to these specific values.

[0187] It should be noted that terms such as "first," "second," etc. may be used to describe various elements herein, but are not limited to these terms. These terms are used only to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the teachings of the present application. No ordering between a first element and a second element is implied.

[0188] References to "one example" or "example" or "one embodiment" or "embodiment" and other variations are often used to convey that a particular feature, structure, characteristic, etc. (as described in connection with the example / embodiment) is included in at least one example / embodiment. Thus, appearances of the terms "in one example" or "in an example" or "in some examples" or "in one embodiment" or "in an embodiment" and any other variations appearing in various places in this application do not necessarily refer to the same example.

[0189] Similarly, references herein to "according to an embodiment" or "in an embodiment" and other variations are often used to convey that a particular feature, structure, or characteristic (as described in connection with an embodiment) may be included in at least one embodiment. Thus, the appearances of "according to an embodiment" or "in an embodiment" in various places in the specification do not necessarily refer to the same embodiment, nor are independent or alternative embodiments mutually exclusive of other embodiments.

[0190] The reference numerals of the drawings appearing in the claims are for illustrative purposes only and do not limit the scope of the claims. Although not explicitly stated, the present embodiments / examples and modifications can be used in any combination or subcombination.

[0191] It is understood that when a figure is shown as a flow chart, a block diagram of the corresponding apparatus is also provided. Similarly, it is understood that when a figure is shown as a block diagram, a flow chart of the corresponding method / process is also provided.

[0192] Some figures include arrows that indicate the primary direction of communication in a communication path, however, it should be understood that communication can occur in the opposite direction to the illustrated arrow.

[0193] Various embodiments relate to decoding. As used herein, "decoding" may cover all or part of the process of, for example, performing received point cloud frames (which may include a received bitstream encoding one or more point cloud frames) to generate a final output suitable for display or further processing in the reconstructed point cloud domain. In various embodiments, such a process may include one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process may optionally include the processes performed by the decoders of various embodiments described herein.

[0194] As a further example, in one embodiment, "decoding" may refer to only inverse quantization, in one embodiment, "decoding" may refer to entropy decoding, in another embodiment, "decoding" may refer to differential decoding, and in another embodiment, "decoding" may refer to a combination of inverse quantization, entropy decoding, and differential decoding. Depending on the context specifically described, it is obvious and understandable to one skilled in the art whether the term "decoding process" refers specifically to a subset of operations or to a more general decoding process.

[0195] Various embodiments relate to encoding. As at least in the context of the discussion of "decoding" above, "encoding" as used herein can cover all or part of the process of, for example, processing an input point cloud frame to generate an encoded bitstream. In various embodiments, this type of process includes one or more of the processes typically performed by an encoder. In various embodiments, such processes can include, or alternatively include, the processes performed by the encoder of each embodiment described herein.

[0196] As a further example, in one embodiment, "encoding" may refer only to quantization, in one embodiment, "encoding" may refer only to entropy coding, in another embodiment, "encoding" may refer only to differential coding, and in another embodiment, "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Depending on the context in which a particular description is made, it may be clear and easy to understand for one skilled in the art whether the term "encoding process" refers specifically to a subset of operations or to a more general encoding process.

[0197] Additionally, this application may refer to "obtaining" various pieces of information. Obtaining information may include one or more of estimating information, calculating information, predicting information, or looking up information from memory.

[0198] Additionally, this application may refer to "accessing" various pieces of information, which may include one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0199] Additionally, the present application may refer to "receiving" various pieces of information. Like "access," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from memory). Alternatively, terms of operations such as storing information, processing information, transmitting information, moving information, copying information, deleting information, calculating information, determining information, predicting information, or estimating information are typically associated with "receiving."

[0200] Additionally, as used herein, the term "signal" specifically refers to instructing a corresponding decoder to do something specific. For example, in some embodiments, an encoder sends a signal to communicate specific information, such as the number or coordinates of points in a point cloud or sensor setting parameters. In this manner, the same parameters can be used on the encoder and decoder sides in some embodiments. Thus, for example, the encoder can send specific parameters to the decoder (explicit signaling), allowing the decoder to use the same specific parameters. Conversely, if the decoder has specific parameters and other parameters, signaling that does not require transmission (indirect signaling) can be used to inform the decoder and facilitate selection of the specific parameters. Various embodiments achieve bit savings by avoiding transmission of any actual functions. It should be appreciated that signaling can be accomplished in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc., are used to transmit information to a corresponding decoder. While the above relates to the verb form of the word "signal," the word "signal" may also be used as a noun in this specification.

[0201] While several embodiments have been described above, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or deleted to produce other embodiments. Moreover, as will be appreciated by those skilled in the art, other structures and processes may be substituted for the disclosed structures and processes, thereby producing embodiments that perform essentially the same function(s) in essentially the same way(s) to achieve at least essentially the same result(s) as the disclosed embodiments. Accordingly, these and other embodiments are contemplated herein.

Claims

1. 1. A method for encoding position information of points of a point cloud into a bitstream comprising at least one data unit, the points being contained in a cubic volume associated with a leaf node of an octree structure having a maximum depth d; - at least three vertices are located on the edges of each of said cubic volumes, each edge having at most one vertex, and the positional information of points included in the cubic volumes is represented by a triangle connecting said at least three vertices; The method comprises: - encoding occupancy information of the nodes of said octree structure, said occupancy information representing the presence of points of said point cloud contained in said cubic volume; for each current cubic volume containing at least one point of the point cloud, encoding Trisoup information, the Trisoup information representing the presence of vertices on edges of the current cubic volume and the positions of the vertices along the edges; The occupancy information of a leaf node and the Trisoup information of other leaf nodes are coded into the data unit (DU) in an interleaved manner; method.

2. 1. A method for decoding position information of points of a point cloud from a bitstream comprising at least one data unit, the points being contained in a cubic volume associated with a leaf node of an octree structure having a maximum depth d; - at least three vertices are located on the edges of each of said cubic volumes, each edge having at most one vertex, and the positional information of points included in the cubic volumes is represented by a triangle connecting said at least three vertices; The method comprises: - decoding occupancy information of the nodes of said octree structure, said occupancy information representing the presence of points of said point cloud contained in said cubic volume; for each current cubic volume containing at least one point of the point cloud, decoding Trisoup information, the Trisoup information representing the presence of vertices on edges of the current cubic volume and the positions of vertices along the edges; The occupancy information of a leaf node and the Trisoup information of other leaf nodes are coded into the data unit (DU) in an interleaved manner; method.

3. The method comprises: The first list (L ｄ－１ ) and obtaining (320, 420, 520, 620) the first list (L ｄ－１ ) storing the nodes of depth d-1 ordered according to the encoding order or the decoding order based on the coordinates of the nodes in the three-dimensional system; The first list (L ｄ－１ ) into a series of bits BSoct1 of the data unit (DU) / encode (340) the occupancy information of p child nodes of depth d of the node of the first list (L ｄ－１ ) and decoding occupancy information of p child nodes of depth d in the node (440).

3. The method according to claim 1 or 2.

4. The method further comprises: ｄ－１ For a first child node (k=0) of a first node in the data unit (DU), the method further comprises the step of encoding (350) Trisoup information into / decoding (450) the series of bits BStris1 of the data unit (DU), the Trisoup information being encoded into / from the series of bits BStris1 of the data unit (DU); The method of claim 3.

5. The method further comprises: ｄ－１ ) for the child nodes of the node in The first list (L ｄ－１ ) until the encoding / decoding of the occupancy information of the last child node (Nmax-th child node) of the node in Encoding (360) / decoding (460) occupancy information into / from the series of bits BSoct2 of said data unit (DU); The method further comprises the steps of alternately encoding (360) Trisoup information into a series of bits BStris2 of the data unit (DU) and decoding (460) Trisoup information from the series of bits BStris2 of the data unit (DU), The method of claim 4.

6. The method further comprises the step of encoding / decoding (370, 470) Trisoup information of the remaining child nodes (p=Nmax-k) of the series of bits BStris3 added to the data unit (DU), The method of claim 5.

7. The method further comprises: ｔｒｉｓ ) and obtaining (540, 640) the second list (L ｔｒｉｓ ) stores child nodes of a node at depth d-1, said nodes having the same coordinates in said three-dimensional system, said nodes being ordered according to a first encoding order or a first decoding order, and said child nodes being ordered according to a second encoding order or a second decoding order; The method of claim 3.

8. The method comprises: The second list (L ｔｒｉｓ ) for the first child node (k=0) of the node in encoding / decoding octree information of a first child node of a node having an updated same coordinate (x=1) in a series of bits BSoct1 of the data unit (DU), where the updated same coordinate is equal to the same coordinate plus 1; The method of claim 7.

9. The method comprises: The first list (L ｄ－１ ) until the encoding / decoding of the occupancy information of the last child node (Nmax-th) child node of the node in Encoding (560) Trisoup information of the next child node of the node with the same updated coordinates into a series of bits BStris2 of the data unit (DU) / decoding (660) Trisoup information of the next child node of the node with the same updated coordinates from the series of bits BStris2 of the data unit (DU); encoding (560) / decoding (660) the occupancy information of the next child node (k=k+1) into a series of bits BSoct2 of said data unit (DU) / decoding the occupancy information of the next child node (k=k+1) from the series of bits BSoct2 of said data unit (DU); in an interleaved manner, The method of claim 8.

10. The method further comprises: ｔｒｉｓ ) into the series of bits BStris3 of the data unit (DU) / encode (570) the Trisoup information of the remaining child nodes (Nmax-k) of the data unit (DU) into the series of bits BStris3 of the data unit (DU) / encode (570) the Trisoup information of the remaining child nodes (Nmax-k) of the data unit (DU) into the series of bits BStris3 of the data unit (DU) ｔｒｉｓ ) further comprising the step of decoding (670) the Trisoup information of the remaining child nodes (Nmax-k) of 10. The method of claim 9.

11. the encoding order and / or the decoding order and / or the first encoding order and / or the first decoding order and / or the second encoding order and / or the second decoding order is a Morton order or a raster scan order; The method according to any one of claims 3 to 7.

12. A bitstream formatted to contain coded positions of points of a point cloud obtained by the method of any one of claims 1, 3 to 11.

13. comprising means for carrying out one of the methods according to any one of claims 1 to 11, Device.

14. A computer program product containing instructions, When the program is executed by one or more processors, the program causes the one or more processors to perform the method according to any one of claims 1 to 11. Computer programming products.

15. A program code instruction for carrying out the method according to any one of claims 1 to 11 is included. Non-transitory storage media.

Images