Method and apparatus for coding presence flags for point clouds and data stream including presence flags

Abstract

A method of encoding geometry information of a geometry graph of a point cloud into a bitstream is described. The point cloud is represented by a plurality of cubes and an occupied cube is modeled by one or more triangles. At least one triangle has at least one respective vertex on an edge of the occupied cube. The geometry information comprises a presence flag signaling a presence of a vertex. The method comprises: constructing (S100), for a current edge, context information based on one or more or all of: occupancy information of neighboring cubes adjoining the current edge, and vertex position information of already encoded neighboring edges of the current edge, the neighboring edges being edges having a common point with the current edge, using (S102) the context information to select a coding probability of an entropy codec, and encoding (S104) the presence flag of the current edge by the entropy codec and using the selected coding probability.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to European Patent Application No. 22167194.4, filed on 7 April 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to data compression, and more particularly to methods and apparatus for encoding and decoding geometric information of the geometry (or geometric structure) of point clouds. Embodiments of this disclosure relate to methods and apparatus for encoding and decoding presence flags that signal the presence of vertices of triangles on the sides of a cube representing a portion of a point cloud, and to data or bitstreams in which presence flags are encoded. Background Technology

[0004] Data compression is used in communications and computer networks to efficiently store, transmit, and reproduce information. For example, point clouds have recently gained attention as a format for representing three-dimensional (3D) data due to their versatility in representing all types of 3D objects or scenes. Therefore, many use cases can be addressed using point clouds, including…

[0005] Post-production of the film

[0006] • Real-time 3D immersive experience or VR / AR (virtual reality / augmented reality) applications,

[0007] • Free-viewpoint videos, such as those used for watching sports.

[0008] Geographic Information Systems, also known as cartography,

[0009] Cultural heritage, such as storing scanned copies of rare items in digital form,

[0010] • Autonomous driving, including 3D mapping of the environment and real-time LiDAR data acquisition (LiDAR: light detection and ranging = a method of measuring distance by illuminating a target with a laser and measuring the reflected light with a sensor).

[0011] Therefore, there is a need for methods and apparatuses that can compress point cloud data more efficiently and / or more effectively. Summary of the Invention

[0012] The present disclosure provides a method of encoding geometry information of a geometry of a point cloud into a bitstream, the point cloud being represented by a plurality of cubes, an occupied cube being modeled by one or more triangles, at least one triangle having at least one respective vertex on an edge of the occupied cube, and the geometry information including a presence flag signaling the presence of a vertex, the method comprising, for a current edge,

[0013] The context information is constructed based on one or more or all of:

[0014] - occupancy information of neighboring cubes adjoining the current edge,

[0015] - vertex position information of already encoded neighboring edges of the current edge, the neighboring edges being edges having a common point with the current edge,

[0016] using the context information to select a coding probability of an entropy codec, and

[0017] encoding the presence flag of the current edge by the entropy codec and using the selected coding probability.

[0018] The present disclosure provides a method of decoding geometry information of a geometry of a point cloud from a bitstream, the point cloud being represented by a plurality of cubes, an occupied cube being modeled by one or more triangles, at least one triangle having at least one respective vertex on an edge of the occupied cube, and the geometry information including a presence flag signaling the presence of a vertex, the method comprising, for a current edge,

[0019] The context information is constructed based on one or more or all of:

[0020] - occupancy information of neighboring cubes adjoining the current edge,

[0021] - vertex position information of already decoded neighboring edges of the current edge, the neighboring edges being edges having a common point with the current edge,

[0022] using the context information to select a coding probability of an entropy codec, and

[0023] decoding the presence flag of the current edge by the entropy codec and using the selected coding probability.

[0024] Optionally, the context information is constructed based on one or more or all of:

[0025] - a count of all occupied neighboring cubes belonging to the subset of neighboring cubes,

[0026] - whether all neighboring cubes belonging to the subset of neighboring cubes are occupied,

[0027] wherein the subset of adjacent cubes comprises one or more or all of:

[0028] - adjacent cubes sharing a current edge,

[0029] - adjacent cubes having a corner as a starting point of a current edge,

[0030] - adjacent cubes having a corner as an end point of a current edge.

[0031] Optionally, the context information is constructed based on one or more or all of:

[0032] - values of already encoded / decoded presence flags associated with adjacent edges of the current edge,

[0033] - positions of vertices on already encoded / decoded adjacent edges of the current edge,

[0034] - counts of vertices on already encoded / decoded adjacent edges of the current edge, which are at a distance below a predetermined threshold from the current edge,

[0035] - counts of vertices on already encoded / decoded adjacent edges of the current edge, which are at a distance within a predetermined interval from the current edge.

[0036] Optionally, the context information is constructed based on

[0037] - a number of adjacent edges for which the already encoded / decoded presence flag is true and a number of adjacent edges for which the already encoded / decoded presence flag is false, or

[0038] - only a number of adjacent edges for which the already encoded / decoded presence flag is false.

[0039] Optionally, constructing the context information comprises quantizing the position of the vertex on the already encoded / decoded adjacent edge to a coarser precision than the precision at which the position of the vertex on the already encoded / decoded adjacent edge is encoded into the bitstream.

[0040] Optionally, to select the coding probability of the entropy codec, the method uses

[0041] - a best unary binarization (OBUF) mechanism with dynamic update, or

[0042] - a context adaptive binary arithmetic coding (CABAC) mechanism.

[0043] Optionally, the point cloud is modeled using a TriSoup coding scheme.

[0044] The present disclosure provides a non-transitory computer readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of the present invention.

[0045] The present disclosure provides an apparatus for encoding geometry information of a geometry of a point cloud into a bitstream, the point cloud being represented by a plurality of cubes, an occupied cube being modeled by one or more triangles, at least one triangle having at least one respective vertex on an edge of the occupied cube, and the geometry information comprising a presence flag signaling the presence of a vertex, the apparatus comprising:

[0046] a context information construction module configured to construct, for a current edge, context information based on one or more or all of:

[0047] - occupancy information of neighboring cubes that are adjacent to the current edge,

[0048] - vertex position information of already encoded neighboring edges of the current edge, a neighboring edge being an edge having a common point with the current edge,

[0049] a selection module configured to select, using the context information provided by the context information construction module, a coding probability of an entropy codec, and

[0050] an encoding module configured to encode, by the entropy codec and using the selected coding probability, the presence flag of the current edge.

[0051] The present disclosure provides an apparatus for decoding geometry information of a geometry of a point cloud from a bitstream, the point cloud being represented by a plurality of cubes, an occupied cube being modeled by one or more triangles, at least one triangle having at least one respective vertex on an edge of the occupied cube, and the geometry information comprising a presence flag signaling the presence of a vertex, the apparatus comprising:

[0052] a context information construction module configured to construct, for a current edge, context information based on one or more or all of:

[0053] - occupancy information of neighboring cubes that are adjacent to the current edge,

[0054] - vertex position information of already decoded neighboring edges of the current edge, a neighboring edge being an edge having a common point with the current edge,

[0055] a selection module configured to select, using the context information provided by the context information construction module, a coding probability of an entropy codec, and

[0056] a decoding module configured to decode, by the entropy codec and using the selected coding probability, the presence flag of the current edge.

[0057] Optionally, the context information construction module comprises one or more or all of the following sub-modules:

[0058] - an occupancy information obtaining sub-module configured to obtain occupancy information of adjacent cubes adjoining the current edge,

[0059] - a vertex presence obtaining sub-module configured to obtain vertex presence on the adjacent edge that has been encoded / decoded,

[0060] - a vertex position obtaining sub-module configured to obtain vertex position along the adjacent edge that has been encoded / decoded.

[0061] Optionally, the vertex presence obtaining sub-module is configured to obtain a value of an encoded / decoded presence flag associated with the adjacent edge of the current edge.

[0062] Optionally, the vertex position obtaining sub-module is configured to obtain one or more or all of the following:

[0063] - a position of a vertex on the adjacent edge of the current edge that has been encoded / decoded,

[0064] - a count of vertices on the adjacent edge of the current edge that have been encoded / decoded, the vertices being at a distance from the current edge lower than a predetermined threshold,

[0065] - a count of vertices on the adjacent edge of the current edge that have been encoded / decoded, the vertices being at a distance from the current edge within a predetermined interval.

[0066] The present disclosure provides a data stream in which geometry information of a geometric graph of a point cloud is encoded, the point cloud being represented by a plurality of cubes, an occupied cube being modeled by one or more triangles, at least one triangle having at least one respective vertex on an edge of the occupied cube, and the geometry information comprising a presence flag signaling the presence of the vertex, the data stream comprising, for a current edge,

[0067] The presence flag of the current edge encoded into the data stream by an entropy encoder and using a coding probability, the coding probability being selected using context information,

[0068] wherein the context information is constructed based on one or more or all of the following:

[0069] - occupancy information of adjacent cubes adjoining the current edge,

[0070] - vertex position information of an encoded adjacent edge of the current edge, the adjacent edge being an edge having a common point with the current edge.

[0071] The technical solutions provided by the embodiments of the present disclosure have the following beneficial effects.

[0072] While compression of TriSoup data is not very efficient in the prior art, embodiments of this disclosure provide improved compression of vertex flags or presence flags that indicate the presence of a TriSoup vertex on an edge of an occupied leaf node / volume. In the prior art, vertex flags are compressed using a binary entropy codec with a dedicated fixed context, resulting in poor compression. This disclosure introduces the use of better context information to select a context or entropy codec (e.g., by applying a best-in-class binary codec mechanism with dynamic updates (so-called OBUF)), leading to a better estimate of the probability that a vertex or presence flag is false (0) or true (1), which in turn leads to better compression of the flag by the entropy codec.

[0073] Embodiments of this disclosure obtain relevant contextual information relating to the value of the vertex flag.

[0074] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0075] The accompanying drawings are illustrative and intended to explain this disclosure, and are not to be construed as limiting this disclosure to the illustrated embodiments.

[0076] Figure 1 The diagram illustrates the TriSoup vertices along the edges of the body associated with the leaf nodes of the octree representation of the point cloud geometry.

[0077] Figure 2 The diagram illustrates a volume comprising two TriSoup triangles, each with its vertices on an edge of the volume.

[0078] Figure 3 The diagram illustrates a volume including a TriSoup triangle, where two vertices of the TriSoup triangle lie on the sides of the volume and one vertex of the TriSoup triangle lies inside the volume.

[0079] Figure 4 The diagram illustrates the process of selecting the triangle to be constructed inside the leaf node, where Figure 4 (a) The illustration shows the absence of triangles. Figure 2 The body, Figure 4 (b) The diagram illustrates a 2D surface of a triangle projected vertically onto the volume, and Figure 4 (c) The diagram illustrates a 2D surface of a triangle using the vertical horizontal projection of a volume;

[0080] Figure 5 Fig. illustrates a volume with two TriSoup triangles, which are constructed under the assumption that the horizontal axis is the principal axis; Figure 2

[0081] Figure 6 Fig. illustrates ray tracing of TriSoup triangles rendered as decoded point cloud;

[0082] Figure 7 Fig. illustrates a flowchart of a method of encoding geometry information of a geometry of a point cloud into a bitstream according to an embodiment of the present disclosure;

[0083] Figure 8 Fig. illustrates a flowchart of a method of decoding geometry information of a geometry of a point cloud from a bitstream according to an embodiment of the present disclosure;

[0084] Figure 9 Fig. illustrates a data stream according to an embodiment of the present disclosure;

[0085] Figure 10 Fig. illustrates a flowchart of an encoding method according to an embodiment of the present disclosure;

[0086] Figure 11 Fig. illustrates a flowchart of a decoding method according to an embodiment of the present disclosure;

[0087] Figure 12 Fig. illustrates 4 leaf nodes / volumes sharing the current edge k (illustrated by the arrow);

[0088] Figure 13 Fig. illustrates 8 leaf nodes / volumes, one corner of which is the start or end of the current edge k (illustrated by the arrow);

[0089] Figure 14 Fig. illustrates the unique edge k’ (shown by the dashed arrow) parallel to the current edge k (illustrated by the arrow) and pointing to its start;

[0090] Figure 15 Fig. illustrates the 4 perpendicular edges k’ (illustrated by the dashed arrows) orthogonal to the current edge k (illustrated by the arrow) and pointing to or starting from its start;

[0091] Figure 16 Fig. illustrates the number N of already encoded / decoded perpendicular edges k’ depending on the direction of the current edge k; perp (k);

[0092] Figure 17 Fig. illustrates the TriSoup vertex V par and its quantized position p on the already encoded / decoded unique edge k’​par,Q ;

[0093] Figure 18 illustrates the position of the vertex belonging to the already encoded / decoded edge k’ is “near” vs. “far”;

[0094] Figure 19 illustrates the position of the vertex belonging to the already encoded / decoded edge k’ is “near” vs. “mid” vs. “far”;

[0095] Figure 20 illustrates a block diagram of an apparatus / encoder for encoding vertex flags s k according to an embodiment;

[0096] Figure 21 illustrates a block diagram of an apparatus / decoder for decoding vertex flags s k according to an embodiment;

[0097] Figure 22 illustrates a block diagram of an embodiment of a context information construction module that can be used in an encoder of Figure 20 or a decoder of Figure 21 ; and

[0098] Figure 23 illustrates a block diagram illustrating an electronic device configured to implement an image processing method according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0099] Illustrative embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help with the understanding. These should be considered in the context of the overall description and they should not be taken to limit the scope of the disclosure. Therefore, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the disclosure. Also, the description below is presented in terms of a number of acts that can be performed or completed by one or more computing devices. These acts can be performed or completed by various elements of a computing device, such as processing elements, device drivers, firmware, operating system, or a combination thereof. For the sake of presentation, the various elements of the computing device are presented as interconnected, but in practice, they can be separate, and possibly distributed, as well. Also, different implementations include different types of machines, with different combinations of means for performing the described acts. In addition, the modules can communicate over a series of one or more networks, which can each themselves include sub-networks. In fact, a network- enabled device can communicate over network(s) when only a local (e.g., intranet) portion of the network is used. In one embodiment, the network includes the Internet in addition to local area networks. Networks can also be implemented as virtual networks, including virtual private networks (VPNs). In addition, the various network-enabled devices can communicate over the network(s) using a variety of technologies, including, but not limited to, wireless technologies, wired technologies, or a combination thereof.

[0100] In the present disclosure, the terms “node”, “volume” and “sub-volume” can be used interchangeably. It should be understood that a node is associated with a volume or sub-volume. A node is a particular point on a tree, which can be an internal node or a leaf node. A volume or sub-volume is a bounded physical space represented by a node. In some cases, the term “volume” can be used to refer to the largest bounded space defined to contain a point cloud. For the purpose of constructing a tree structure of interconnected nodes for coding point cloud data, a volume can be recursively divided into sub-volumes.

[0101] In the present disclosure, the term “and / or” is intended to cover all possible combinations and sub-combinations of the listed elements, including any one of the listed elements, any sub-combination, or all of the elements, and does not necessarily preclude additional elements.

[0102] In this disclosure, the phrase“at least one of… or…” is intended to cover any one or more of the listed elements, including any one of the listed elements alone, any subcombination of elements, or all of the elements, without necessarily excluding any additional elements.

[0103] In this disclosure, as is evident from the context of the described embodiments relating to encoding / decoding geometry information into / from a bitstream, the term“coding” refers to either“encoding” or“decoding”. Likewise, the term“coder” refers to either an“encoder” or a“decoder”.

[0104] A point cloud is a collection of points in a three-dimensional coordinate system. The points are generally intended to represent the outer surface of one or more objects. Each point has a position or location in the three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be in a Cartesian coordinate system or any other coordinate system. The points can have other associated properties, such as color, which can also be a three-component value in some cases, such as R, G, B or Y, Cb, Cr. Other associated properties can include transparency, reflectivity, normal vector, etc., depending on the desired application of the point cloud data.

[0105] A point cloud can be static or dynamic. For example, a detailed scan or drawing of an object or terrain can be static point cloud data. A LiDAR-based environmental scan for machine vision purposes can be dynamic, as the point cloud at least potentially changes over time, e.g., with each successive scan of the volume. Thus, a dynamic point cloud is a temporally ordered sequence of point clouds.

[0106] As mentioned above, point cloud data can be used for a variety of applications or use cases, including preservation, such as scanning historical or cultural items, mapping, machine vision (e.g., for autonomous or semi-autonomous cars), and virtual or augmented reality systems. Dynamic point cloud data for applications such as machine vision can be quite different from static point cloud data for applications such as preservation. For example, car vision typically involves a relatively small resolution, colorless, highly dynamic point cloud obtained at a high capture frequency by a LiDAR or similar sensor. The purpose of such a point cloud is not for human consumption or viewing, but for machine object detection / classification in a decision process. For example, a typical LiDAR frame contains on the order of tens of thousands of points, whereas high-quality virtual reality applications require millions of points. It can be expected that as computing speeds increase and new applications or use cases are discovered, higher resolution data will be required over time.

[0107] In other words, a point cloud is a collection of points located in a 3D space, optionally each point being attached with additional values. These additional values are often referred to as point attributes. A point cloud can thus be considered as a combination of geometry (3D position of each point) with attributes. Attributes can be for example, three-component colors, material attributes like reflectance and / or two-component normal vectors of a surface associated with a point. Point clouds can be captured by various types of devices like camera arrays, depth sensors, the mentioned LiDARs, scanners or they can be computer generated (for example in movie post-production use cases). Depending on the use case, a point cloud can have from thousands to billions of points for graphics applications.

[0108] The raw representation of a point cloud requires very high number of bits per point, at least a dozen of bits per spatial component X, Y or Z and optionally more bits for one or more attributes, for example three times 10 bits for colors. The practical deployment of point cloud based applications or use cases thus requires compression techniques that enable to store and distribute point clouds with reasonable storage and transmission infrastructure. In other words, while point cloud data is useful, the lack of efficient and effective compression (i.e. encoding and decoding processes) can hinder its adoption and deployment. A particular challenge for coding point clouds is to code the geometry of the point cloud, which does not occur in other data compression like audio or video, and the tendency of point clouds to be sparsely distributed makes the efficient coding of the positions of the points more challenging.

[0109] Until recently, point cloud compression (also referred to as PCC) has not been addressed by the mass market and no standardized point cloud codec was available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also referred to as Moving Picture Experts Group or MPEG, started work items on point cloud compression, which resulted in two standards, namely:

[0110] • MPEG-I Part 5 (ISO / IEC 23090-5), also referred to as Video-based Point Cloud Compression V-PCC,

[0111] • MPEG-I Part 9 (ISO / IEC 23090-9), also referred to as Geometry-based Point Cloud Compression G-PCC.

[0112] The first version of the V-PCC standard and the first version of the G-PCC standard were completed in 2020 and 2022, respectively.

[0113] The V-PCC encoding method compresses a point cloud by performing multiple projections of the 3D object to obtain two-dimensional (2D) patches packed into images or into videos when processing moving point clouds. Then, the images or videos are compressed using existing image / video codecs, allowing to leverage already deployed image and video solutions. By its nature, V-PCC is only efficient on dense and continuous point clouds, as in case of non-smooth patches obtained from the projection of sparse geometry data acquired from e.g. LiDAR, the image / video codec cannot compress non-smooth patches.

[0114] The G-PCC encoding method has two schemes for compressing geometry.

[0115] • The first scheme is based on an occupancy tree representation of the point cloud geometry, e.g. by an octree representation, a quadtree representation or a binary tree representation. In a tree-based structure, the bounding three-dimensional volume of the point cloud is recursively split into sub-volumes. The nodes of the tree correspond to the sub-volumes. The decision whether to further split a sub-volume can be based on the resolution of the tree and / or whether any points are contained in the sub-volume. Leaf nodes can have an occupancy flag indicating whether their associated sub-volume contains points. A split flag can signal whether a node has child nodes, i.e. whether the current volume has been further split into sub-volumes. A commonly used tree structure is the octree. In this structure, the volumes / sub-volumes are all cubes and each split of a sub-volume results in eight other sub-volumes / sub-cubes. Another commonly used tree structure is the KD-tree, in which volumes, like cubes, are recursively bisected by a plane orthogonal to one of the axes. The octree is a special case of the KD-tree, in which the volumes are divided by three planes, each orthogonal to one of the three axes.

[0116] In other words, occupied nodes are split until a certain size is reached and occupied leaf nodes provide the positions of the points, typically at the center of these nodes. By using a neighbor-based prediction technique, a high level of compression of dense point clouds can be obtained. Sparse point clouds are also handled by directly encoding the positions of the points within nodes having a non-minimal size, by stopping the tree construction when only isolated points exist in a node. This stopping technique is also referred to as direct coding mode (DCM).

[0117] • The second scheme is based on a prediction tree, in which each node represents a 3D position of a point and the relationship between nodes is a spatial prediction from parent to child nodes. This method can only handle sparse point clouds and offers the advantage of lower latency and simpler decoding compared to using an occupancy tree. However, the compression performance is only slightly better when compared to the first scheme, but the encoding is complex as a best predictor needs to be found centrally among a long list of potential predictors when building the prediction tree.

[0118] In both approaches, attribute coding (i.e. attribute encoding and attribute decoding) is performed after the entire geometry coding, resulting in a two-pass coding process. Low latency can be obtained by using slices that decompose the 3D space into independently coded sub-volumes, without the need for prediction between sub-volumes. However, this can severely impact compression performance when many slices are used.

[0119] One use case of particular interest is the transmission of dynamic AR / VR point clouds, where dynamic means that the point cloud evolves over time. Moreover, AR / VR point clouds are often locally 2D, in the sense that they represent the surface of an object most of the time. Therefore, AR / VR point clouds are highly connected (also referred to as dense), in the sense that points are rarely isolated, but have many neighbors. Thus, dense or solid point clouds represent continuous surfaces, with a resolution such that the volumes (also referred to as small cubes or voxels) associated with the points are in contact with each other, without exhibiting any visible holes on the surface. Such point clouds are typically used in AR / VR environments, and are viewed by the end user through a device such as a TV, a smartphone or a headset including AR / VR glasses. The point cloud can be transmitted to the device or can be stored locally. Many AR / VR applications use a moving point cloud, rather than a static point cloud, that changes over time. Therefore, the amount of data can be large, requiring compression. For example, when using an octree representation of the geometry of the point cloud as described above, lossless compression can achieve slightly less than 1 bit per point (or 1 bpp). However, this can be insufficient for real-time transmission, which can involve millions of points per frame, at a frame rate of up to 50 frames per second (fps), resulting in several hundreds of megabytes per second of data.

[0120] Therefore, lossy compression schemes can be used, typically requiring to provide a visual quality that remains acceptable while keeping the compression within the bandwidth available in the transmission channel, while maintaining real-time transmission of the frames. In many applications, bitrates as low as 0.1 bpp have made real-time transmission possible, meaning that with the help of lossy compression, the compression of a point cloud is ten times higher than when applying a lossless coding scheme.

[0121] A codec based on MPEG-I Part 5 (ISO / IEC 23090-5) or V-PCC can achieve such low bitrates by using the lossy compression of a video codec that compresses 2D frames obtained from the projection of the point cloud on a plane. The geometry is represented by a series of projected patches that are assembled into a frame, where each patch is a small local depth map. However, VPCC is not general-purpose and is limited to a narrow type of point clouds that do not exhibit locally complex geometry (such as trees or hair, etc.), because the projected depth maps obtained are not smooth enough to be efficiently compressed by a video codec.

[0122] In another aspect, a pure 3D compression technique can handle any kind of point cloud. For example, G-PCC can provide in the future a lossy compression that also allows compressing dense point clouds as well as V-PCC intra coding, while keeping the versatility of G-PCC in order to handle any kind of point cloud, like dense point clouds, point clouds obtained by LiDAR or point clouds representing 3D maps. To implement such a G-PCC mechanism, a so-called TriSoup coding scheme can be applied on a first layer based on an octree. The TriSoup coding scheme is currently under discussion in the standardization working group JTC1 / SC29 / WG7 of ISO / IEC. When considering the possibility to derive a lossy scheme from G-PCC, there are basically three approaches to obtain a lossy scheme from the octree representation used by the 3-PCC codec, namely

[0123] • down-sampling + (lossless) encoding + re-up-sampling

[0124] • locally modifying the voxels at the encoder side

[0125] • locally modeling the point cloud.

[0126] The first approach basically consists in down-sampling the whole point cloud to a smaller resolution, losslessly coding the down-sampled point cloud and then up-sampling it after decoding. There are many up-sampling schemes, such as super-resolution, artificial intelligence, AI, or learning-based 3D post-processing, which can provide good peak signal-to-noise ratio, PSNR, results when the down-sampling is not too aggressive, e.g. not more than two times in each direction. However, even if the metrics show a good PSNR, the visual quality is still controversial and not well controlled.

[0127] The second approach allows the encoder to locally adjust the point cloud so that the coding of the octree requires a lower bitrate. For this purpose, the points can be slightly moved in order to obtain occupancy information that can be better predicted by neighboring nodes, resulting in a lossless encoding of the modified octree at a lower bitrate. Unfortunately, however, this approach only leads to a small reduction in bitrate.

[0128] The third approach is to code the geometry using a tree, like an octree, up to a certain resolution, e.g. up to NxNxN blocks, where N can be e.g. 4, 8 or 16. Then, the tree is coded using a lossless scheme like the G-PCC scheme. The tree itself does not require a high bitrate as it does not go down to the deepest depth and there are only a small number of leaf nodes compared to the number of points in the point cloud. Then, in each NxNxN block, the point cloud is modeled by a local model. This model can be an average plane or a set of triangles like in the TriSoup coding scheme mentioned above, which is now described in more detail.

[0129] The TriSoup coding scheme locally models the point cloud by using a set of triangles without explicitly providing connectivity information, which is the reason for its name, the "soup of triangles". As mentioned above, each NxNxN block defines a volume associated with a leaf node and in each NxNxN block or volume, the point cloud is locally modeled using a set of triangles, where the vertices of the triangles are coded along the edges of the volume associated with the leaf node of the tree. Figure 1 A volume 100 associated with a leaf node is illustrated, which is defined by twelve edges 1001 to 100 12 A designed cube.

[0130] A part of the point cloud enclosed by the volume 100 is modeled by at least one triangle having at least one vertex on one of the edges 1001 to 100 12 In the example illustrated, five vertices 1 to 5 are illustrated, where vertices 1 to 4 are located on edges 1002, 1001, 1008 and 1007, respectively. Vertex 5 is not located on any edge but inside the volume 100. In other words, for locally modeling the point cloud, one or more triangles are used and a triangle can have at least one vertex located on an edge of the volume, like vertices 1 to 4, while other triangles can have one or two of their vertices not located on an edge but inside the volume 100 (like vertex 5) or at the boundary of the volume 100. Figure 1 Vertices located on edges are shared between leaf nodes having a common edge, which means that at most one vertex is coded per edge belonging to at least one leaf node, so that the continuity of the model is guaranteed by the leaf nodes. The coding of a TriSoup vertex requires two information per edge:

[0131] • a vertex flag indicating whether a TriSoup vertex is present on the edge, also referred to as a presence flag in this document, and

[0132] • the position of the vertex along the edge if a vertex is present.

[0133] Thus, the coded data comprises the octree data plus the TriSoup data. For example, the vertex flag can be coded by an adaptive binary arithmetic coder, which codes the vertex flag using one specific context for the vertex flag on edges of length N = 2 s The position of the vertex on an edge of length N = 2s is coded with unit precision by pushing s bits into the bitstream, i.e. by bypassing / non-entropy coding s bits.

[0134]

[0135] Figure 2 ​The diagram illustrates a body 100 associated with leaf nodes, comprising two TriSoup triangles 102 and 104. Vertices 1 to 4 of each of these two TriSoup triangles 102 and 104 are located on the edges of body 100 (see Figure 100). Figure 1 On 1002, 1001, 1008, and 1004. Triangle 102 includes vertices 1, 2, and 3, while triangle 104 includes vertices 1, 3, and 4. Therefore, a triangle can be constructed if there are at least three vertices on the edges of volume 100. Naturally, inside volume 100 associated with leaf nodes, besides... Figure 2 Any other combination of triangles besides the triangle shown is possible. Furthermore, not all vertices of one or more triangles inside body 100 are necessarily on the edges of body 100; on the contrary, one or two vertices of a triangle can be located anywhere inside body 100. Figure 3 The diagram illustrates volume 100, which includes vertices 1 to 4 located on the corresponding edges of volume 100 (e.g., ...). Figure 2 ) and another vertex 5 located within body 100. Inside body 100, four triangles 106 to 112 are constructed by TriSoup vertices 1 to 5, wherein the first triangle 106 includes vertices 1, 2 and 5, the second triangle 108 includes vertices 2, 3 and 5, the third triangle 110 includes vertices 3, 5 and 5, and the fourth triangle 112 includes vertices 1, 4 and 5.

[0136] The triangle to be constructed inside volume 100 is based on the following three-step process, including:

[0137] 1. Determine the dominant direction along one of the three axes.

[0138] 2. Sort the TriSoup vertices according to the dominant direction.

[0139] 3. Construct triangles based on an ordered list of vertices.

[0140] Figure 4 The diagram illustrates the selection process. Figure 2 The process of constructing triangles within body 100 associated with leaf nodes. Figure 4 (a) illustrates this again. Figure 2 The body is 100, but there is no triangle. Figure 4 (b) and Figure 4 (c) The diagram illustrates the process on two axes, namely the vertical axis or the z-axis. Figure 4 (b) and the horizontal axis or x-axis ( Figure 4 (c)).

[0141] like Figure 4(c) shown, the first test along the vertical axis is performed by vertically projecting the volume or cube 100 and TriSoup vertices on the 2D plane, i.e. testing from the top. Then, the vertices are ordered in a clockwise order with respect to the center of the projection node 114, which in the illustrated example is a square. As Figure 4 (c) shown, based on the ordered vertices, triangles are constructed in a fixed rule, in Figure 4 The example involves four vertices and systematically constructs triangles 102, 104 to include vertices 1, 2, and 3 of a first triangle and vertices 1, 3, and 4 of a second triangle. When there are only three vertices, the only possible triangle is the one including vertices 1, 2, and 3, and when there are five vertices, triangles including vertices (1, 2, 3), (1, 3, 5), and (4, 5, 1), etc. can be constructed using the fixed rule. This can be repeated up to 12 vertices.

[0142] When viewed from the left of Figure 4 (a), the second test along the horizontal axis is performed by horizontally projecting the cube 100 and TriSoup vertices on the 2D plane, resulting in Figure 4 (b) shown. When the vertices are ordered in a clockwise order with respect to the center of the projection node 100, the triangles 102, 104 include vertices 1, 2, and 3 of a first triangle and vertices 1, 3, and 4 of a second triangle, as Figure 4 (b) shown.

[0143] As can be seen, the vertical projection Figure 4 (c) exhibits the largest 2D total surface of the triangles, such that the principal axis is chosen as the vertical axis or z-axis, and according to Figure 4 (c) shown, the order of the vertical projection is obtained for the TriSoup triangles to be constructed, which in turn results in Figure 2 triangles inside the volume depicted. It is to be noted that when considering the horizontal axis as the principal axis, this leads to a different construction of the triangles inside the volume 100 as Figure 5 depicted, and Figure 5 depicted. Figure 4 Figures illustrating the volume 100, where the triangles 102, 104 are constructed in the order of the vertices as Figure 6 (b) shown, assuming the principal axis to be the horizontal axis.

[0144] By appropriately choosing the principal axis by maximizing the projected surface, a continuous reconstruction of the point cloud without holes can be achieved.

[0145] The rendering of the TriSoup triangles is performed by ray tracing, and the collection of all points rendered by ray tracing will form the decoded point cloud. Figure 2The diagram illustrates the rendering of vertices 1, 2, and 3. Figure 6 Ray tracing of a TriSoup triangle 102. Rays (such as...) Figure 6 The light ray 118 in the image travels along a path parallel to the axis (e.g., ...). Figure 7 The ray is emitted along the z-axis. The origin of the ray is a voxelized point with integer precision corresponding to the sampling position required for rendering. Then, the intersection 120 of ray 118 and triangle 102 is voxelized, that is, rounded to the nearest point at the desired sampling position, and added to the list of rendering points. After applying the TriSoup encoding / decoding scheme to all leaf nodes, that is, after constructing triangles and obtaining intersections via ray tracing, copies of identical points in all lists of rendering points are discarded (i.e., only one voxel is retained among all voxels sharing the same 3D position), thus obtaining a unique set of decoded points.

[0146] When a point cloud is locally modeled by applying a TriSoup encoding scheme using a set of triangles for each leaf node or volume as described above, so-called TriSoup data is also provided. For example, TriSoup data includes information about the vertices of the corresponding triangles for the volume. However, compression of TriSoup data is not efficient in the prior art. For example, in the prior art, compression of vertex flags indicating the presence of TriSoup vertices on edges of occupied volumes associated with leaf nodes utilizes an entropy codec (such as a binary entropy codec) with a dedicated fixed context. This disclosure is based on the finding that compression of TriSoup data can be improved by improving the estimation of the probability that a vertex flag or presence flag is true or false. More specifically, in accordance with the teachings of this disclosure, improved contextual information is introduced to select the context or entropy codec, instead of compressing vertex flags by using an entropy codec with a dedicated fixed context (such as a binary entropy codec) as in the prior art. This results in improved probability estimation, which in turn produces better compression of vertex flags through the entropy codec. Therefore, according to the method of the present invention, the compression efficiency of TriSoup data can be improved by obtaining contextual information related to the value of the vertex flag, thereby better estimating the probability of the flag being true or false, which in turn leads to improved compression of the flag by the entropy codec.

[0147] According to an embodiment, the context information (CI) can be based on the occupancy information of neighboring bodies adjacent to the current edge, and / or on the vertex position information of the neighboring edges that has been encoded / decoded. The vertex position information may include vertex flags and / or the vertex positions on the edges that have been encoded / decoded.

[0148] Figure 2A flowchart of a method of encoding geometry information of a geometry of a point cloud into a bitstream according to an embodiment of the disclosure is illustrated. The point cloud is represented by a plurality of cubes, and an occupied cube is modeled by one or more triangles, as Figure 3 and Figure 7 illustrated. At least one triangle has at least one respective vertex on an edge of the occupied cube. The geometry information includes a presence flag signaling the presence of a vertex. According to an embodiment, as Figure 8 depicted, the method comprises the steps of, for a current edge of a cube,

[0149] S100: constructing context information based on one or more or all of:

[0150] • occupancy information of neighboring cubes that are adjacent to the current edge,

[0151] • vertex position information of already encoded neighboring edges of the current edge, the neighboring edges being edges that have a common point with the current edge.

[0152] S102: selecting a coding probability of an entropy codec using the context information.

[0153] S104: encoding the presence flag of the current edge by the entropy codec and using the selected coding probability.

[0154] Figure 8 A flowchart of a method of decoding geometry information of a geometry of a point cloud from a bitstream according to an embodiment of the disclosure is illustrated. The point cloud is represented by a plurality of cubes, and an occupied cube is modeled by one or more triangles. At least one triangle has at least one respective vertex on an edge of the occupied cube. The geometry information includes a presence flag signaling the presence of a vertex. According to an embodiment, as Figure 9 depicted, the method comprises the steps of, for a current edge of a cube

[0155] S200: constructing context information based on one or more or all of:

[0156] • occupancy information of neighboring cubes that are adjacent to the current edge,

[0157] • vertex position information of already decoded neighboring edges of the current edge, the neighboring edges being edges that have a common point with the current edge,

[0158] S202: selecting a coding probability of an entropy codec using the context information.

[0159] S204: decoding the presence flag of the current edge by the entropy codec and using the selected coding probability.

[0160] Figure 9 A data stream 200 is illustrated in which geometry information of a geometry of an encoded point cloud is signaled according to an embodiment of the disclosure. The point cloud is represented by a plurality of cubes and a portion of the point cloud enclosed by an occupied cube is modeled by one or more triangles. At least one triangle has at least one respective vertex on an edge of the occupied cube. The geometry information comprises a presence flag 202 signaling the presence of a vertex. For example, the data stream or bitstream 200 can be provided by an encoder 300 performing the inventive method for encoding the presence flag 202 into the data stream 200. The data stream 200 is transmitted via a wired or wireless transmission medium 204, like a cable or a radio link, to a decoder 400 and the decoder 400 decodes the presence flag 202 from the data stream 200. Thus, according to embodiments, as Figure 10 depicted, for a current edge, the data stream 200 comprises the presence flag 202 for the current edge encoded into the data stream 200 by an entropy coder and using a coding probability. The coding probability is selected using context information and the context information is constructed based on one or more or all of:

[0161] • occupancy information of neighboring cubes adjoining the current edge,

[0162] • vertex position information of already encoded neighboring edges of the current edge, which are edges having a common point with the current edge.

[0163] Embodiments of the invention are now described in more detail. For the following description, edges of at least one occupied volume are indexed by an index k. An occupied volume is a volume associated with an occupied leaf node of a tree representing the underlying geometry of a point cloud. The presence of a TriSoup vertex on edge k is signaled by a vertex flag or presence flag s k which can be true (1) or can be false (0). According to embodiments of the disclosure, in order to code the vertex flag s k associated with the current edge k, an entropy coder or a context to be used by the entropy coder is selected based on context information CI, which is constructed using one or more or all of: k

[0164] • occupancy of neighboring volumes associated with neighboring nodes of the tree,

[0165] • one or more vertex flags k’ of neighboring already encoded edges k’, and

[0166] • one or more vertex positions p k’ on the neighboring already encoded edges k’. k’ . ​

[0167] Figure 11 A flow chart of an encoding method according to embodiments of the present disclosure is illustrated. For a current edge k, at step S100, the occupancy of neighboring nodes / volumes e l , a m and b n is obtained as illustrated at step S106. The neighboring nodes / volumes include the volumes e l , a m and b n intersecting the current edge k as described in more detail below. Furthermore, at step S108, the vertex presence (as vertex flag s k’ ) of neighboring already encoded edges k’ is obtained and at step S110, the vertex position p k’ along the vertex of the neighboring already encoded edges k’ is obtained. Based on the information obtained in steps S106 to S110, at step S112, context information CI k is constructed. Embodiments for constructing the context information CI k are described in more detail below.

[0168] At step S102, the context information CI k is used to select a context or entropy codec in order to obtain coding probabilities based on the context information CI k for encoding the vertex flag.

[0169] At step S104, the vertex flag s k of the current edge k is received and encoded into the bitstream 200 by the entropy codec using the selected coding probabilities.

[0170] According to embodiments, at step S102, the coding probabilities of the entropy codec used for encoding the vertex flag s k are selected, e.g. by applying a context adaptive binary arithmetic coding, CABAC, mechanism, or by applying an optimal binary coder, OBUF, mechanism with dynamic updates, e.g. as described in EP 3 633 857 Al, the content of which is incorporated herein by reference.

[0171] Figure 10 A decoding method according to embodiments of the present disclosure is illustrated, which is basically the same as the method described above with reference to Figure 11 the difference being that the vertex flag s k is decoded from the bitstream 200 by the codec instead of being encoded into the bitstream. Figure 8 Steps S200 to S204 described above with reference to Figure 10 are illustrated and are directed to a current edge k, in the same way as described above with reference to Figure 10In the same way as described, in steps S206 to S210, information about occupancy of the neighboring bodies, presence of a vertex on the neighboring already decoded edges and position of the vertex along the neighboring already decoded edges is determined in order to build, in step S212, the context information CI k . In step S202, as also mentioned above with reference to Figure 12 The described context or entropy codec is selected. In step S204, the vertex presence flag or vertex flag s k .

[0172] The above described embodiment of encoding / decoding of the vertex flag has been explained with reference to the processing for the current edge k, i.e. the above described encoding / decoding procedure is repeated for all edges of the occupied body to obtain the respective information whether a vertex is present on a particular edge. The edge k is oriented from the start point to the end point and in the following figures the currently processed edge k is denoted as an arrow from the start point to the end point, i.e. the arrow is the end point of the edge k. If a TriSoup vertex V k is present on the edge k, its position p k can be provided along the edge with respect to the start point position of the edge such that the position p k is a scalar information.

[0173] There are 12 neighboring bodies intersecting the current edge k, i.e.

[0174] • 4 neighboring bodies el to e4 sharing the current edge k as shown in Figure 12 Fig. 2a, which illustrates the four leaf nodes / bodies el to e4 sharing the current edge k shown as an arrow. Figure 13

[0175] • 4 neighboring bodies al to a4 having one corner as start point of the current edge k.

[0176] • 4 neighboring bodies bl to b4 having one corner as end point of the current edge k.

[0177] Figure 14 Fig. 2b and 2c respectively illustrate the 8 leaf nodes / bodies al to a4 and bl to b4 having one corner as start point or end point of the current edge k. The 12 occupancy bits associated with the 12 neighboring bodies e l , a m and b n are used to build the context information CI k (e l , a m , b n ). An embodiment of this building will be provided later.

[0178] The vertex flag s k ​Encoding is performed in a predetermined order, which is also referred to as the TriSoup edge order. According to an embodiment, as described above, certain information (such as the presence flag s k’ and the vertex position p k’ ) from edge k' that has been encoded / decoded relative to the TriSoup edge order can be used to construct context information CI k for encoding the current vertex flag s k , where k' < k. For example, when assuming that the three axes of a 3D space are labeled as x, y, and z, the TriSoup edge order can be the lexicographical order in the 6 - dimensional representation of the edges (x start , y start , z start , x end , y end , z end ), where (x start , y start , z start ) are the coordinates of the starting point of the edge, and (x end , y end , z end ) are the coordinates of the ending point. Using this order, the edge k' parallel to the current edge k and pointing to its starting point has always been encoded / decoded, as Figure 14 shown, Figure 14 the only edge k' is illustrated with a dashed arrow. The only edge k' is parallel to the current edge k represented by an arrow and points to the starting point of the current edge. The vertex V k’ on the only edge k' is illustrated, such that in the example of Figure 15 , for the only edge k', the vertex flag s k’ is true. In addition, the position p k’ of the vertex V k’ of the only edge k' is indicated, and as mentioned above, this position p k’ can be scalar information measured from the starting point of the only edge k'.

[0179] In addition, there are 4 edges k' perpendicular to the current edge and pointing to the starting point of the current edge k or starting from the starting point of the current edge k, as Figure 15 shown, Figure 15 the 4 perpendicular edges k' are illustrated with dashed arrows. The 4 perpendicular edges k' are orthogonal to the current edge k and point to the starting point of the current edge or start from the starting point of the current edge. Given the TriSoup edge order, Figure 16 some or all of the 4 perpendicular edges k' in Figure 16 have been encoded / decoded. As illustrated in perp(k) depends on the orientation of the current edge k. When applying the above mentioned dictionary TriSoup edge order, the number of already encoded / decoded vertical edges is

[0180] • If the current edge k is parallel to the x-axis, then N perp (k) = 4, as illustrated in Figure 16 (a).

[0181] • If the current edge k is parallel to the y-axis, then N perp (k) = 3, as illustrated in Figure 16 (b).

[0182] • If the current edge k is parallel to the z-axis, then N perp (k) = 2, as illustrated in Figure 13 (c).

[0183] According to embodiments of the present disclosure, for N perp (k) already encoded / decoded vertical edges k', the vertex flag s k’ and the vertex position p k’ of the existing vertex V k’ may be used to construct the context information CI k .

[0184] It should be noted that edges k' which are perpendicular to the current edge k and which point to the end point of the current edge or which start from the end point of the current edge are not encoded / decoded, thus these edges cannot naturally be used to construct the context information CI k .

[0185] Thus, according to the above described embodiments, one or more or all of the following can be used to construct the context information CI:

[0186] • The occupancy state of neighboring bodies, e.g. CI k (e l , a m , b n ).

[0187] • The presence of one or more vertices on already encoded / decoded edges, e.g. CI k ({s k’ |k' < k}).

[0188] • The position of a vertex, e.g. CI k ({p k’ |k' < k}).

[0189] Embodiments for constructing the context information CI k based on the occupancy of neighboring values are now described.

[0190] Contextual information CI based on occupancy of neighboring bodies k of construction

[0191] It has been found that the occupancy information associated with 12 neighboring bodies e l , a m , b n can be compressed to construct the context information CI k without significantly affecting the compression efficiency. Assuming N e is the number of bodies e l occupied out of 4, where not all bodies are unoccupied (otherwise the current edge is not a TriSoup edge), the following inequalities hold:

[0192] 1 ≤ N e ≤ 4

[0193] Assuming N a is the number of bodies a m occupied out of 4, and N b is the number of bodies b n occupied out of 4 (see e.g. Figure 15 ), the following inequalities hold:

[0194] 0 ≤ N a ≤ 4 and 0 ≤ N b ≤ 4

[0195] It has been found that constructing the context information CI e based on N a , N b , N k instead of the 12 bits representing the occupancy provides good compression performance. According to embodiments, a part of the context information CI k may be the following number

[0196] 5*4*N a +4*N b +(N e -1).

[0197] According to further embodiments, it has been found that the maximum occupancy is a good predictor of the value of the vertex flag s k such that this part of the context information CI k may be further compressed to 3 bits of information as follows:

[0198] 4*(1 if N a == 4 else 0)+2*(1 if N b == 4 else 0)+(1 if N e == 4 else 0).

[0199] According to further embodiments, the above methods can be combined such that e.g. the context information CIk A portion of it can be the following numbers

[0200] 8*max(0,N a -1)+2*max(0,N b -1)+(if N e If the value is 4, the value is 1; otherwise, the value is 0.

[0201] Context information CI is constructed based on the already coded neighboring edge k' k

[0202] Adjacent vertex V k’ quantity

[0203] As described above, according to the embodiment, context information CI can be constructed based on the already encoded / decoded adjacent edges k'. k For example, based on adjacent vertices V k’ The number of adjacent edges. A maximum of 5 adjacent edges have been encoded / decoded, as shown in the reference above. Figure 14 The described N perp (k) vertical sides, and the reference above Figure 14 The described parallel edge k' or unique edge k'. Assume TV(k) is the number of encoded / decoded edges k' on which TriSoup vertices exist, i.e., the number of vertex flags s that are true. k’ And assuming NTV(k) is the number of edges k' on which there is no TriSoup vertex, then the following equation holds:

[0204] N perp (k)+1=TV(k)+NTV(k).

[0205] Therefore, according to the embodiments, context information CI can be constructed based on the quantities TV(k) and NTV(k). k .

[0206] According to a further embodiment, it has been found that because TV(k) and vertex V k Position p k The correlation is weak, and using the value of NTV(k) alone provides sufficiently good compression results. Therefore, according to this embodiment, the context information CI can be constructed based solely on NTV(k). k .

[0207] Relative positions of adjacent vertices

[0208] According to a further embodiment, the relative positions of adjacent vertices can be used to construct context information (CI). k When considering the unique edge k' parallel to the current edge k and pointing to its origin, as referenced above... Figure 17 The TriSoup vertex belonging to this edge, as described, can be represented as V.par ,like Figure 17 As shown, except for the TriSoup vertex V that has been encoded / decoded on the unique edge k' par In addition, Figure 17 The quantization position p is also illustrated. par,Q It has been discovered that vertex V par Along the position p of the already encoded / decoded edge k' par,Q Does a vertex exist on the current edge k (vertex flag s)? k A good indicator of the value of (p). Vertex position can be quantized as p par,Q , such as in Figure 15 As illustrated in the diagram, quantization can be used to construct contextual information (CI). k The two-bit value.

[0209] According to a further embodiment, it has been found that the already encoded TriSoup V k’ At position p on the vertical side k’ (see Figure 18 ) is also the vertex marker s k A good indicator of the value. For example, such as Figure 18 As shown, one or more positions p on the vertical edge k' can be... k’ Simplified to a binary value, this indicates whether a vertex is "closer" or "farther" from the starting point of the current edge k. For example... Figure 18 As illustrated, if vertex V k If the distance from the endpoint of edge k' (the starting point of the current edge k) is less than a certain threshold or a predetermined threshold, then vertex V is... k The position p on the vertical side k' k’ The indication is the first position. Figure 17 In this context, the first position is referred to as the position "closest" to the current edge k. Let vertex V... k’ The position p that is more than the threshold distance from the starting point of the current edge k k’ The position "farthest" from the current edge k is called the position. The nearest vertex V is obtained. k’ The number N near And it is used to build context information CI k The following inequalities hold:

[0210] 0≤N near ≤4.

[0211] According to the embodiment, vertex V par The position p on the unique edge k' par (see Figure 19 This can also be considered for counting near vertices, and in such an embodiment, the following inequality holds:

[0212] 0≤Nnear ≤ 5.

[0213] According to embodiments, the number of near-vertices can be limited to obtain a so-called N near ’ of 2 bits, which is subject to the following inequality:

[0214] 0 ≤ N near ’ = min(3, N near ) ≤ 3.

[0215] According to embodiments, when building the context information CI k , all information about the position p k’ of the already encoded / decoded vertex V k’ may be combined. According to embodiments, part of the context information CI k may be

[0216] 4 * p par,Q + N near ’.

[0217] According to embodiments, information about the relative position of the neighboring vertices can be combined with the occupancy information of the neighboring bodies, and it has been found that, for example, the following 6-bit context word is a very strong predictor of the value of the vertex flag s k .

[0218] W = 16 * (N e - 1) + 4 * p par,Q + N near ’

[0219] According to further embodiments, the word W can be enriched with any of the above described context information, such as, for example, the information of (N a == 4), (N b == 4) and NTV(k).

[0220] According to still further embodiments, which relate to building the context information CI k based on the relative position of the vertex on the neighboring edges, instead of reducing the position p k’ to a binary value, it is also possible to reduce it to a ternary value, which indicates whether the position p k’ on the already encoded / decoded edge k’ is within one of several intervals defined along the vertical edge k’. For example, as illustrated in Figure 19 , the intervals comprise the position p k’The distance is divided into three intervals: far, middle, and near. The far interval begins at the starting point of edge k' and extends along a portion of the length of edge k'. The near interval ends at the ending point of edge k', which is the starting point of the current edge k, and extends along a portion of the length of edge k'. The middle interval is the length of edge k' between the near and far intervals. When considering... Figure 20 When the ternary value is shown in the figure, the following equation applies:

[0221] N near +N mid +N far =TV(k)≤5.

[0222] The number of near / middle / far vertices can be limited to obtain 2 bits of information N. near '、N mid 'and N far '.

[0223] 0≤N near =min(3,N) near )≤3

[0224] 0≤Nmid'=min(3,N mid )≤3

[0225] 0≤N far =min(3,N) far )≤3.

[0226] According to the embodiment, in constructing context information CI k At that time, it is possible to combine information about the already encoded / decoded vertex V. k’ Position p k’ Information. According to an embodiment, context information CI... k A portion may be

[0227] 32*N far '+8*N mid '+2*N near '+p par,Q .

[0228] According to an embodiment, when combining the occupancy information described above regarding adjacent volumes with ternary information regarding the relative positions of vertices on adjacent edges, it has been found that the following contextual terms are vertex identifiers. k A very strong predictor of values.

[0229] W = 32 * W1 + W2

[0230] in

[0231] W1 = 32 * N far '+8*N mid' + 2 * N near ' + p par,Q

[0232] W2= 8 * max(0, N a - 1) + 2 * max(0, N b - 1) + (N e == 4).

[0233] So far, the inventive concept has been described with reference to embodiments relating to methods for encoding / decoding geometry information (i.e. vertex flags) into / from a bitstream. According to further embodiments, the disclosure also provides apparatuses for encoding / decoding geometry information of a geometry graph of a point cloud into / from a bitstream.

[0234] Figure 21 Fig. illustrates a block diagram of an apparatus for encoding vertex flags s k according to an embodiment. The apparatus 300 comprises the following modules:

[0235] a context information construction module 302 for constructing context information for the current edge based on one or more or all of:

[0236] • occupancy information of neighboring cubes that neighbor the current edge,

[0237] • vertex position information of already encoded neighboring edges of the current edge, which are edges having a common point with the current edge.

[0238] a selection module 304 for selecting a coding probability of an entropy codec using the context information provided by the context information construction module 302.

[0239] an encoding module 306 for encoding the presence flag or the vertex flags s k of the current edge k through the entropy codec and using the selected coding probability.

[0240] Figure 20 Fig. illustrates a block diagram of an apparatus for decoding vertex flags s k according to an embodiment of the disclosure. The apparatus 400 comprises the following modules:

[0241] a context information construction module 402 for constructing context information for the current edge based on one or more or all of:

[0242] • occupancy information of neighboring cubes that neighbor the current edge,

[0243] • vertex position information of already decoded neighboring edges of the current edge, which neighboring edges are edges having a common point with the current edge.

[0244] a selection module 404 for selecting a coding probability of an entropy codec using the context information provided by the context information construction module 402.

[0245] a decoding module 406 for decoding the presence flag or the vertex flag s k of the current edge k by the entropy codec and using the selected coding probability.

[0246] According to embodiments, the context information construction module 302 / 402 described above with reference to Figure 21 and Figure 22 may comprise one or more sub-modules. Figure 20 Fig. illustrates a block diagram of an embodiment of a context information construction module 302 / 402 which can be used in an encoder of Figure 21 or a decoder of Figure 23 . The context information construction module can comprise one or more of the following sub-modules:

[0247] an occupancy information obtaining sub-module 500 for obtaining occupancy information of neighboring cubes e l , a m , b n which are adjacent to the current edge k.

[0248] a vertex presence obtaining sub-module 502 for obtaining vertex presence on already encoded / decoded neighboring edges k’.

[0249] a vertex position obtaining sub-module 504 for obtaining vertex positions along already encoded / decoded neighboring edges k’.

[0250] The present disclosure further provides in embodiments an electronic device, a computer readable storage medium and a computer program product.

[0251] While some aspects of the disclosed concepts have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where corresponding blocks of the apparatus represent a corresponding step of the method. Similarly, aspects described in the context of a method

[0252] Figure 23 is a block diagram illustrating an electronic device 900 according to an embodiment of the present disclosure.

[0253] Electronic devices are intended to represent a wide variety of form factors of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices can also represent various form factors of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections, and their functions, as well as the software implemented by the electronic device, are meant only to be examples and are not intended to limit the present disclosure to a particular implementation or implementations.

[0254] Reference ​ The device 900 includes a computing unit 901 for performing various appropriate actions and processes in accordance with computer program instructions stored in a read-only memory (ROM) 902 or loaded into a random access memory (RAM) 903 from a storage unit 908. In the RAM 903, various programs and data used for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0255] Components in the device 900 connected to the I / O interface 905 include an input unit 906 such as a keyboard, a mouse, an output unit 907 such as various types of displays, speakers, a storage unit 908 such as a magnetic disk, an optical disk, and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0256] The computing unit 901 can be formed from various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the image processing method described above can be performed. In some embodiments, the computing unit 901 can be configured to perform the image processing method in any other suitable manner (e.g., by means of firmware).

[0257] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuits, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0258] Program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions and / or operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0259] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0260] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0261] The systems and technologies described herein can be implemented on computing systems that include back-end components (e.g., as data servers), or middleware components (e.g., application servers), or front-end components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or computing systems that include such back-end components, middleware components, front-end components, or any combination thereof. The components of the system can be interconnected via any form of digital data communication medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0262] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system and solves the management difficulties and weak business scalability of traditional physical hosts and Virtual Private Servers (VPS). Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0263] It should be understood that the steps can be reordered, added, or deleted using the various forms of processes shown above. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0264] The specific embodiments described above do not limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for encoding geometric information of a point cloud into a bitstream (200), the point cloud being represented by a plurality of cubes (100), each occupied cube (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100). k ), and the geometric information includes signaling vertices (V) k The existence of the marker (s) k The method includes: For the current edge (k), Construct (S100) context information (CI) based on one or more or all of the following: k ): - Occupancy information of the adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k). - The vertex position information of the encoded adjacent edges (k') of the current edge (k), wherein the adjacent edges (k') are edges that share a common vertex with the current edge (k). Using the context information (CI) k To select the encoding probability of the (S102) entropy encoder, and The presence flag (s) of the current edge (k) is determined by the entropy encoder and using the selected encoding probability. k Encode (S104).

2. The method of claim 1, wherein the context information (CI) is constructed based on at least one of the following: k ): - A count of all occupied adjacent cubes (e1 to e4, a1 to a4, b1 to b4) belonging to a subset of adjacent cubes. - Whether all adjacent cubes (e1 to e4, a1 to a4, b1 to b4) belonging to a subset of adjacent cubes are occupied. The subset of adjacent cubes (e1 to e4, a1 to a4, b1 to b4) includes one or more or all of the following: - The adjacent cubes (e1 to e4) that share the current edge (k). - The adjacent cubes (a1 to a4) that have a corner that is the starting point of the current edge (k). - The adjacent cubes (b1 to b4) that have a corner that is the endpoint of the current edge (k).

3. The method of claim 1, wherein the context information (CI) is constructed based on at least one of the following: k ): - Encoded presence flags (s) associated with the adjacent edge (k') of the current edge (k). k’ The value of ) - Vertex (V) k’ The position (p) of the already encoded adjacent edge (k') of the current edge (k) k’ ), - Vertex (V) k’ The count (N) on the already encoded adjacent edges (k') of the current edge (k) near The vertex (V) k’ The distance from the current edge (k) is less than a predetermined threshold. - Vertex (V) k’ The count (N) on the already encoded adjacent edges (k') of the current edge (k) near N mid N far The vertex (V) k’ The distance from the current edge (k) is within a predetermined interval.

4. The method of claim 3, wherein the context information (CI) is constructed based on the following: k ): - The already encoded existence flag (s) k’ The number of true adjacent edges (k') (TV(k)) and the already encoded existence flag (s) k’ ) is the number of false adjacent edges (k') (NTV(k)), or - Only the already encoded existence flags (s) k’ ) represents the number of false adjacent edges (k') (NTV(k)).

5. The method of claim 3 or 4, wherein the context information (CI) is constructed. k This includes including the vertex (V) k’ The position (p) on the already encoded adjacent edge (k') k’ ) quantize to the ratio of the vertex (V) k’ The position (p) on the already encoded adjacent edge (k') k’ The precision of the encoding into the bit stream (200) is coarser.

6. The method according to any one of claims 1 to 4, wherein, in order to select the encoding probability of the entropy encoder, the method uses the following: - With the best binary encoder OBUF mechanism that features dynamic updates, or - Context-adaptive binary arithmetic coding (CABAC) mechanism.

7. The method according to any one of claims 1 to 4, wherein the point cloud is modeled using the TriSoup coding scheme.

8. A method for decoding geometric information of a point cloud from a bitstream (200), the point cloud being represented by a plurality of cubes (100), each occupied cube (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100). k ), and the geometric information includes signaling vertices (V) k The existence of the marker (s) k The method includes: For the current edge (k), Construct (S200) context information (CI) based on one or more or all of the following: k ): - Occupancy information of the adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k). - The vertex position information of the decoded adjacent edges (k') of the current edge (k), wherein the adjacent edges (k') are edges that share a common vertex with the current edge (k). Using the context information (CI) k To select the decoding probability of the (S202) entropy decoder, and The presence flag (s) of the current edge (k) is decoded (S204) using the entropy decoder and the selected decoding probability. k ).

9. The method of claim 8, wherein the context information (CI) is constructed based on at least one of the following: k ): - A count of all occupied adjacent cubes (e1 to e4, a1 to a4, b1 to b4) belonging to a subset of adjacent cubes. - Whether all adjacent cubes (e1 to e4, a1 to a4, b1 to b4) belonging to a subset of adjacent cubes are occupied. The subset of adjacent cubes (e1 to e4, a1 to a4, b1 to b4) includes one or more or all of the following: - The adjacent cubes (e1 to e4) that share the current edge (k). - The adjacent cubes (a1 to a4) that have a corner that is the starting point of the current edge (k). - The adjacent cubes (b1 to b4) that have a corner that is the endpoint of the current edge (k).

10. The method of claim 8, wherein the context information (CI) is constructed based on at least one of the following: k ): - The decoded existence flag (s) associated with the adjacent edge (k') of the current edge (k). k’ The value of ) - Vertex (V) k’ The position (p) of the already decoded adjacent edge (k') of the current edge (k) k’ ), - Vertex (V) k’ The count (N) on the decoded adjacent edges (k') of the current edge (k). near The vertex (V) k’ The distance from the current edge (k) is less than a predetermined threshold. - Vertex (V) k’ The count (N) on the decoded adjacent edges (k') of the current edge (k). near N mid N far The vertex (V) k’ The distance from the current edge (k) is within a predetermined interval.

11. The method of claim 10, wherein the context information (CI) is constructed based on the following: k ) - The already decoded existence flag (s) k’ The number of true adjacent edges (k') (TV(k)) and the existence flag that has been decoded (s) k’ ) is the number of false adjacent edges (k') (NTV(k)), or - Only the existence flags that have been decoded (s) k’ ) represents the number of false adjacent edges (k') (NTV(k)).

12. The method of claim 10 or 11, wherein the context information (CI) is constructed. k This includes including the vertex (V) k’ The position (p) on the already decoded adjacent edge (k') k’ ) quantize to the ratio of the vertex (V) k’ The position (p) on the already decoded adjacent edge (k') k’ The precision of the encoding into the bit stream (200) is coarser.

13. The method according to any one of claims 8 to 11, wherein, in order to select the decoding probability of the entropy decoder, the method uses the following: - The best binary decoder OBUF mechanism with dynamic updates, or - Context-adaptive binary arithmetic decoding CABAC mechanism.

14. The method according to any one of claims 8 to 11, wherein the point cloud is modeled using the TriSoup decoding scheme.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7 or 8 to 14.

16. An apparatus (300) for encoding geometric information of a point cloud into a bit stream (200), the point cloud being represented by a plurality of cubes (100), each occupied cube (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100). k ), and the geometric information includes signaling vertices (V) k The existence of the marker (s) k The device includes: Contextual Information (CI) k ) Build module (302), the context information (CI) k The building module (302) is configured to build context information (CI) for the current edge (k) based on one or more or all of the following: k ): - Occupancy information of the adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k). - The vertex position information of the encoded adjacent edges (k') of the current edge (k), wherein the adjacent edges (k') are edges that share a common vertex with the current edge (k). Selection module (304), the selection module (304) is configured to use the context information (CI) k ) Context information provided by the building module (CI) k To select the encoding probability of the entropy encoder, and Encoding module (306), which is configured to pass the entropy encoder and use the selected encoding probability to the presence flag (s) of the current edge (k). k Encode it.

17. The apparatus of claim 16, wherein the context information (CI) k The 302, 402 module includes one or more of the following submodules: - Occupancy information acquisition submodule (500), which is configured to acquire occupancy information of adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k). - Vertex Existence Acquisition Submodule (502), which is configured to acquire the existence of vertices on the already encoded adjacent edges (k'). - Vertex position acquisition submodule (504), which is configured to acquire vertex positions along the already encoded adjacent edges (k').

18. The apparatus of claim 17, wherein the vertex existence obtaining submodule (502) is configured to obtain an encoded existence flag (s) associated with the adjacent edge (k') of the current edge (k). k’ The value of ).

19. The apparatus of claim 17 or 18, wherein the vertex position obtaining submodule (504) is configured to obtain at least one of the following: - Vertex (V) k’ The position (p) of the already encoded adjacent edge (k') of the current edge (k) k’ ), - Vertex (V) k’ The count (N) on the already encoded adjacent edges (k') of the current edge (k) near The vertex (V) k’ The distance from the current edge (k) is less than a predetermined threshold. - Vertex (V) k’ The count (N) on the already encoded adjacent edges (k') of the current edge (k) near N mid N far The vertex (V) k’ The distance from the current edge (k) is within a predetermined interval.

20. An apparatus (400) for decoding geometric information of a point cloud from a bitstream (200), the point cloud being represented by a plurality of cubes (100), each occupied cube (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100). k ), and the geometric information includes signaling vertices (V) k The existence of the marker (s) k The device includes: Contextual Information (CI) k ) Build module (402), the context information (CI) k The building module (402) is configured to build context information (CI) for the current edge (k) based on one or more or all of the following: k ): - Occupancy information of the adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k). - The vertex position information of the decoded adjacent edges (k') of the current edge (k), wherein the adjacent edge (k') is the edge that shares a common point with the preceding edge (k). Selection module (404), the selection module (404) is configured to use the context information (CI) k The context information (CI) provided by the building module k To select the decoding probability of the entropy decoder, and Decoding module (406), configured to decode the presence flag (s) of the current edge (k) using the entropy decoder and with the selected decoding probability. k ).

21. The apparatus of claim 20, wherein the context information (CI) k The 302, 402 module includes one or more of the following submodules: - Occupancy information acquisition submodule (500), which is configured to acquire occupancy information of adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k). - Vertex Existence Acquisition Submodule (502), which is configured to acquire the existence of vertices on the decoded adjacent edges (k'). - Vertex position acquisition submodule (504), which is configured to acquire the vertex position along the already decoded adjacent edge (k').

22. The apparatus of claim 21, wherein the vertex existence obtaining submodule (502) is configured to obtain a decoded existence flag (s) associated with the adjacent edge (k') of the current edge (k). k’ The value of ).

23. The apparatus of claim 21 or 22, wherein the vertex position obtaining submodule (504) is configured to obtain at least one of the following: - Vertex (V) k’ The position (p) of the currently edge (k) on the already decoded adjacent edge (k') k’ ), - Vertex (V) k’ The count (N) on the decoded adjacent edges (k') of the current edge (k). near The vertex (V) k’ The distance from the current edge (k) is less than a predetermined threshold. - Vertex (V) k’ The count (N) on the decoded adjacent edges (k') of the current edge (k). near N mid N far The vertex (V) k’ The distance from the current edge (k) is within a predetermined interval.

Images