Apparatus for encoding and decoding vertex position of point cloud and data stream comprising vertex position

By constructing context information and selecting appropriate entropy codecs, the problem of low vertex position encoding efficiency in point cloud data compression is solved, and a more efficient compression effect is achieved.

CN118947108APending Publication Date: 2024-11-12BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380029706.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-07
Filing Date
2023-02-22
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, the compression efficiency of point cloud data is not high, especially when using TriSoup data, the encoding method of vertex positions is bypass encoding, resulting in an inefficient compression effect.

Method used

By building context information, selecting the appropriate entropy codec, using context information to encode and decode vertex positions in the geometry of the point cloud, avoiding bypass encoding, and adopting dynamically updated optimal binary codecs or context-adaptive binary Arithmetic codec to improve the compression efficiency of vertex positions.

Benefits of technology

More efficient compression of point cloud data is achieved, and the compression effect of vertex positions is improved through more accurate probability estimation and encoding, and the inefficiency of bypass encoding is avoided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118947108A_ABST
    Figure CN118947108A_ABST
Patent Text Reader

Abstract

A method of encoding geometry information of a geometry of a point cloud into a bitstream is described. The point cloud is represented by a plurality of cubes, and the occupied cubes are modeled by one or more triangles. The at least one triangle has at least one respective vertex on the sides of the occupied cube. The geometry information includes vertex positions of one or more vertices. The method comprises: for a vertex on a current edge, constructing (S100) context information based on one or more or all of: occupancy information of an adjacent cube adjoining the current edge, vertex position information of an encoded adjacent edge of the current edge, the adjacent edge (k ') is an edge having a common point with the current edge, using (S102) the context information to select a codec probability of the entropy codec, and encoding (S104), by the entropy codec and using the selected codec probability, a vertex position of the vertex on the current edge.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority from European patent application No. “22167198.5” filed on April 7, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to data compression, and more particularly to methods and apparatus for encoding geometric information of a point cloud geometry. Embodiments of the present disclosure relate to methods and apparatus for encoding and decoding vertex positions of triangles on the sides of a cube representing a portion of a point cloud, and data or a bitstream in which the vertex positions are encoded. Background Art

[0004] Data compression is used in communications and computer networks to efficiently store, transmit, and reproduce information. For example, point clouds have recently gained attention as a format for representing three-dimensional (3D) data due to their versatility in representing all types of 3D objects or scenes. Therefore, many use cases can be addressed with point clouds, including

[0005] Film post-production,

[0006] Real-time 3D immersive telepresence or VR / AR (virtual reality / augmented reality) applications,

[0007] Free viewpoint video, e.g. for watching sports,

[0008] Geographic Information Systems, also known as Cartography,

[0009] Cultural heritage, such as scanning rare objects and storing them in digital form,

[0010] Autonomous driving, including 3D mapping of the environment and real-time LiDAR data acquisition (LiDAR: Light Detection and Ranging = a method of measuring distance (ranging) by illuminating a target with laser light and measuring the reflected light with a sensor).

[0011] Therefore, there is a need for methods and apparatus that provide more efficient and / or more effective compression of point cloud data. Summary of the invention

[0012] The present disclosure provides a method for encoding geometric information of a geometric figure of a point cloud into a bit stream, wherein the point cloud is represented by a plurality of cubes, the occupied cubes are modeled by one or more triangles, at least one triangle has at least one corresponding vertex on an edge of the occupied cube, and the geometric information includes vertex positions of the one or more vertices, the method comprising: for a vertex on a current edge,

[0013] Build contextual information based on one, more, or all of the following:

[0014] - occupancy information of neighboring cubes adjacent to the current edge,

[0015] - The vertex position information of the encoded adjacent edge of the current edge, which is the edge that has a common point with the current edge,

[0016] Use context information to select the encoding and decoding probabilities of the entropy codec, and

[0017] The vertex position on the current edge is encoded by the entropy codec using the selected codec probability.

[0018] The present disclosure provides a method for decoding geometric information of a point cloud from a bit stream, wherein the point cloud is represented by a plurality of cubes, the occupied cubes are modeled by one or more triangles, at least one triangle has at least one corresponding vertex on an edge of the occupied cube, and the geometric information includes vertex positions of the one or more vertices, the method comprising: for a vertex on a current edge,

[0019] Build contextual information based on one, more, or all of the following:

[0020] - occupancy information of neighboring cubes adjacent to the current edge,

[0021] - The vertex position information of the decoded adjacent edges of the current edge, which are the edges that have common points with the current edge,

[0022] Use context information to select the encoding and decoding probabilities of the entropy codec, and

[0023] The vertex position on the current edge is decoded by the entropy codec using the selected codec probability.

[0024] Optionally, the context information is constructed based on one or more or all of the following:

[0025] - the count of all occupied neighboring cubes that belong to a subset of neighboring cubes,

[0026] - whether all adjacent cubes belonging to a subset of adjacent cubes are occupied,

[0027] Where the subset of adjacent cubes includes one or more or all of the following:

[0028] - neighboring cubes that share the current edge,

[0029] - the adjacent cube that has a corner that is the start point of the current edge,

[0030] - The adjacent cube that has a corner that is the endpoint of the current edge.

[0031] Optionally, the context information is constructed based on one or more or all of the following:

[0032] - the value of the encoded / decoded presence flag associated with the neighboring edges of the current edge,

[0033] - the position of the vertex on the encoded / decoded adjacent edges of the current edge,

[0034] - the count of vertices on the encoded / decoded neighboring edges of the current edge whose distance from the current edge is below a predefined threshold,

[0035] -The count of vertices on the encoded / decoded adjacent edges of the current edge whose distance from the current edge is within a predetermined interval.

[0036] Optionally, context information is constructed based on

[0037] - the number of adjacent edges for which the presence flag has been encoded / decoded as true, and the number of adjacent edges for which the presence flag has been encoded / decoded as false, or

[0038] - The number of adjacent edges that have only been encoded / decoded for which the presence flag is false.

[0039] Optionally, constructing the context information includes quantizing the position of the vertex on the already encoded / decoded adjacent edge to a coarser precision than encoding the position of the vertex on the already encoded / decoded adjacent edge into the bitstream.

[0040] Optionally, the vertex position is represented by one or more bits, context information is constructed for each bit of the vertex position, and each bit of the vertex position is encoded / decoded by a binary entropy codec using a coding / decoding probability determined based on the context information of the corresponding bit.

[0041] Optionally, the context information of the bits of the vertex position is further constructed based on previously encoded and decoded bits.

[0042] Optionally, to select the encoding and decoding probability of the entropy codec, the method uses the following

[0043] - an optimal binary codec OBUF mechanism with dynamic update, or

[0044] -Context-Adaptive Binary Arithmetic Codec (CABAC) mechanism.

[0045] Optionally, the point cloud is modeled using the TriSoup encoding and decoding scheme.

[0046] The present disclosure provides a non-transitory computer-readable storage medium, comprising instructions, which, when executed by a computer, cause the computer to perform the method of the present invention.

[0047] The present disclosure provides an apparatus for encoding geometric information of a geometric figure of a point cloud into a bit stream, wherein the point cloud is represented by a plurality of cubes, the occupied cubes are modeled by one or more triangles, at least one triangle has at least one corresponding vertex on an edge of the occupied cube, and the geometric information includes vertex positions of the one or more vertices, the apparatus comprising:

[0048] A context information building module, the context information building module is configured to build context information for the current edge based on one or more or all of the following:

[0049] - occupancy information of neighboring cubes adjacent to the current edge,

[0050] - The vertex position information of the encoded adjacent edges of the current edge, where the adjacent edges are the edges that have common points with the current edge,

[0051] a selection module configured to select a codec probability of an entropy codec using the context information provided by the context information building module, and

[0052] The encoding module is used to encode the vertex position of the vertex on the current edge by using an entropy codec and a selected coding probability.

[0053] The present disclosure provides an apparatus for decoding geometric information of a geometric figure of a point cloud from a bitstream, the point cloud being represented by a plurality of cubes, the occupied cubes being modeled by one or more triangles, at least one triangle having at least one corresponding vertex on an edge of the occupied cube, and the geometric information comprising vertex positions of the one or more vertices, the apparatus comprising:

[0054] A context information building module, the context information building module is configured to build context information for the current edge based on one or more or all of the following:

[0055] - occupancy information of neighboring cubes adjacent to the current edge,

[0056] - The vertex position information of the decoded adjacent edges of the current edge. The adjacent edges are the edges that have common points with the current edge.

[0057] a selection module configured to select a codec probability of an entropy codec using the context information provided by the context information building module, and

[0058] A decoding module is configured to decode the vertex position of the vertex on the current edge by using an entropy codec and the selected codec probability.

[0059] Optionally, the context information building module includes one or more or all of the following sub-modules:

[0060] - an occupancy information obtaining submodule, the occupancy information obtaining submodule being configured to obtain occupancy information of adjacent cubes adjacent to the current edge,

[0061] - a vertex existence obtaining submodule, the vertex existence obtaining submodule being configured to obtain the existence of vertices on adjacent edges that have been encoded / decoded,

[0062] - A vertex position obtaining submodule, which is configured to obtain vertex positions along adjacent edges that have been encoded / decoded.

[0063] The present disclosure provides a data stream in which geometric information of a geometric figure of a point cloud is encoded, the point cloud is represented by a plurality of cubes, the occupied cube is modeled by one or more triangles, at least one triangle has at least one corresponding vertex on an edge of the occupied cube, and the geometric information includes vertex positions of the one or more vertices, the data stream comprising: for a vertex on a current edge,

[0064] The vertex position on the current edge is encoded into the data stream through the entropy codec and the codec probability is selected using context information.

[0065] The context information is constructed based on one, more, or all of the following:

[0066] - occupancy information of neighboring cubes adjacent to the current edge,

[0067] -Vertex position information of the encoded adjacent edges of the current edge, which are edges that have common points with the current edge.

[0068] The technical solution provided according to the embodiments of the present disclosure has the following beneficial effects.

[0069] Although in the prior art, compression of TriSoup data is not very efficient, embodiments of the present disclosure provide improved compression of the positions of vertices on the edges of occupied volumes associated with leaf nodes. In the prior art, vertex positions are encoded into a bitstream by simply pushing bits into the bitstream without compression, which is also known as bypass encoding. The present disclosure introduces the use of better context information to select a context or entropy codec (such as by applying, for example, an optimal binary codec (so-called OBUF) mechanism with dynamic update (update on the fly)), and results in better estimates of the probabilities of vertex positions, which in turn results in better compression of the positions by the entropy coder.

[0070] Embodiments of the present disclosure obtain relevant context information related to the position of the vertex.

[0071] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood based on the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings are illustrative and serve to explain the present disclosure and are not to be considered to limit the present disclosure to the illustrated embodiments.

[0073] Figure 1 illustrates TriSoup vertices along the edges of volumes associated with leaf nodes of an octree representation of point cloud geometry;

[0074] Figure 2 The figure shows a body including two TriSoup triangles, each of which has a vertex on an edge of the body;

[0075] Figure 3 illustrates a volume including a TriSoup triangle, two of the vertices of the TriSoup triangle being on an edge of the volume and one of the vertices of the TriSoup triangle being inside the volume;

[0076] Figure 4 illustrates the process for selecting triangles to be built inside a leaf node, where Figure 4 (a) shows the triangle Figure 2 The body, Figure 4 (b) illustrates a triangular 2D surface using a vertical projection of the volume, and Figure 4 (c) illustrates a triangular 2D surface using vertical and horizontal projections of the volume;

[0077] Figure 5 Illustration of a triangle with two TriSoup triangles Figure 2The two TriSoup triangles are constructed under the assumption that the horizontal axis is the principal axis;

[0078] Figure 6 Illustration of ray tracing of rendering TriSoup triangles into a decoded point cloud;

[0079] Figure 7 A flowchart of a method for encoding geometric information of a point cloud geometry into a bitstream according to an embodiment of the present disclosure is illustrated;

[0080] Figure 8 A flowchart of a method for decoding geometric information of a point cloud from a bitstream according to an embodiment of the present disclosure is illustrated;

[0081] Fig. 9 illustrates a data flow according to an embodiment of the present disclosure;

[0082] Fig.10 A flowchart of an encoding method according to an embodiment of the present disclosure is illustrated;

[0083] Fig.11 A flowchart of a decoding method according to an embodiment of the present disclosure is illustrated;

[0084] Fig.12 The diagram shows 4 leaf nodes / bodies that share the current edge k (illustrated by arrows);

[0085] Fig.13 8 leaf nodes / bodies are shown, one corner of which is the starting point or end point of the current edge k (shown by an arrow);

[0086] Fig.14 A unique edge k' (illustrated by a dashed arrow) is illustrated that is parallel to the current edge k (illustrated by an arrow) and points to its starting point;

[0087] Fig.15 Four vertical edges k' (illustrated by dashed arrows) are illustrated that are orthogonal to the current edge k (illustrated by arrows) and point to or start from its starting point;

[0088] Fig.16 Graph showing the number N of vertical edges k' that have been encoded / decoded depending on the direction of the current edge k perp (k);

[0089] Fig.17 Figure 2.3 shows the encoded / decoded TriSoup vertex V. par and its quantized position p on the only edge k' that has been encoded / decoded par,Q ;

[0090] Fig.18illustrates the positions of vertices belonging to a perpendicular, already encoded / decoded edge k', "near" versus (vs.) "far";

[0091] Fig.19 The positions of the vertices belonging to the vertical encoded / decoded edge k' are illustrated as "near" vs. "middle" vs. "far";

[0092] Fig. 20 FIG. 1 is a diagram illustrating a method for encoding a vertex position p according to an embodiment. k A block diagram of a device / encoder;

[0093] Fig.21 FIG. 1 is a diagram illustrating a method for decoding a vertex position p according to an embodiment. k A block diagram of a device / decoder;

[0094] Fig. 22 The diagram shows the Fig. 20 The encoder or Fig.21 A block diagram of an embodiment for implementing a context information building module in a decoder of ; and

[0095] Fig.23 A block diagram illustrating an electronic device configured to implement an image processing method according to an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0096] The illustrative embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to aid understanding, which should be considered as merely illustrative. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0097] In this disclosure, the terms "node", "volume", and "sub-volume" may be used interchangeably. It should be understood that a node is associated with a volume or a sub-volume. A node is a specific point on a tree, which may be an internal node or a leaf node. A volume or sub-volume is a bounded physical space represented by a node. In some cases, the term "volume" may be used to refer to the largest bounded space defined to contain a point cloud. For the purpose of constructing a tree structure of interconnected nodes for encoding and decoding point cloud data, a volume may be recursively divided into sub-volumes.

[0098] In the present disclosure, the term “and / or” is intended to cover all possible combinations and subcombinations of the listed elements, including any one element, any subcombination or all of the elements listed individually, and does not necessarily exclude additional elements.

[0099] In the present disclosure, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any one of the listed elements alone, any subcombination, or all of the elements, without necessarily excluding any additional elements, and also not necessarily requiring all elements.

[0100] In the present disclosure, as will be apparent from the context of the described embodiments involving encoding / decoding geometric information into / from a bitstream, the term "coding" refers to either "encoding" or "decoding". Likewise, the term "coder" refers to either "encoder" or "decoder".

[0101] A point cloud is a collection of points in a three-dimensional coordinate system. These points are usually intended to represent the outer surface of one or more objects. Each point has a position or orientation in a three-dimensional coordinate system. The position can be represented by three coordinates (X, Y, Z), which can be Cartesian coordinates or any other coordinate system. These points can have other associated attributes, such as color, which can also be a three-component value such as R, G, B or Y, Cb, Cr in some cases. Other associated attributes may include transparency, reflectivity, normal vectors, etc., depending on the desired application of the point cloud data.

[0102] Point clouds can be static or dynamic. For example, a detailed scan or drawing of an object or terrain can be static point cloud data. LiDAR-based scans of an environment for machine vision purposes can be dynamic in that the point cloud at least potentially changes over time, e.g., with each successive scan of a volume. Thus, a dynamic point cloud is a time-ordered sequence of point clouds.

[0103] As described above, point cloud data can be used for a variety of applications or use cases, including conservation, such as scanning historical or cultural objects, mapping, machine vision (e.g., for autonomous or semi-autonomous cars), and virtual or augmented reality systems. Dynamic point cloud data for applications such as machine vision can be very different from static point cloud data such as used for conservation purposes. For example, automotive vision typically involves relatively small resolution, colorless, highly dynamic point clouds acquired at a high capture frequency via LiDAR or similar sensors. Such point clouds are not intended for human consumption or viewing, but rather for machine object detection / classification in decision-making processes. For example, a typical LiDAR frame contains on the order of tens of thousands of points, while high-quality virtual reality applications require millions of points. It can be expected that higher resolution data will be required over time as computing speeds increase and new applications or use cases are discovered.

[0104] In other words, a point cloud is a collection of points located in 3D space, optionally with additional values ​​attached to each point. These additional values ​​are often referred to as point attributes. Thus, a point cloud can be considered a combination of geometry (the 3D position of each point) and attributes. Attributes can be, for example, three-component color, material properties (such as reflectivity), and / or a two-component normal vector of a surface associated with a point. Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, the mentioned LiDAR, scanners, or they can be computer generated (for example in movie post-production use cases). Depending on the use case, point clouds may have thousands to billions of points for cartography applications.

[0105] The raw representation of point clouds requires a very high number of bits per point, at least a dozen bits per spatial component X, Y, or Z, and optionally more bits for one or more attributes, such as three times 10 bits for color. Therefore, practical deployment of point cloud-based applications or use cases requires compression techniques that can store and distribute point clouds with reasonable storage and transmission infrastructure. In other words, while point cloud data is useful, the lack of effective and efficient compression (i.e., encoding and decoding processes) may hinder its adoption and deployment. A particular challenge in encoding and decoding point clouds is encoding and decoding the geometry of the point cloud, which does not arise in the case of other data compression (such as audio or video), and the tendency of point clouds to be sparsely distributed makes it more challenging to effectively encode and decode the locations of the points.

[0106] Until recently, point cloud compression (also known as PCC) had not been addressed by the mass market and no standardized point cloud codec was available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, launched work items on point cloud compression, which resulted in two standards, namely:

[0107] MPEG-I Part 5 (ISO / IEC 23090-5), also known as Video-based Point Cloud Compression V-PCC,

[0108] MPEG-I Part 9 (ISO / IEC 23090-9), also known as Geometry-based Point Cloud Compression G-PCC.

[0109] The first version of the V-PCC standard and the first version of the G-PCC standard were completed in 2020 and 2022, respectively.

[0110] The V-PCC coding method compresses point clouds by performing multiple projections of 3D objects to obtain two-dimensional (2D) patches that are packed into images or, when processing moving point clouds, into videos. The images or videos are then compressed using existing image / video codecs, allowing the use of already deployed image and video solutions. By its nature, V-PCC is only effective on dense and continuous point clouds, because image / video codecs cannot compress non-smooth patches in the case of non-smooth patches obtained from projections of sparse geometric data collected, such as LiDAR.

[0111] The G-PCC codec has two schemes for compressing geometric graphics.

[0112] The first approach is based on an occupancy tree representation of the point cloud geometry, for example, by an octree representation, a quadtree representation, or a binary tree representation. In a tree-based structure, the bounding three-dimensional volume of the point cloud is recursively divided into sub-volumes. The nodes of the tree correspond to sub-volumes. The decision whether to further divide a sub-volume can be based on the resolution of the tree and / or whether any points are contained in the sub-volume. A leaf node can have an occupancy flag indicating whether its associated sub-volume contains a point. The split flag can signal whether a node has child nodes, i.e., whether the current volume has been further split into sub-volumes. A commonly used tree structure is an octree. In this structure, the volume / sub-volume is a cube, and each split of a sub-volume results in eight other sub-volumes / sub-cubes. Another commonly used tree structure is a KD-tree, in which a volume (such as a cube) is recursively split into two by a plane orthogonal to one of the axes. An octree is a special case of a KD-tree, in which the volume is divided by three planes, each of which is orthogonal to one of the three axes.

[0113] In other words, occupied nodes are split until a certain size is reached, and the occupied leaf nodes provide the locations of the points, usually in the centers of these nodes. By using neighbor-based prediction techniques, a high level of compression can be obtained for dense point clouds. Sparse point clouds are also handled by directly encoding the locations of points within nodes with non-minimum sizes, by stopping the tree construction when only isolated points exist in the node. This stopping technique is also known as Direct Codec Mode (DCM).

[0114] The second scheme is based on a prediction tree, where each node represents the 3D position of a point and the relationship between nodes is the spatial prediction from the parent node to the child node. This method can only process sparse point clouds and offers the advantages of lower latency and simpler decoding compared to using occupancy trees. However, the compression performance is only slightly better than the first scheme, but the encoding is complicated due to the need to intensively find the best predictor in a long list of potential predictors when building the prediction tree.

[0115] In both schemes, attribute encoding and decoding (i.e., attribute encoding and attribute decoding) are performed after the entire geometry encoding and decoding, resulting in a two-pass encoding and decoding process. Low latency can be achieved by using slices that decompose the 3D space into independently encoded and decoded sub-volumes without the need for prediction between sub-volumes. However, this can severely affect compression performance when many slices are used.

[0116] A particularly interesting use case is the transmission of dynamic AR / VR point clouds, where dynamic means that the point cloud evolves over time. In addition, AR / VR point clouds are typically local 2D because they represent the surface of an object most of the time. Therefore, AR / VR point clouds are highly connected (also called dense) because points are rarely isolated but have many neighbors. Therefore, dense or solid point clouds represent continuous surfaces with a resolution that allows the volumes associated with the points (also called small cubes or voxels) to touch each other without showing any visible holes on the surface. Such point clouds as mentioned above are typically used in AR / VR environments and are viewed by end users through devices such as TVs, smartphones, or headsets including AR / VR glasses. The point cloud can be transmitted to the device or can be stored locally. Many AR / VR applications use mobile point clouds, rather than static point clouds, which change over time. Therefore, the amount of data may be large and needs to be compressed. For example, when an octree representation of the geometry of the above point cloud is applied, lossless compression can achieve slightly less than 1 bit per point (or 1bpp). However, this may not be sufficient for real-time transmission, which may involve millions of points per frame at frame rates as high as 50 frames per second (fps), resulting in hundreds of megabytes of data per second.

[0117] Therefore, lossy compression schemes can be used, usually with the requirement to maintain acceptable visual quality by providing compression sufficient to fit the compressed data within the bandwidth available in the transmission channel, while maintaining real-time transmission of the frames. In many applications, bit rates as low as 0.1bpp already make real-time transmission possible, which means that with lossy compression, the compression of the point cloud is ten times higher than when a lossless codec scheme is applied.

[0118] Codecs based on MPEG-I Part 5 (ISO / IEC 23090-5) or V-PCC can achieve such low bitrates by using lossy compression with a video codec that compresses 2D frames obtained from the projection of a point cloud onto a plane. The geometry is represented by a series of projected patches assembled into a frame, where each patch is a small local depth map. However, VPCC is not general and is limited to narrow types of point clouds that do not exhibit local complex geometry (such as trees or hair, etc.), because the obtained projected depth maps are not smooth enough to be effectively compressed by the video codec.

[0119] On the other hand, pure 3D compression techniques can handle any type of point cloud. For example, G-PCC may in the future provide lossy compression that also allows to compress dense point clouds as well as V-PCC intra coding, while maintaining the versatility of G-PCC to handle any type of point cloud, such as dense point clouds, point clouds obtained by LiDAR or point clouds representing 3D maps. In order to implement this G-PCC mechanism, the so-called TriSoup codec scheme can be applied on the first layer based on an octree. Currently, the TriSoup codec scheme is being discussed in the standardization working group JTC1 / SC29 / WG7 of ISO / IEC. When considering the possibility of obtaining a lossy scheme from G-PCC, there are basically three ways to obtain a lossy scheme through the octree representation used by the 3-PCC codec, namely

[0120] Downsampling + (lossless) encoding + re-upsampling

[0121] Modify voxels locally on the encoder side

[0122] Local modeling of point clouds.

[0123] The first approach basically consists of downsampling the entire point cloud to a smaller resolution, losslessly encoding and decoding the downsampled point cloud, and then upsampling after decoding. There are many upsampling schemes, such as super-resolution, artificial intelligence AI, or learning-based 3D post-processing, which can provide good peak signal-to-noise ratio (PSNR) results when the downsampling is not too aggressive (for example, no more than a factor of two in each direction). However, even if the indicator shows a good PSNR, the visual quality is still controversial and not well controlled.

[0124] The second approach allows the encoder to locally adjust the point cloud so that the encoding and decoding of the octree requires a lower bitrate. For this purpose, the points can be slightly moved in order to obtain occupancy information that can be better predicted by neighboring nodes, resulting in lossless encoding of the modified octree at a lower bitrate. Unfortunately, however, this approach only results in a small bitrate reduction.

[0125] A third approach is to encode and decode the geometry using a tree (such as an octree) to a certain resolution, for example to N×N×N blocks, where N can be, for example, 4, 8 or 16. This tree is then encoded and decoded using a lossless scheme such as the G-PCC scheme. The tree itself does not require a high bit rate because it does not go to the deepest depth and has only a small number of leaf nodes compared to the number of points in the point cloud. Then, in each N×N×N block, the point cloud is modeled by a local model. This model can be a mean plane or a set of triangles as in the TriSoup codec scheme mentioned above, which is now described in more detail.

[0126] The TriSoup encoding scheme models the point cloud locally by using a set of triangles without explicitly providing connectivity information, which is why its name is derived from the term "soup of triangles". As mentioned above, each N×N×N block defines a volume associated with a leaf node, and in each N×N×N block or volume, the point cloud is locally modeled using a set of triangles, where the vertices of the triangles are encoded along the edges of the volume associated with the leaf nodes of the tree.

[0127] The portion of the point cloud enclosed by volume 100 is modeled by at least one triangle, which is located between edges 1001 and 100 12 There is at least one vertex on one of them. Figure 1 In the example of , five vertices 1 to 5 are illustrated, where vertices 1 to 4 are located on edges 1002, 1001, 1008, and 1007, respectively. Vertex 5 is not located on any edge but is located within volume 100. In other words, in order to locally model the point cloud, one or more triangles are used, and a triangle may have at least one vertex located on an edge of the volume, such as vertices 1 to 4, while other triangles may have one or two vertices (such as vertex 5) of the vertices of the other triangle that are not located on the edge but are within volume 100, or at the boundary of volume 100.

[0128] Vertices on the edges are shared between leaf nodes with common edges, which means that each edge belonging to at least one leaf node encodes and decodes at most one vertex, thus ensuring the continuity of the model through the leaf nodes. The encoding and decoding of TriSoup vertices requires two pieces of information for each edge:

[0129] a vertex flag indicating whether a TriSoup vertex exists on the edge, also referred to herein as a presence flag, and

[0130] If the vertex exists, the vertex position along the edge.

[0131] Thus, the coded data includes the octree data plus the TriSoup data. For example, the vertex flags can be coded by an adaptive binary arithmetic codec that uses a specific context to code the vertex flags, and the length is N=2 s The positions of the vertices on the edge of are encoded and decoded with unit precision by pushing s bits into the bitstream, i.e. by bypassing / non-entropy encoding and decoding s bits.

[0132] Figure 2 The figure shows a volume 100 associated therewith including two TriSoup triangles 102 and 104, wherein the vertices 1 to 4 of the two TriSoup triangles 102 and 104 are located at the edges of the volume 100 (see Figure 1 ) 1002, 1001, 1008 and 1004. Triangle 102 includes vertices 1, 2 and 3, while triangle 104 includes vertices 1, 3 and 4. Therefore, if there are at least three vertices on the edge of volume 100, a triangle can be constructed. Naturally, inside volume 100 associated with a leaf node, except Figure 2 Any other triangle combination than the triangle shown is possible. In addition, one or more triangles inside body 100 do not necessarily have all vertices on the edge of body 100. Instead, one or two vertices of the triangle may be located anywhere inside body 100. Figure 3 A body 100 is shown, which includes vertices 1 to 4 located on corresponding sides of the body 100 (eg, Figure 2 ) and an additional vertex 5 located within volume 100. Inside volume 100, four triangles 106 to 112 are constructed from TriSoup vertices 1 to 5, where first triangle 106 includes vertices 1, 2, and 5, second triangle 108 includes vertices 2, 3, and 5, third triangle 110 includes vertices 3, 5, and 5, and fourth triangle 112 includes vertices 1, 4, and 5.

[0133] The triangles to be constructed inside the volume 100 are based on the following three-step process, including:

[0134] 1. Determine the dominant direction along one of the three axes.

[0135] 2. Sort the TriSoup vertices according to the dominant direction.

[0136] 3. Build a triangle based on the ordered list of vertices.

[0137] Figure 4 The diagram shows the Figure 2 The process of constructing triangles inside the volume 100 associated with the leaf node, Figure 4 (a) shows again Figure 2 The volume is 100, but there are no triangles. Figure 4 (b) and Figure 4 (c) illustrates the process on two axes, the vertical or z-axis ( Figure 4 (b)) and the horizontal or x-axis ( Figure 4 (c)).

[0138] like Figure 4 As shown in (c), the first test along the vertical axis is performed by projecting the volume or cube 100 and the TriSoup vertices vertically on the 2D plane, i.e., testing from the top. The vertices are then sorted in clockwise order relative to the center of the projected node 114, which is a square in the illustrated example. Figure 4As shown in (c), triangles are constructed based on ordered vertices according to fixed rules. Figure 4 In the example of , four vertices are involved, and triangles 102, 104 are systematically constructed to include vertices 1, 2 and 3 of the first triangle and vertices 1, 3 and 4 of the second triangle. When there are only three vertices, the only possible triangle is the one that includes vertices 1, 2 and 3, and when there are five vertices, a fixed rule can be used to construct triangles that include vertices (1, 2, 3), (1, 3, 5), and (4, 5, 1), etc. This can be repeated for up to 12 vertices.

[0139] When from Figure 4 A second test along the horizontal axis is performed by projecting the cube 100 and the TriSoup vertices horizontally on the 2D plane, yielding Figure 4 (b) shows the projection 116. When the vertices are ordered in clockwise order relative to the center of the projection node 100, the triangles 102 and 104 include vertices 1, 2, and 3 of the first triangle and vertices 1, 3, and 4 of the second triangle, as shown in FIG. Figure 4 (b) as shown.

[0140] from Figure 4 It can be seen that the vertical projection ( Figure 4 (c)) shows the maximum total 2D surface of the triangle, such that the principal axis is chosen to be the vertical axis or z-axis, and according to Figure 4 The order of vertical projections shown in (c) yields the TriSoup triangles to be constructed, which in turn yields Figure 2 The triangle inside the body depicted. Note that when considering the horizontal axis as the principal axis, this leads to Figure 5 Different configurations of triangles within the depicted volume 100, Figure 5 A volume 100 is shown in which triangles 102, 104 are arranged in a manner similar to that shown in FIG. Figure 4 (b) The vertex order shown is constructed.

[0141] The principal axes are appropriately selected by maximizing the projected surface, thus enabling continuous reconstruction of point clouds without holes.

[0142] The rendering of TriSoup triangles is performed by ray tracing, and the set of all points rendered by ray tracing will form the decoded point cloud. Figure 6 The diagram shows a method for rendering a grid including vertices 1, 2, and 3. Figure 2 Ray tracing of the TriSoup triangle 102. Rays (such as Figure 6 The light ray 118 in the image is parallel to the axis (e.g. Figure 6The light ray 118 is emitted in the direction of the z-axis in the rendering. The origin of the light ray is a point of voxelized coordinates with an integer precision corresponding to the sampling position required for rendering. The intersection 120 of the light ray 118 with the triangle 102 is then voxelized, i.e., rounded to the nearest point at the desired sampling position, and added to the list of rendering points. After the TriSoup encoding and decoding scheme is applied to all leaf nodes, i.e., after the triangles are constructed and the intersections are obtained by ray tracing, the copies of the same points in the lists of all rendering points are discarded (i.e., only one voxel is retained among all voxels sharing the same 3D position), thereby obtaining a set of decoded unique points.

[0143] When the TriSoup encoding and decoding scheme is applied to locally model a point cloud by using a set of triangles for each leaf node or volume in the manner described above, so-called TriSoup data is also provided. For example, the TriSoup data includes information about the vertices of the corresponding triangles of the volume. However, in the prior art, the compression of the TriSoup data is not efficient. The vertex V k The position p k can be quantized so that the length of the edge is divided by two 2 of the quantization interval Nb The power of N, so that the position is represented by the integer N b In the prior art, for 1 and N b i between each bit B i Both are encoded into the bitstream by simply pushing bits into the bitstream without compression, which is also called bypass coding. However, the bypass bits B i It is very inefficient in compression and this is one of the main reasons why the TriSoup coding scheme has not been adopted in the G-PCC point cloud codec.

[0144] The present disclosure solves this problem and provides a method that allows for a compressed representation of a vertex V. k Position p along edge k k The method of obtaining information of the vertex V based on the context information, that is, according to the present disclosure, the above disadvantageous bypass coding is avoided. The embodiment of the present disclosure is based on the following discovery: a method of obtaining information of the vertex V based on the context information k Position p along edge k k The compression of information allows the predicted position p to be obtained k According to an embodiment, when an integer number of bits is used to represent position p k When the context information is used to obtain the corresponding bit B i The probability of the value of , and this probability is used by a codec (such as a binary entropy codec) to allow compression and avoid simple bypass encoding of the corresponding bits.

[0145] According to an embodiment, the context information CI may be based on occupancy information of neighboring bodies adjacent to the current edge, and / or based on vertex position information of encoded / decoded neighboring edges. The vertex position information may include vertex flags and / or vertex positions on encoded / decoded edges.

[0146] Figure 7 The flowchart of the method for encoding geometric information of the geometric figures of the point cloud into a bit stream according to an embodiment of the present disclosure is illustrated. For example, the point cloud is represented by a plurality of cubes, and the occupied cubes are modeled by one or more triangles, such as Figure 2 and Figure 3 As shown. At least one triangle has at least one corresponding vertex on the edge of the occupied cube. The geometric information includes vertex positions of one or more vertices. According to an embodiment, Figure 7 As depicted, the method comprises the following steps: for a current edge of the cube,

[0147] S100: Constructing context information based on one or more or all of the following:

[0148] Occupancy information of the adjacent cubes adjacent to the current edge,

[0149] Vertex position information of the encoded adjacent edges of the current edge, which are edges that have common points with the current edge.

[0150] S102: Use the context information to select the encoding and decoding probability of the entropy codec.

[0151] S104: Encode the vertex position on the current edge through an entropy codec and using the selected coding probability.

[0152] Figure 8 A flow chart of a method for decoding geometric information of a point cloud from a bitstream according to an embodiment of the present disclosure is illustrated. The point cloud is represented by a plurality of cubes, and the occupied cubes are modeled by one or more triangles. At least one triangle has at least one corresponding vertex on an edge of the occupied cube. The geometric information includes vertex positions of one or more vertices. According to an embodiment, as Figure 8 As depicted, the method comprises the following steps, for the current edge of the cube,

[0153] S200: Constructing context information based on one or more or all of the following:

[0154] Occupancy information of the adjacent cubes adjacent to the current edge,

[0155] The vertex position information of the decoded adjacent edges of the current edge, which are edges that have common points with the current edge,

[0156] S202: Use context information to select the encoding and decoding probability of the entropy codec.

[0157] S204: Decode the vertex position of the vertex on the current edge by using the entropy codec and the selected coding and decoding probability.

[0158] Fig. 9 A data stream 200 according to an embodiment of the present disclosure is illustrated, in which geometric information of the geometry of a point cloud is encoded. The point cloud is represented by a plurality of cubes, and the portion of the point cloud enclosed by the occupied cubes is modeled by one or more triangles. At least one triangle has at least one corresponding vertex on the edge of the occupied cube. The geometric information includes vertex positions 202 of one or more vertices that signal the presence of the vertex. For example, the data stream or bit stream 200 may be provided by an encoder 300 that performs the method of the present invention for encoding the vertex positions 202 into the data stream 200. The data stream 200 is transmitted to a decoder 400 via a wired or wireless transmission medium 204 (such as a cable or a radio link), and the decoder 400 decodes the vertex positions 202 from the data stream 200. Therefore, according to an embodiment, as Fig. 9 As depicted, for a current edge, data stream 200 includes vertex positions 202 for the current edge encoded into data stream 200 by an entropy codec and using codec probabilities. The codec probabilities are selected using context information, and the context information is constructed based on one or more or all of the following:

[0159] Occupancy information of the adjacent cubes adjacent to the current edge,

[0160] Vertex position information of the encoded adjacent edges of the current edge, which are edges that have common points with the current edge.

[0161] Embodiments of the present invention are now described in more detail. For the following description, edges belonging to at least one occupied volume are indexed by index k. An occupied volume is a volume associated with an occupied leaf node of a tree representing the underlying geometry of a point cloud. The position of a TriSoup vertex on edge k is represented by at least one bit B i According to an embodiment of the present disclosure, in order to indicate the position of each bit B of the vertex on the current edge k, i For encoding, the entropy codec or the context to be used by the entropy codec is selected based on context information CI (which may depend on the bit index i), which is constructed using one or more or all of the following:

[0162] The occupancy of the neighbors associated with the neighboring nodes of the tree

[0163] One or more vertex labels k' of adjacent already encoded edge k', and

[0164] One or more vertices V k’ The vertex position p on the adjacent encoded edge k' k’ .

[0165] Fig.10 A flowchart of a method for encoding information representing the position of a vertex on a current edge according to an embodiment of the present disclosure is illustrated. Fig.10 In the embodiment, the information representing the position of the vertex is assumed to be at least one bit B i .according to Fig.10 In the depicted embodiment, for the current edge k, the vertex mark s is k Encoded into the bitstream 200, as shown in step S120. Vertex flag s k Signaling vertex V k In step S122, check whether the vertex or the existence flag s exists on the current edge k. k Is true, and if the vertex flag s k is false (0), the vertex position is not encoded for the current edge k, and the encoding method continues to the next current edge k+1, as shown in step S124. On the other hand, if the vertex flag s k If (1) is true, then according to the above reference Figure 7 The method described above converts the current vertex V k The vertex position p along the current edge k k is encoded into the bit stream 200. Fig.10 As shown, initially, in step S100, context information is constructed, which, according to the depicted embodiment, includes steps S106, S108, and S110. In step S106, adjacent nodes / volumes are obtained. l 、a m and b n The adjacent nodes / volumes include the volume e that intersects with the current edge k. l 、a m and b n , as described in more detail below. In addition, in step S108, the existence of the vertex of the adjacent encoded edge k' is obtained (such as the vertex flag s k’ ), and in step S110, obtain the vertex position p of the vertex along the adjacent encoded edge k' k’ Based on the information obtained in steps S106 to S110, in step S112, context information CIi is constructed. The following describes in more detail the process for constructing context information CIi. i Embodiment of the invention.

[0166] exist Fig.10 In the embodiment, the information indicating the position is considered to be at least one bit B i , using the context information obtained in step S100, for each bit B i In step S102, based on the context information CI i The probability of selecting a codec. In step S104, bit B i is encoded into the bitstream 200. For this process, initially, the index i is set to 1, as shown in S126. Furthermore, steps S112, S102 and S104 are repeated until the position p is represented. k All bits of B i have all been encoded into the bitstream 200, and for each iteration, the index i is increased by 1, as shown in step S128.

[0167] According to an embodiment, in step S102, a method for determining the vertex position p is selected. k The encoding probability of an entropy codec is encoded, for example by applying a context-adaptive binary arithmetic codec CABAC mechanism, or by applying an optimal binary codec OBUF mechanism with dynamic updating, for example as described in EP 3 633 857 A1, the content of which is incorporated herein by reference.

[0168] Fig.11 The decoding method according to an embodiment of the present disclosure is illustrated, which is basically the same as the above reference Fig.10 The method is the same as described above, except that the codec decodes from the bitstream 200 a representation of the vertex position p k At least one bit B i , rather than encoding the at least one bit into the bit stream. Fig.11 In the depicted embodiment, for the current edge k, the vertex flag s is decoded from the bitstream 200 k , as shown in step S220. Vertex mark s k Signal whether vertex V exists on the current edge k k In step S222, check the vertex flag or the existence flag s k Is true, and if the vertex flag s k is false (0), then no vertex position is encoded or decoded for the current edge k, and the decoding method proceeds to the next current edge k+1, as shown in step S224. On the other hand, if the vertex flag s k If (1) is true, then according to the above reference Figure 8 The method described above decodes the current vertex V from the bitstream k The vertex position p along the current edge k k For the current edge k, Fig.10 In the same manner as described, in steps S206 to S210, information regarding occupancy of adjacent bodies, the presence of vertices on adjacent already decoded edges, and vertex positions along adjacent already decoded edges is determined in order to construct context information CI in step S212. i . In step S202, as also referred to above Fig.10 as described, a context or entropy codec is selected.

[0169] As in Fig.10 the embodiment of, the information representing the position is considered to be at least one bit B i , and using the context information obtained in step S200, for each bit B i , in step S202, based on context information CI i the probability of selecting a codec, in step S204, bit B is decoded from bitstream 200. i . For this process, initially, index i is set to 1, as shown in step S226. Further, steps S212, S202, and S204 are repeated until all bits B representing the position have been decoded from bitstream 200 i , and for each iteration, index i is incremented by 1, as shown in step S228.

[0170] According to a further embodiment, for i' < i, the context information CI i for encoding bit B i can also be constructed based on previously encoded bits B i’ . As shown in step S112 (or step S212), in addition to the context information of steps S106 to S110 (or steps S206 to S210), previously encoded bits B i’ are provided. According to such an embodiment, due to the good correlation from higher bits to lower bits, it is advantageous to encode bits B i from the highest weight to the lowest weight. In other words, the context information CI i can be a function of the previously encoded bits, i.e., context information CI i ({B i’ |i' < i}).

[0171] The above-described embodiments of encoding / decoding of vertex positions have been described with reference to the processing of the current edge k, i.e., the above encoding / decoding process is repeated for all edges of the occupied body to obtain the corresponding information representing the vertex positions on the edges. Edge k is oriented from a starting point to an ending point, and in the following figures, the currently processed edge k is represented as an arrow from the starting point to the ending point, i.e., the arrow is the ending point of edge k. If there is a TriSoup vertex V on edge kk , then its position p k can be provided along the edge relative to the starting position of the edge, such that the position p k is scalar information.

[0172] There are 12 adjacent bodies intersecting the current edge k, that is

[0173] · Four adjacent bodies e1 to e4 sharing the current edge k, as Fig.12 shown, Fig.12 The figure illustrates four leaf nodes / bodies e1 to e4 sharing the current edge k shown by the shared arrow.

[0174] · Four adjacent bodies a1 to a4 having a corner that is the starting point of the current edge k.

[0175] · Four adjacent bodies b1 to b4 having a corner that is the ending point of the current edge k.

[0176] Fig.13 The figure respectively illustrates eight leaf nodes / bodies a1 to a4 and b1 to b4, one of whose corners is the starting point or the ending point of the current edge k. The 12 occupancy bits associated with the 12 adjacent bodies e l , a m and b n are used to construct the context information CI i (e l , a m , b n ). An example of this construction will be provided later.

[0177] The vertex position p associated with the edge k k is encoded in a predetermined order, which is also referred to as the TriSoup edge order. According to an embodiment, as described above, certain information (such as the presence flag s k’ and the vertex position p from the edge k' that has been encoded / decoded relative to the TriSoup edge order k’ ) can be used to construct the context information CI i to encode the current vertex position p k , where k' < k. For example, when assuming that the three axes of the 3D space are labeled as x, y, z, the TriSoup edge order can be the lexicographical order in the 6D representation of the edge (x start , y start , z start , x end , y end , z end ), where (x start , y start , z start ) are the coordinates of the starting point of the edge, and (x end , yend ,z end ) are the coordinates of the end point. Using this order, the edge k' that is parallel to the current edge k and points to its starting point has always been encoded / decoded, such as Fig.14 As shown, Fig.14 The unique edge k' is illustrated with a dotted arrow, which is parallel to the current edge k represented by the arrow and points to the starting point of the current edge. The vertex V on the unique edge k' is illustrated k’ , so that Fig.14 In the example, for the unique edge k', the vertex label s k’ True. In addition, the vertex V that indicates the only edge k' k’ The position p k’ , and as mentioned above, the position p k’ It can be scalar information measured from the starting point of the unique edge k'.

[0178] In addition, there are 4 edges k' that are perpendicular to the current edge and point to the starting point of the current edge k or start from the starting point of the current edge k, such as Fig.15 As shown, Fig.15 The dashed arrows illustrate four vertical edges k', which are orthogonal to the current edge k and point to or start from the starting point of the current edge. Given the TriSoup edge order, Fig.15 Some or all of the four vertical edges k' in have been encoded / decoded. Fig.16 As shown, the number N of vertical edges k' that have been encoded / decoded perp (k) depends on the direction of the current edge k. When applying the lexicographic TriSoup edge order mentioned above, the number of vertical edges that have been encoded / decoded is

[0179] If the current edge k is parallel to the x-axis, then N perp (k) = 4, e.g. Fig.16 (a) As shown in the figure

[0180] If the current edge k is parallel to the y-axis, then N perp (k) = 3, e.g. Fig.16 (b) As shown in the figure

[0181] If the current edge k is parallel to the z-axis, then N perp (k) = 2, such as Fig.16 (c) As shown in the figure.

[0182] According to an embodiment of the present disclosure, for N perp (k) encoded / decoded vertical edges k', vertex labels s k’ and the existing vertex V k’ The vertex position p k’ Can be used to build context information CIi .

[0183] It should be noted that the edges k' that are perpendicular to the current edge k and point to the end point of the current edge or start from the end point of the current edge are not encoded / decoded, so these edges cannot be used to construct the context information CI naturally. i .

[0184] Therefore, according to the above described embodiments, the context information CI may be constructed using one or more or all of the following items: i :

[0185] The occupancy status of the neighbor, such as CI i (e l , a m , b n ).

[0186] The presence of one or more vertices on an already encoded / decoded edge, e.g. CI i ({s k’ |k' <k})。

[0187] The position of the vertex, such as CI i ({p k’ |k' <k})。

[0188] We now describe a method for constructing context information CI based on the occupancy of neighboring values. i Embodiment of the invention.

[0189] Contextual information CI based on neighboring body occupancy i Construction

[0190] It has been found that it is possible to compress the 12 adjacent bodies l 、a m 、b n Associated occupancy information to build contextual information CI i , without significantly affecting the compression efficiency. Assuming N e It is 4 bodies l The number of occupied volumes in , where not all volumes are unoccupied (otherwise the current edge is not a TriSoup edge), then the following inequality holds:

[0191] 1≤N e ≤4

[0192] Assume N a It is 4 bodies a m The number of occupied bodies in b It is 4 bodies n The number of occupied bodies in Fig.13 ), then the following inequality holds:

[0193] 0≤N a ≤4 and 0≤N b ≤4

[0194] It has been found that based on N e 、N a 、N b Instead of representing the 12 bits occupied by the CI i Provides good compression performance. According to an embodiment, the context information CI i Part of can be the following numbers

[0195] 5*4*N a +4*N b +(N e -1).

[0196] According to a further embodiment, it has been found that the maximum occupancy is the position p of the vertex k A good predictor of , making the context information CI i This part can be further compressed into 3 bits of information as follows:

[0197] 4*(if N a = = 4 is 1, otherwise 0) + 2*(if N b = = 4 is 1, otherwise 0) + (if N e ==4 is 1, otherwise it is 0).

[0198] According to a further embodiment, the above methods may be combined so that, for example, the context information CI i Part of can be the following numbers

[0199] 8*max(0,N a -1)+2*max(0,N b -1)+(if N e ==4 is 1, otherwise it is 0).

[0200] Construct context information CI based on the encoded adjacent edge k' i

[0201] Adjacent Vertex V k’ Number of

[0202] As described above, according to an embodiment, the context information CI may be constructed based on the adjacent edge k′ that has been encoded / decoded. i , for example, based on the adjacent vertices V k’ At most 5 adjacent edges have been encoded / decoded, that is, the reference above Fig.15 Described N perp (k) vertical edges, and the above reference Fig.14The parallel edge k' or the unique edge k' described. Let TV(k) be the number of encoded / decoded edges k' on which there is a TriSoup vertex, i.e., with the vertex flag s true k’ , and assuming that NTV(k) is the number of edges k' on which no TriSoup vertex exists, then the following equality holds:

[0203] N perp (k)+1=TV(k)+NTV(k).

[0204] Therefore, according to an embodiment, the context information CI can be constructed based on the quantity i .

[0205] According to a further embodiment, it has been found that since TV(k) is related to the vertex V k The position p k The correlation is weak, and the value of NTV(k) alone also provides a sufficiently good compression result. Therefore, according to such an embodiment, the context information CI can be constructed based only on NTV(k). i .

[0206] The relative positions of adjacent vertices

[0207] According to a further embodiment, the relative positions of adjacent vertices may be used to construct the context information CI i When considering the only edge k' that is parallel to the current edge k and points to its starting point, as mentioned above Fig.14 As described above, the TriSoup vertices belonging to this edge can be represented as V par ,like Fig.17 As shown, except for the encoded / decoded TriSoup vertex V on the only edge k' par In addition, Fig.17 Also shown is its quantized position p par,Q It has been found that the vertex V par Position p along the encoded / decoded edge k' par,Q is the position p of the vertex on the current edge k k The vertex position can be quantized as p par,Q , as in Fig.17 As shown in the figure, quantification can be used to construct context information CI i A two-bit value.

[0208] According to a further embodiment, it has been found that the encoded TriSoup V k’ Position p on the vertical edge k’ (See Fig.15 ) is also the position of the vertex p k For example, Fig.18 As shown, one or more positions p on the vertical edge k' can be k’ Simplified to a binary value, the binary value indicates whether the vertex is "closer" or "farther" from the starting point of the current edge k. Fig.18 As shown, if the vertex V k If the distance from the end point of edge k' (the starting point of the current edge k) is lower than a certain threshold or a predetermined threshold, the vertex V k Position p on vertical edge k' k’ Indicates the first position. Fig.18 In the example above, this first position is called the position “close to” the current edge k. k’ The position p whose distance from the starting point of the current edge k is greater than the threshold k’ It is called the position "far away" from the current edge k. Get the near vertex V k’ The number N near And use it to build context information CI i . The following inequality holds:

[0209] 0≤N near ≤4.

[0210] According to an embodiment, the vertex V par Position p on the only edge k' par (See Fig.17 ) can also be considered for counting near vertices, and in such an embodiment, the following inequality holds:

[0211] 0≤N near ≤5.

[0212] According to an embodiment, the number of close vertices may be limited to obtain a value referred to as N near ' 2 bits of information, for which the following inequality applies:

[0213] 0≤N near '=min(3,N near )≤3.

[0214] According to the embodiment, when constructing the context information CI i When , you can combine the vertices V that have been encoded / decoded k’ The position p k’ According to an embodiment, the context information CI i Part of can be

[0215] 4*p par,Q +N near '.

[0216] According to an embodiment, information about the relative positions of neighboring vertices can be combined with the occupancy information of neighboring bodies, and it has been found that, for example, the following 6-bit context word is the position p of the vertex k A very strong predictor of .

[0217] W=16*(N e -1)+4*p par,Q +N near '

[0218] According to further embodiments, word W may be enriched with any of the contextual information described above, such as (N a = =4), (N b ==4) and NTV(k) information.

[0219] The context information CI is constructed based on the relative positions of vertices on adjacent edges. i Still a further embodiment, instead of setting the position p k’ Reduced to a binary value, it can also be reduced to a ternary value indicating the position p on the vertical encoded / decoded edge k' k’ is within one of several intervals defined along the vertical edge k'. For example, Fig.19 As shown in the figure, the interval includes the indicated position p k’ The far interval starts at the starting point of edge k' and extends along a portion of the length of edge k'. The near interval ends at the end point of edge k', which is the starting point of the current edge k, and extends along a portion of the length of edge k'. The middle interval is the length of edge k' between the near interval and the far interval. Fig.19 When the ternary value is shown in the figure, the following equation applies:

[0220] N near +N mid +N far =TV(k)≤5.

[0221] The number of near / middle / far vertices can be limited to obtain 2 bits of information N near '、N mid ' and N far '.

[0222] 0≤N near '=min(3,N near )≤3

[0223] 0≤Nmid'=min(3,N mid )≤3

[0224] 0≤N far '=min(3,Nfar ) ≤ 3。

[0225] According to an embodiment, when constructing the context information CI i information about the already encoded / decoded vertex V k’ at position p k’ can be combined. According to an embodiment, a part of the context information CI i can be

[0226] 32 * N far ’ + 8 * N mid ’ + 2 * N near ’ + p par,Q .

[0227] According to an embodiment, when combining the occupancy information described above about adjacent bodies and the ternary information about the relative position of the vertex on the adjacent edge, it has been found that the following context word is a very strong predictor of the position p k of vertex V k .

[0228] W = 32 * W1 + W2

[0229] where

[0230] W1 = 32 * N far ’ + 8 * N mid ’ + 2 * N near ’ + p par,Q

[0231] W2 = 8 * max(0, N a - 1) + 2 * max(0, N b - 1) + (N e == 4).

[0232] Context information CI based on previously coded bits Bi' i Construction

[0233] According to an embodiment representing the vertex position with multiple bits, the previously encoded bit Bi’ representing the position p k can be used as part of the context information CI i to encode the current bit B i (i’ < i). For example, using the word W introduced above, the context information might be

[0234] CI i = 2 i * W + B i-1 …B1

[0235] So far, the concepts of the present invention have been described with reference to embodiments relating to methods for encoding / decoding geometric information (i.e., vertex positions) into / from a bitstream. According to further embodiments, the present disclosure also provides an apparatus for encoding / decoding geometric information of a point cloud's geometry into / from a bitstream.

[0236] Fig. 20 FIG. 1 is a diagram illustrating a method for encoding a vertex position p according to an embodiment. k The device 300 includes the following modules:

[0237] A context information building module 302 is used to build context information for the current edge based on one or more or all of the following items:

[0238] Occupancy information of the adjacent cubes adjacent to the current edge,

[0239] Vertex position information of the encoded adjacent edges of the current edge, which are edges that have common points with the current edge.

[0240] The selection module 304 is configured to use the context information provided by the context information construction module 302 to select the encoding and decoding probability of the entropy codec.

[0241] The encoding module 306 is used to encode the vertex position p of the current edge k by using the entropy codec and the selected codec probability. k to encode.

[0242] Fig.21 FIG. 1 illustrates a method for decoding a vertex position p according to an embodiment of the present disclosure. k The device 400 includes the following modules:

[0243] A context information building module 402 is used to build context information for the current edge based on one or more or all of the following items:

[0244] Occupancy information of the adjacent cubes adjacent to the current edge,

[0245] Vertex position information of the decoded adjacent edges of the current edge, which are edges that have common points with the current edge.

[0246] The selection module 404 is configured to use the context information provided by the context information construction module 402 to select the encoding and decoding probability of the entropy codec.

[0247] The decoding module 406 is used to decode the vertex position p of the current edge k by using the entropy codec and the selected codec probability. k to decode.

[0248] According to the embodiment, the above reference Fig. 20 and Fig.21 The depicted context information building module 302 / 402 may include one or more sub-modules. Fig. 22 A block diagram of an embodiment for implementing a context information building module 302 / 402 is shown, which may be used in Fig. 20 The encoder or Fig.21 The context information building module may include one or more of the following sub-modules:

[0249] The occupancy information obtaining submodule 500 is used to obtain the adjacent cubes e adjacent to the current edge k. l 、a m 、b n occupancy information.

[0250] The vertex existence obtaining submodule 502 is used to obtain the existence of vertices on the adjacent edge k' that has been encoded / decoded.

[0251] The vertex position obtaining submodule 504 is used to obtain the vertex positions along the adjacent edge k' that has been encoded / decoded.

[0252] The present disclosure further provides electronic devices, computer-readable storage media, and computer program products in embodiments.

[0253] Although some aspects of the disclosed concepts have been described in the context of an apparatus, it is apparent that these aspects also represent descriptions of corresponding methods, where blocks or devices correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of corresponding blocks or items or features of corresponding apparatus.

[0254] Fig.23 is a block diagram illustrating an electronic device 900 according to an embodiment of the present disclosure.

[0255] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described only as examples and are not intended to limit the embodiments of the present disclosure described and / or claimed herein.

[0256] refer to Fig.23 , the device 900 includes a computing unit 901 for performing various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data for the operation of the storage device 900 may also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0257] The components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse; an output unit 907, such as various types of displays, speakers; a storage unit 908, such as a disk, an optical disk; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0258] The computing unit 901 may be formed by various general and / or special processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the image processing method described above may be performed. In some embodiments, the computing unit 901 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0259] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuits, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0260] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions and / or operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0261] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0262] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including sound input, voice input, or tactile input.

[0263] The systems and techniques described herein may be implemented on a computing system that includes a back-end component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a front-end component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes such back-end components, middleware components, front-end components, or any combination thereof. The components of the system may be interconnected by any form of digital data communication medium (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0264] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in a cloud computing service system and solves the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (referred to as "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0265] It should be understood that the steps can be reordered, added or deleted by using the various forms of processes shown above. For example, the steps described in the present disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present disclosure can be achieved, and this document is not limited.

[0266] The above specific embodiments do not limit the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement made within the principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for encoding geometric information of a geometric figure of a point cloud into a bit stream (200), the point cloud being represented by a plurality of cubes (100), the occupied cubes (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100) k ), and the geometric information includes one or more vertices (V k )’s vertex position (p k ), the method comprising: For the vertices on the current edge (k), Context information (CI) is constructed (S100) based on one or more or all of the following: i ): - occupancy information of neighboring cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k), - vertex position information of an encoded adjacent edge (k') of the current edge (k), wherein the adjacent edge (k') is an edge having a common point with the current edge (k), Using the context information (CI i ) to select (S102) the encoding and decoding probability of the entropy codec, and The vertices (V) are decoded by the entropy codec and using the selected codec probabilities. k ) on the vertex position (p) of the current edge (k) k ) is encoded (S104).

2. A method for decoding geometric information of a point cloud from a bit stream (200), the point cloud being represented by a plurality of cubes (100), the occupied cubes (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100) k ), and the geometric information includes one or more vertices (V k )’s vertex position (p k ), the method comprising: For the vertices on the current edge (k), Context information (CI) is constructed (S200) based on one or more or all of the following: i ): - occupancy information of neighboring cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k), - vertex position information of the decoded adjacent edge (k') of the current edge (k), wherein the adjacent edge (k') is an edge having a common point with the current edge (k), Using the context information (CI i ) to select (S202) the encoding and decoding probability of the entropy codec, and The vertex (V) is decoded (S204) by the entropy codec and using the selected codec probability. k ) on the vertex position (p) of the current edge (k) k ).

3. The method according to claim 1 or 2, wherein the context information (CI) is constructed based on one or more or all of the following: i ): - count of all occupied neighboring cubes (e1 to e4, a1 to a4, b1 to b4) that belong to a subset of neighboring cubes, - whether all adjacent cubes belonging to a subset of adjacent cubes (e1 to e4, a1 to a4, b1 to b4) are occupied, Wherein the subset of adjacent cubes includes one or more or all of the following: - said neighboring cubes (e1 to e4) sharing said current edge (k), - said neighboring cube (a1 to a4) having a corner which is the starting point of said current edge (k), - the adjacent cube (b1 to b4) having a corner which is the end point of the current edge (k).

4. A method according to any one of the preceding claims, wherein the context information (CI) is constructed based on one or more or all of the following: i ): - the presence flag (s) of the encoded / decoded edge (k') associated with the neighboring edge (k) of the current edge (k) k’ ), -Vertex (V k’ ) on the encoded / decoded adjacent edge (k') of the current edge (k) k’ ), -Vertex (V k’ ) counts the number of adjacent edges (k') that have been encoded / decoded of the current edge (k), and the vertex (V k’ ) is less than a predetermined threshold value from the current edge (k), -Vertex (V k’ ) counts the number of adjacent edges (k') that have been encoded / decoded of the current edge (k), and the vertex (V k’ ) is within a predetermined interval from the current edge (k).

5. The method according to claim 4, wherein the context information (CI) is constructed based on the following items: i ) - the existence flag of the encoded / decoded k’ ) is true, and the existence flag (s k’ ) is false, or - Only the existence flags that have been encoded / decoded (s k’ ) is the number of adjacent edges (k') for which .

6. The method according to claim 4 or 5, wherein the context information (CI) is constructed i ) includes placing the vertex (V k’ ) on the adjacent edge (k') that has been encoded / decoded k’ ) is quantized as a ratio of the vertex (V k’ ) on the adjacent edge (k') that has been encoded / decoded k’ ) is encoded into the bitstream (200) with a coarser precision.

7. A method according to any one of the preceding claims, wherein The vertex position (p k ) consists of one or more bits (B i )express, is the vertex position (p k ) for each bit (B i ) constructs the context information (CI i ),and The vertex position (p k ) for each bit (B i ) is passed through a binary entropy codec using a bit-based i ) of the context information (CI i ) determines the encoding / decoding probability for encoding / decoding.

8. The method according to claim 7, wherein the vertex position (p k ) bits (B i ) of the context information (CI i ) is further based on the previously encoded bits (B i-1 ) to construct.

9. A method according to any one of the preceding claims, wherein to select the codec probability of the entropy codec, the method uses the following - an optimal binary codec OBUF mechanism with dynamic update, or -Context-Adaptive Binary Arithmetic Codec (CABAC) mechanism.

10. The method according to any of the preceding claims, wherein the point cloud is modeled using a TriSoup encoding and decoding scheme.

11. A non-transitory computer-readable storage medium comprising instructions, which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 10.

12. A device (300) for encoding geometric information of a geometric figure of a point cloud into a bit stream (200), the point cloud being represented by a plurality of cubes (100), the occupied cubes (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100) k ), and the geometric information includes one or more vertices (V k )’s vertex position (p k ), the device comprising: Contextual Information (CI i ) construction module (302), the context information (CI i ) construction module (302) is configured to construct context information (CI) for the current edge (k) based on one or more or all of the following items: i ): - occupancy information of neighboring cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k), - vertex position information of an encoded adjacent edge (k') of the current edge (k), wherein the adjacent edge (k') is an edge having a common point with the current edge (k), A selection module (304) is configured to use the context information (CI i )Building module provides context information (CI i ) to select the encoding and decoding probability of the entropy codec, and The encoding module (306) is used to encode the vertex (V k ) on the vertex position (p) of the current edge (k) k ) for encoding.

13. A device (400) for decoding geometric information of a geometric figure of a point cloud from a bit stream (200), the point cloud being represented by a plurality of cubes (100), the occupied cubes (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100) k ), and the geometric information includes one or more vertices (V k )’s vertex position (p k ), the device comprising: Contextual Information (CI i ) construction module (402), the context information (CI i ) construction module (402) is configured to construct context information (CI) for the current edge (k) based on one or more or all of the following items: i ): - occupancy information of neighboring cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k), - Vertex position information of the decoded adjacent edge (k') of the current edge (k), wherein the adjacent edge (k') is an edge having a common point with the previous edge (k), A selection module (404) is configured to use the context information (CI i ) The context information (CI) provided by the building module i ) to select the encoding and decoding probability of the entropy codec, and A decoding module (406) configured to decode the vertex (V) through the entropy codec and using the selected codec probability k ) on the vertex position (p) of the current edge (k) k ).

14. The apparatus according to claim 11 or 12, wherein the context information (CI i ) The building module (302, 402) includes one or more or all of the following sub-modules: - an occupancy information obtaining submodule (500), the occupancy information obtaining submodule (500) being configured to obtain occupancy information of adjacent cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k), - a vertex existence obtaining submodule (502), the vertex existence obtaining submodule (502) being configured to obtain the existence of vertices on the adjacent edge (k') that has been encoded / decoded, - A vertex position obtaining submodule (504), which is configured to obtain vertex positions along the already encoded / decoded adjacent edge (k').

15. A data stream (200) in which geometric information of a point cloud geometry is encoded, the point cloud being represented by a plurality of cubes (100), the occupied cubes (100) being modeled by one or more triangles (102 to 112), at least one triangle having at least one corresponding vertex (V) on an edge of the occupied cube (100) k ), and the geometric information includes one or more vertices (V k )’s vertex position (p k ), the data stream (200) comprises: For the vertices on the current edge (k), The vertices (V) are encoded into the data stream (200) using the entropy codec and the codec probabilities. k ) on the vertex position (p) of the current edge (k) k ), using context information (CI i ) to select the encoding and decoding probability, The context information (CI) is constructed based on one or more or all of the following items: i ): - occupancy information of neighboring cubes (e1 to e4, a1 to a4, b1 to b4) adjacent to the current edge (k), - Vertex position information of an already encoded adjacent edge (k') of the current edge (k), wherein the adjacent edge (k') is an edge having a common point with the current edge (k).