Encoding / decoding positions of points of point cloud contained in cuboid volume

By quantizing and encoding vertex positions in the cuboid volume, combining the centroid point and residual position, the TriSoup encoding scheme is improved, solving the problem of low transmission efficiency of dense point clouds, and real-time transmission and high speed of dynamic AR/VR applications are achieved Quality visual effects.

CN118985133AActive Publication Date: 2024-11-19BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202380025336.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-25
Filing Date
2023-02-21
Publication Date
2024-11-19
Estimated Expiration
2043-02-21

AI Technical Summary

Technical Problem

Existing point cloud compression technology is inefficient when dealing with dense point clouds, especially in dynamic AR/VR applications, and traditional methods are difficult to achieve real-time transmission and good visual quality.

Method used

The TriSoup encoding scheme is improved by quantizing the vertex positions in the cuboid volume and encoding their information, combining the centroid points and residual positions, to optimize the bit rate and distortion trade-off.

Benefits of technology

It improves the encoding efficiency and decoding quality of point clouds, adapts to the real-time transmission needs of dynamic AR/VR applications, reduces the bit rate, and maintains good visual performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118985133A_ABST
    Figure CN118985133A_ABST
Patent Text Reader

Abstract

Methods and apparatus are provided for encoding / decoding positions of points of a point cloud contained in a cuboid volume, the positions being represented in a three-dimensional coordinate system, at least three vertices being located on edges of the cuboid volume, one vertex per edge. The encoding method comprises:-obtaining (110) quantized positions (pQ) of the at least three vertices by quantizing the positions of the at least three vertices on the basis of a vertex quantization step size ([Delta] V), the vertex quantization step size ([Delta] V) depending on at least one quantization parameter; -encoding (120) vertex information defining quantization positions (pQ) of at least three vertices into the bitstream; and-encoding (130) at least one quantization parameter into the bitstream.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to and the benefit of European Patent Application No. 22305367.9, filed on March 25, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to point cloud compression, and in particular to methods and apparatus for encoding / decoding positions of points of a point cloud contained in a cuboid volume. Background Art

[0004] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one embodiment of the present disclosure described and / or claimed below. This discussion is considered to be helpful in providing the reader with background information to facilitate a better understanding of various aspects of the present disclosure.

[0005] Point clouds have recently gained attention as a format for representing 3D data due to their versatility in representing all types of physical objects or scenes. Point clouds can be used for various purposes such as cultural heritage / architecture, where objects such as statues or buildings are scanned in 3D to share the spatial configuration of the object without sending or visiting the object. Moreover, it is a way to ensure that the knowledge of the object is preserved in cases where the object may be destroyed; for example, a temple destroyed by an earthquake. Such point clouds are usually static, colorful, and huge.

[0006] Another use case is in topology and cartography, where using a 3D representation allows maps to be more than just flat and can include landforms. Google Maps is now a good example of a 3D map, but uses a mesh instead of a point cloud. However, a point cloud can be a suitable data format for a 3D map, and such point clouds are often static, colorful, and large.

[0007] Virtual Reality (VR), Augmented Reality (AR) and immersive worlds have become hot topics recently and are foreseen by many as the future of 2D flat videos. The basic idea is to immerse the viewer in the surrounding environment, while standard TV only allows the viewer to watch the virtual world in front of him / her. There are several levels of immersion, depending on the degree of freedom the viewer has in the environment. Point clouds are a good format candidate for distributing VR / AR worlds.

[0008] The automotive industry, and more specifically the foreseeable autonomous vehicles, is also an area where point clouds can be used extensively. Autonomous vehicles should be able to "probe" their environment to make good driving decisions based on the presence and nature of their closest nearby objects and the road configuration detected.

[0009] A point cloud is a collection of points in three-dimensional (3D) space, optionally with additional values ​​attached to each point. These additional values ​​are often called attributes. Attributes can be, for example, three-component color, material properties (such as reflectivity), and / or a two-component normal vector of the surface associated with the point.

[0010] A point cloud is therefore a combination of geometric data (the location of points in 3D space, usually represented by 3D Cartesian coordinates x, y, and z) and attributes.

[0011] Point clouds can be sensed by various types of devices, such as arrays of cameras, depth sensors, lasers (light detection and ranging, also known as LiDAR), radars, or can be generated by computers (e.g., in movie post-production). Depending on the use case, point clouds can have thousands to billions of points for mapping applications. The raw representation of point clouds requires a very high number of bits per point, at least a dozen bits per Cartesian coordinate x, y, or z, and optionally more bits for attribute(s), such as three times 10 bits for color.

[0012] In many applications, it is very important to be able to distribute point clouds to end users or store them in servers by consuming only a reasonable amount of bitrate or storage space while maintaining an acceptable (or preferably very good) quality of experience. Efficient compression of these point clouds is a key point in making the distribution chain of many immersive worlds practical.

[0013] For distribution and visualization by the end user, for example on AR / VR glasses or any other 3D-enabled device, the compression can be lossy (as in video compression). Other use cases do require lossless compression, such as medical applications or autonomous driving, to avoid changing the results of decisions obtained from subsequent analysis of the compressed and transmitted point cloud.

[0014] Until recently, point cloud compression (aka PCC) has not been addressed for the mass market and no standardized point cloud codecs were available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, aka the Moving Picture Experts Group or MPEG, initiated a work item on point cloud compression. This resulted in two standards, namely

[0015] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (aka V-PCC)

[0016] MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-based Point Cloud Compression (aka G-PCC)

[0017] The V-PCC encoding method compresses point clouds by performing multiple projections on 3D objects to obtain 2D tiles packed into images (or videos when processing dynamic point clouds). The obtained images or videos are then compressed using existing image / video codecs, allowing full utilization of already deployed image and video solutions. By its nature, V-PCC is only efficient on dense and continuous point clouds, as image / video codecs cannot compress non-smooth tiles, such as those obtained from the projection of sparse geometric data sensed by LiDAR.

[0018] The G-PCC coding method has two schemes for compressing the sensed sparse geometry data.

[0019] The first scheme is based on an occupancy tree, locally any type of tree among octrees, quadtrees or binary trees, representing the point cloud geometry. Occupied nodes (i.e. nodes associated with a cuboid volume containing at least one point of the point cloud) are split until a certain size is reached and the occupied leaf nodes provide the 3D positions of the points, for example at the centers of these nodes. The occupancy information is carried by occupancy data (binary data, flags) that signal the occupancy state of each child of the node. A high level of compression of the occupancy data for dense point clouds can be obtained by using neighbor-based prediction techniques. Sparse point clouds can also be addressed by directly encoding the positions of points of non-minimum size within a node, stopping the tree construction when only isolated points are present in a node; this technique is called direct coding mode (DCM).

[0020] The second scheme is based on prediction trees, where each node represents the 3D position of a point, and the parent / child relationship between nodes represents the spatial prediction from parent to child. This approach can only solve sparse point clouds and offers the advantages of lower latency and simpler decoding than occupancy trees. However, the compression performance is only slightly better than the first occupancy-based approach, and the encoding is also complex because the encoder must focus on finding the best predictor (in a long list of potential predictors) when constructing the prediction tree.

[0021] In both schemes, the attribute (de)coding is performed after the geometry (de)coding is done, effectively resulting in two (de)codings. Thus, joint geometry / attribute low latency is obtained by using slices that decompose the 3D space into independently coded sub-volumes without prediction between the sub-volumes. This can severely impact compression performance when many slices are used.

[0022] An important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. In addition, AR / VR point clouds are usually locally 2D, as they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly connected (or called dense), that is, points are rarely isolated, but have many neighbors.

[0023] A dense (or solid) point cloud represents a continuous surface with a resolution such that the cuboid volumes (small cubes) associated with the points touch each other without exhibiting any visible holes in the surface.

[0024] Such point clouds are typically used in AR / VR environments and viewed by end users through devices such as televisions (TVs), smartphones, or headsets. They are transmitted to the device or usually stored locally. Many AR / VR applications use moving point clouds that change over time, rather than static point clouds. Therefore, the point cloud data is bulky and must be compressed.

[0025] Today, lossless compression based on an octree representation of the geometry of a point cloud frame (a time instance of a point cloud) can achieve compression of slightly less than one bit per point (1bpp). This may not be sufficient for real-time transmission, which may involve millions of points per point cloud frame at frame rates of up to 50 frames per second (fps), resulting in hundreds of megabits of data per second.

[0026] Therefore, lossy compression can be used, usually with the requirement to maintain acceptable visual quality while compressing sufficiently to fit within the bandwidth provided by the transmission channel. In most applications, real-time transmission of the point cloud frames must be ensured, imposing complexity and latency constraints on the compression scheme. In many applications, bit rates as low as 0.1bpp (10 times more compressed than lossless encoding) are already achievable for real-time transmission.

[0027] V-PCC can achieve such low bitrates by using lossy compression with a video codec that compresses 2D frames (pictures) obtained by projecting parts of a point cloud onto a set of planes to obtain projection tiles. The geometry of the point cloud frame is represented by these projection tiles assembled into a frame, each of which is a small local depth map. However, V-PCC is not general and is limited to narrow types of point clouds that do not exhibit locally complex geometry (e.g., trees, hair), because the obtained projected depth maps are not smooth enough to be effectively compressed by video codecs such as HEVC or VVC.

[0028] Pure 3D compression techniques can handle any type of point cloud. It is still an open question whether 3D compression techniques will be able to compete with V-PCC (or any projection + image coding scheme) on dense point clouds. Standardization is still in progress and aims to provide an extension (revision) of the current G-PCC codec to provide a competitive lossy compression that will compress dense point clouds as good as V-PCC internals while maintaining the versatility of G-PCC so that any type of point cloud (dense, sparse, such as LiDAR-based captured point clouds, 3D maps, etc.) can be processed.

[0029] Basically, there are three main ways to obtain a lossy scheme with an octree representation of point clouds, as is done in the current G-PCC codec.

[0030] The first approach consists in downsampling the entire point cloud to a smaller resolution, losslessly encoding the downsampled point cloud, decoding the encoded downsampled point cloud, and then upsampling the decoded downsampled point cloud to its original resolution. Many upsampling schemes have been proposed (such as super-resolution, AI-based upsampling, learning-based 3D post-processing, etc.). The first approach can provide good PSNR (peak signal-to-noise ratio) results when the downsampling is not too aggressive (for example, no more than 2 times in each spatial direction). However, even if this first approach can provide good PSNR results, the visual quality of the upsampled point cloud is still controversial and difficult to control.

[0031] The second approach is to have the encoder locally "adjust" the point cloud so that the encoding of the octree requires less bitrate. To do this, the points of the point cloud can be slightly moved to obtain occupancy information that is better predicted by neighboring nodes, resulting in a lossless encoding of the modified octree at a reduced bitrate. Unfortunately, this approach results in only a small bitrate reduction.

[0032] The third approach is based on the so-called TriSoup encoding scheme, which is being explored in the ISO / IEC standardization working group JTC1 / SC29 / WG7. The TriSoup encoding scheme is the most promising of the three approaches, but still needs some work to mature.

[0033] The TriSoup encoding scheme provides a local model of the geometry of a point cloud by using a set of triangles without explicitly providing connectivity information, hence its name derived from "triangle soup".

[0034] The point cloud geometry is represented in a 3D coordinate system by a 3D tree (usually an octree) of some resolution, say N×N×N cuboid volumes (cubes) associated with leaf nodes, where N can be 4, 8 or 16 for example. The tree is encoded using a lossless coding scheme, such as G-PCC. The tree itself does not require much bitrate since it does not go to the deepest depth and it has a small number of leaf nodes (cuboid volumes) compared to the number of points in the point cloud. Then, in each N×N×N cuboid volume, the point cloud geometry is modeled by some local model consisting of a set of triangles. The vertices of the triangles are encoded along the edges of the cuboid volume associated with the leaf nodes of the tree, such as Figure 1 As shown. Four vertices (dark black dots) along the edges of the cube are shown. These four vertices are used to form two triangles, which will be discussed later.

[0035] The TriSoup encoding scheme is not limited to a specific number of vertices located at the edges of cuboid volumes, but rather specifies at most one vertex per edge. The TriSoup encoding scheme is not limited to a specific number of triangles in a specific cuboid volume. Vertices are shared by cuboid volumes that have in common the edge to which the vertex belongs. This means that at most one vertex is encoded for each edge that belongs to at least one cuboid volume. By doing so, the continuity of the local model can be ensured through the cuboid volumes associated with the leaf nodes of the tree.

[0036] Encoding vertices located on the edge of a cuboid volume requires encoding vertex information for each vertex, the vertex information including a vertex flag indicating whether the vertex exists on the edge of the cuboid volume and, if so, the vertex position along the edge. It is then apparent that the position of the vertex in a three-dimensional coordinate system can be derived from the vertex position along the edge and the positions of the start and end vertices of the cuboid volume defining the edge.

[0037] The vertex flags can be encoded by an adaptive binary arithmetic encoder that uses a specific context to encode the vertex flags. By pushing s bits into the bitstream (bypassing / not doing entropy coding), the vertex flags along the length N=2 can be encoded with unit precision. s The vertex positions of the edges are encoded.

[0038] In summary, encoding point cloud geometry using the TriSoup encoding scheme requires encoding the octree data (e.g., via G-PCC) and encoding the vertex information. As discussed below, the increase in encoded data due to the encoding of the vertex information is more than compensated by the improved reconstruction of the triangulated point cloud.

[0039] Within the cuboid volume associated with a leaf node of the tree, if there are at least three vertices on the edge of the cuboid volume, at least one triangle is constructed from the vertices. When there are more than three vertices on the edge of the cuboid volume, more than one triangle may be constructed.

[0040] There are several construction procedures for constructing such triangles.

[0041] In the current TriSoup encoding scheme, the steps of constructing a triangle are as follows: first, determine the dominant direction along one of the three axes of the three-dimensional coordinate system representing the volume of the cuboid; next, sort at least three vertices based on the determined dominant direction; finally, construct at least one triangle based on the list of at least three ordered vertices.

[0042] The dominant direction is determined by testing each of the three directions along the three axes, and keeping the one that maximizes the total surface of the triangle (looking along the tested directions) as the dominant direction.

[0043] Figure 2 and Figure 3 An illustrative example of a process for determining a dominant direction based on a current TriSoup is shown.

[0044] For the sake of simplicity in the drawings, Figure 2 and Figure 3 Only trials for two axes are depicted.

[0045] Figure 2 An illustrative example is shown when a cuboid volume and four vertices (lower right) are projected onto a 2D plane (upper right) along a vertical axis. The four vertices are then sorted in a clockwise order relative to the center of the projected cuboid volume (square). Then, two triangles (upper left) are constructed based on the ordered vertices according to a fixed rule. The fixed rule here is that when 4 vertices are involved, triangles 123 and 134 are constructed. When there are 3 vertices, the only possible triangle 123 can be constructed. When there are 5 vertices, triangles 123, 134, and 451 can be constructed. And so on, up to 12 vertices. Other fixed rules for constructing triangles from vertices can be used.

[0046] Figure 3 An illustrative example is shown when a cuboid volume and four vertices (lower right) are projected onto a 2D plane (lower left) along a horizontal axis. The four vertices are then sorted in a clockwise order relative to the center of the projected cuboid volume (square). Two triangles are then constructed based on the ordered vertices according to the above fixed rule (upper left).

[0047] exist Figure 2 and Figure 3 In the example of Figure 2) shows the surface with the largest total 2D projection of the triangle on the 2D plane (larger than the total 2D projection area of ​​the triangle produced by horizontal projection on the 2D plane, Figure 3 ). Therefore, the vertical direction is chosen as the dominant direction, and two triangles are constructed from the vertices sorted according to the numbering order of these vertices obtained from the vertical projection.

[0048] The dominant direction is selected by maximizing the total 2D projected area of ​​the triangles on the 2D plane, leading to a continuous reconstruction of the hole-free point cloud.

[0049] In the current TriSoup encoding scheme, a ray tracing process is performed on each cuboid volume of the tree to render the triangles defined within each cuboid volume as rendering points representing (modeling) the points of the point cloud contained in the cuboid volume. The positions of all the rendering points traced by the ray will become the positions of the decoded points of the point cloud. This process is called "voxelization" of the triangles.

[0050] Basically, ray tracing on triangles of a cuboid volume consists in emitting rays in three directions parallel to the axes of the three-dimensional coordinate system, such as Figure 4 The starting point for emitting the ray is a point with integer (voxelized) coordinates, whose precision corresponds to the sampling precision required for rendering. The intersection of the ray with the triangle (the dotted point if any) is then voxelized (= rounded to the nearest point at the desired sampling precision) and added to the list of rendered points.

[0051] After applying the TriSoup encoding scheme to the cuboid volumes associated with all leaf nodes of the tree representing the point cloud geometry, i.e. after constructing triangles for each cuboid volume and obtaining rendering points by ray tracing, copies of the same rendering point in all lists of rendering points are discarded (i.e. only one voxel (rendering point) is kept among all voxels sharing the same three-dimensional position) to obtain a set of decoded (unique) points of the point cloud.

[0052] Constructing triangles following a fixed rule based on ordered vertices can lead to visually awkward results when more than 4 vertices are involved. In fact, for a given set of vertices, other triangles can be constructed that are different from the ones constructed by the above fixed rule, resulting in a different set of rendered points (different visual aspects of the decoded points of the point cloud). Moreover, it turns out that reconstructing triangles based on the above fixed rule is not optimal in terms of PSNR (Peak Signal-to-Noise Ratio) results.

[0053] The problem is to improve the current TriSoup encoding scheme to provide better and more stable visual aspects of the decoded point cloud and improve objective metrics (PSNR).

[0054] At least one embodiment of the present disclosure has been designed in view of the foregoing. Summary of the invention

[0055] The following section presents a simplified summary of at least one embodiment in order to provide a basic understanding of some aspects of the present disclosure. This summary is not an exhaustive overview of the embodiments. It is not intended to identify the key or important elements of the embodiments. The following summary only presents some aspects of at least one of the embodiments in a simplified form as a prelude to a more detailed description provided elsewhere in the document.

[0056] According to a first aspect of the present disclosure, a method for encoding positions of points of a point cloud contained in a cuboid volume into a bitstream is provided, wherein the positions are represented in a three-dimensional coordinate system, and at least three vertices are located on the sides of the cuboid volume. The method comprises obtaining the quantized positions of the at least three vertices by quantizing the positions of the at least three vertices based on a vertex quantization step size, wherein the vertex quantization step size depends on at least one quantization parameter; encoding vertex information defining the quantized positions of the at least three vertices into the bitstream; and encoding the at least one quantization parameter into the bitstream.

[0057] In one embodiment, the encoding method also includes dequantizing the positions of the at least three vertices based on the vertex quantization step to obtain the dequantized positions of the at least three vertices; determining the point in the volume of the rectangular parallelepiped by adding a centroid point and a one-dimensional residual position, wherein the centroid point is defined by the average value of the dequantized positions of the at least three vertices, and the one-dimensional residual position is defined as the product of a normal vector and a residual value; obtaining a quantized residual value by quantizing the residual value based on a residual quantization step, wherein the residual quantization step depends on the at least one quantization parameter; and encoding the quantized residual value into the bit stream.

[0058] According to a second aspect of the present disclosure, a method for decoding positions of points of a point cloud contained in a cuboid volume from a bitstream is provided, wherein the positions are expressed in a three-dimensional coordinate system. The method comprises decoding vertex information from the bitstream, wherein the vertex information defines at least three vertices located on the edge of the cuboid volume, one vertex for each edge; decoding at least one quantization parameter from the bitstream; dequantizing the positions of at least three vertices based on a vertex quantization step size to obtain dequantized positions of the at least three vertices, wherein the vertex quantization step size depends on the at least one quantization parameter; and obtaining the dequantized positions p of the at least three vertices from the dequantized positions p of the at least three vertices. dec At least one triangle is constructed; and the position of the rendering point obtained by voxelizing the at least one triangle represents the position of the point of the point cloud contained in the cuboid volume.

[0059] In one embodiment, the decoding method also includes decoding a quantized residual value from the bit stream; obtaining a dequantized residual value by dequantizing the quantized residual value based on a residual quantization step size, wherein the residual quantization step size depends on the at least one quantization parameter; determining a decoded determined point in the rectangular parallelepiped volume by adding a centroid point and a one-dimensional dequantized residual position, wherein the centroid point is defined by the average value of the dequantized positions of at least three vertices, and the one-dimensional dequantized residual position is defined as the product of a normal vector and the dequantized residual value; and constructing the at least one triangle by the dequantized positions of the at least three vertices and the decoded determined point.

[0060] In one embodiment, the vertex quantization step size is defined by a vertex quantization parameter, the residual quantization step size is defined by a residual quantization parameter, and the residual quantization parameter is equal to the sum of the vertex quantization parameter and at least one other quantization parameter called a QP offset parameter.

[0061] In one embodiment, the vertex quantization parameter and / or the QP offset parameter depends on the size of the cuboid volume.

[0062] In one embodiment, the size of the cuboid volume belongs to a list of vertex quantization parameters, each element of the list corresponds to a specific vertex quantization parameter of a specific size, wherein the list of vertex quantization parameters is encoded into / decoded from the bitstream.

[0063] In one embodiment, the quantization parameter is the sum of the first quantization parameter and the second quantization parameter.

[0064] In one embodiment, the first quantization parameter and the second quantization parameter belong to two different syntax elements encoded into / decoded from the bitstream.

[0065] In one embodiment, at least one quantization parameter of the geometry parameter set is encoded into / decoded from said bitstream.

[0066] In one embodiment, at least one quantization parameter of a slice header, a tile header or a sub-picture header is encoded into / decoded from the bitstream.

[0067] According to a third aspect of the present disclosure, there is provided a bitstream formatted to include encoded point cloud geometry video data obtained according to the method of the first aspect of the present disclosure.

[0068] According to a fourth aspect of the present disclosure, an apparatus is provided, which includes a device for executing one of the methods according to the first aspect and / or the second aspect of the present disclosure.

[0069] According to a fifth aspect of the present disclosure, a computer program product comprising instructions is provided. When the program is executed by one or more processors, the instructions cause the one or more processors to perform the method according to the first aspect and / or the second aspect of the present disclosure.

[0070] According to a sixth aspect of the present disclosure, a non-transitory storage medium is provided, which carries instructions of a program code for executing the method according to the first aspect and / or the second aspect of the present disclosure.

[0071] According to a seventh aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor. The processor is configured to execute one of the methods claimed in any one of claims 1 to 11.

[0072] The specific nature of at least one of the embodiments and other objects, advantages, features and uses of at least one of the embodiments will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Reference will now be made, by way of example, to the accompanying drawings which show embodiments of the present disclosure, and in which:

[0074] Figure 1 A TriSoup cuboid volume including four vertices is schematically shown;

[0075] Figure 2 An illustrative example of a process for determining a dominant direction based on a current TriSoup is shown;

[0076] Figure 3 An illustrative example of a process for determining a dominant direction based on a current TriSoup is shown;

[0077] Figure 4 schematically illustrates an example of ray tracing for rendering points of a point cloud according to one embodiment;

[0078] Figure 5 A schematic block diagram of a method 100 for encoding positions of points of a point cloud contained in a cuboid volume into a bitstream according to at least one embodiment is shown;

[0079] Figure 6 Schematically illustrates an example of a triangle constructed by four vertices of a cuboid volume and a determined decoding point according to one embodiment;

[0080] Figure 7 An example of residual values ​​along a line defined by a centroid point and a normal vector is shown;

[0081] Figure 8A schematic block diagram of a method 200 for decoding positions of points of a point cloud contained in a cuboid volume from a bitstream B according to at least one embodiment is shown;

[0082] Fig. 9 shows an illustrative example of a quantization step size ΔV for dequantized vertex positions of a cuboid volume and a quantization step size ΔC for a residual value α associated with a decoded determination point of the cuboid volume according to one embodiment;

[0083] Fig.10 shows an example of a surface locally defined by points of a point cloud contained by a cuboid volume according to one embodiment; and

[0084] Fig.11 A schematic block diagram of an example of a system in which various aspects and embodiments are implemented is illustrated.

[0085] Similar reference numerals may have been used in different drawings to denote similar components. DETAILED DESCRIPTION

[0086] At least one of the embodiments will be described more fully below with reference to the accompanying drawings, wherein an example of at least one of the embodiments is depicted. However, the embodiments may be implemented in many alternative forms and should not be construed as being limited to the examples set forth herein. Thus, it should be understood that it is not intended to limit the embodiments to the specific forms disclosed. On the contrary, the present disclosure is intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the present disclosure.

[0087] At least one aspect generally relates to point cloud encoding and decoding, another aspect generally relates to transmitting a generated or encoded bitstream, and another aspect generally relates to receiving / accessing a decoded bitstream.

[0088] Furthermore, the present aspects are not limited to MPEG standards such as MPEG-I Part 5 or Part 9 related to point cloud compression, and may be applied, for example, to other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations, including MPEG-I Part 5 and Part 9. Unless otherwise indicated, or technically excluded, the aspects described in this disclosure may be used alone or in combination.

[0089] As with lossy compression, a balance must be found between the bitrate used to encode the vertex information and the quality (PSNR, visual quality) of the decoded point cloud, such that the overall bitrate vs. distortion tradeoff is optimized.

[0090] Briefly, the present invention relates to a TriSoup encoding scheme. It provides quantizing the positions of vertices based on at least one quantization parameter and encoding the at least one quantization parameter into a bitstream. The vertex quantization parameter is decoded and the positions of the TriSoup vertices are dequantized (inverse quantized) based on the at least one decoded quantization parameter.

[0091] Quantizing the positions of the vertices improves the overall bit rate and distortion tradeoff for encoding the positions of the point cloud contained within the cuboid volume.

[0092] Figure 5 A schematic block diagram of a method 100 for encoding positions of points of a point cloud contained in a cuboid volume into a bitstream B according to at least one embodiment is shown.

[0093] Figure 5 The encoding of the positions of the points of the point cloud contained in the cuboid volume is described. Obviously, this method can be applied to all cuboid volumes representing the entire point cloud geometry.

[0094] The positions of the points and vertices of the point cloud contained in the cuboid volume are expressed in a three-dimensional coordinate system. At least three vertices are located on the sides of the cuboid volume.

[0095] In step 110, at least three vertices V are quantized based on the vertex quantization step size ΔV. i Quantize the position p of (i=1,…,M) to obtain the quantized position p of at least three vertices Q , the vertex quantization step size ΔV depends on at least one quantization parameter:

[0096] p Q =round(p / ΔV)

[0097] In the current TriSoup encoding scheme, no quantization of vertex positions is performed because for an N×N×N cuboid volume (node) (N=2 s ), the vertex position is encoded using s bits with the maximum possible precision, making the position accuracy as low as 1.

[0098] In step 120 , vertex information defining the quantized positions of at least three vertices is encoded into a bitstream B.

[0099] The vertex information of a vertex includes a vertex flag indicating whether the vertex exists on an edge of the cuboid volume and, if so, a quantized vertex position along the edge.

[0100] In step 130 , at least one quantization parameter is encoded into the bitstream B.

[0101] In one embodiment, the encoding method 100 may further include steps 140 to 180 .

[0102] In step 140, the quantized positions p of at least three vertices are quantized based on the vertex quantization step size ΔV. Q (Step 110) Dequantize to obtain the dequantized positions p of at least three vertices dec , the vertex quantization step size ΔV depends on at least one quantization parameter.

[0103] Dequantized position p dec Obtained by the following formula:

[0104] p dec =p Q *ΔV

[0105] In step 150, by adding the centroid point C mean and the one-dimensional residual position C res To determine the point C in the volume of the cuboid, the centroid point C mean The dequantized positions p of at least three vertices dec The average value of the one-dimensional residual position C res is defined as the normal vector The product of the residual value α:

[0106]

[0107] Center of mass point C mean The dequantized positions p of at least three vertices dec The average value of Figure 7 ). More precisely, the centroid point C mean The coordinates in the three-dimensional coordinate system are equal to the dequantized positions p of at least three vertices dec The average value of the coordinates.

[0108] The residual value α is a signed scalar value, such as Figure 7 As shown in the picture.

[0109] In one embodiment of step 150, the normal vector The vector can be calculated as the sum of the cross products (also called two vectors and The vector product or cross product between ) and by adding the vector Normalization is defined as follows

[0110]

[0111] Each vector Defined as:

[0112]

[0113] In one embodiment of step 150, the normal vector It can be parallel to the axes of the three-dimensional coordinate system to simplify its determination and the calculation of the residual value α.

[0114] In one embodiment of step 150, the normal vector May be parallel to the dominant direction defined above.

[0115] Such a normal vector is a good approximation of the normal vector calculated above.

[0116] The residual value α is determined and hence the residual position C res is determined so that the adjusted position of point C is closer to the original point of the point cloud contained in the cuboid volume.

[0117] In one embodiment of step 150, all points P of the point cloud belonging to the cuboid volume may be considered. k To determine the residual value α. For each point P k , which is consistent with the line The distance d k Found by:

[0118]

[0119] And if the distance d k If the value is lower than a certain threshold value th (for example, th = 2), point P is used. k The value α is calculated by scalar product (also called inner product or dot product) to obtain the value relative to the centroid point C. mean Point P k The one-dimensional residual r k

[0120]

[0121] Therefore, the residual value α is obtained by the following formula

[0122]

[0123] Where S is point P k A set of such that their distance d k is below a threshold th, and |S| is the number of points belonging to this set.

[0124] In step 160, the residual value α is quantized based on the residual quantization step size ΔC to obtain a quantized residual value α Q , the residual quantization step size ΔC depends on at least one quantization parameter.

[0125] In one embodiment of step 160, the quantized residual value α Q Obtained by the following formula:

[0126] α Q =round(α / ΔC).

[0127] Quantizing the residual value α improves the trade-off between overall bit rate and distortion for encoding the positions of the point cloud contained in the cuboid volume.

[0128] In step 170, the quantized residual value (α Q ) is encoded into bit stream B.

[0129] In one embodiment of step 170, the quantized residual value α Q can be binarized and each bit can be encoded using a binary entropy encoder (such as an arithmetic encoder or a context adaptive binary encoder like CABAC).

[0130] In the variant, the residual value α Q can be binarized as the flag f0 (indicating α Q Is it equal to 0), the symbol (indicating α Q >0 or α Q <0), flag f1 (indicating |α Q | is equal to 1) and the remainder |α encoded by the expGolomb encoder Q |-2.

[0131] In a variation of method 100, in step 180, the dequantized positions p of at least three vertices may be dec and the decoding determination point C determined by dec Construct at least one triangle:

[0132]

[0133] Where IQ(α Q ) is α Q The dequantized value of .

[0134] In one embodiment, IQ(α Q ) can be obtained as

[0135] IQ(α Q )=α Q *ΔC

[0136] In one embodiment of step 180, a dominant direction may be determined along one of the three axes of the three-dimensional coordinate system as discussed above. At least three vertices may be ordered based on the dominant direction, and the dequantized positions p of the at least three ordered vertices may be determined based on the dequantized positions p of the at least three ordered vertices.dec and the decoding determination point C dec Construct at least one triangle.

[0137] The constructed triangles in method 100 result in the positions of the rendered points obtained by voxelization of the constructed triangles. Ray tracing applied to the constructed triangles can be used. These positions represent the positions of the points of the point cloud contained in the cuboid volume. These positions can be used, for example, to encode the attributes of the points of the point cloud contained in the cuboid volume.

[0138] Figure 6 Schematically shows the order of four vertices V i The dequantized position p of (i=1,…,4) dec and the decoding determination point C dec Example of a constructed triangle.

[0139] In this illustrated example, by surrounding the decoded determination point C dec Rotation, by ordered vertices V i (i=1,…,4) constructs four triangles. The four constructed triangles are (V1V2C dec )、(V2V3C dec )、(V3V4C dec ) and (V4V1C dec ). More generally, the M constructed triangles consist of M ordered vertices V i (i=1,…,M) construction.

[0140] Figure 8 A schematic block diagram of a method 200 for decoding positions of points of a point cloud contained in a cuboid volume from a bitstream B according to at least one embodiment is shown.

[0141] In step 210, vertex information is decoded from the bitstream B. The vertex information defines at least three vertices located on the edges of the cuboid volume.

[0142] In step 220, at least one quantization parameter is decoded from the bitstream B.

[0143] In step 140, as discussed above, the positions p of at least three vertices (obtained in step 210) are quantized based on the vertex quantization step size ΔV. Q Dequantize to obtain the dequantized positions p of at least three vertices dec , the vertex quantization step size ΔV depends on at least one quantization parameter (obtained in step 220).

[0144] In step 180, as discussed above, the dequantized positions p of at least three dequantized vertices are calculated. dec Construct at least one triangle.

[0145] The positions of the rendered points obtained by voxelizing (eg, applying ray tracing) at least one triangle represent the positions of the points of the point cloud contained in the cuboid volume.

[0146] According to embodiments and variants of the method 100 , the method 200 further includes steps 230 to 250 .

[0147] In step 230, the quantized residual value α is decoded from the bitstream B. Q .

[0148] In step 240, the residual value α is quantized based on the residual quantization step size ΔC. Q Dequantize to obtain the dequantized (decoded) residual value (IQ(α Q )), the residual quantization step size ΔC depends on at least one quantization parameter,

[0149] IQ(α Q )=α Q *ΔC.

[0150] In step 250, by adding the centroid point C mean and the one-dimensional dequantized residual position to determine the decoded fixed point C in the cuboid volume dec , where the centroid point C mean The dequantized positions p of at least three vertices dec The one-dimensional dequantized residual position is defined as the normal vector and the dequantized residual value IQ(α Q )

[0151] In step 180, as discussed above, the dequantized positions p of at least three vertices are then dec and the decoding determination point C dec Construct at least one triangle.

[0152] Fig. 9 The quantization step size ΔV for dequantizing the vertex positions of the cuboid volume and the determination point C for decoding the cuboid volume are shown. dec Illustrative example of quantization step size ΔC for the associated residual value α.

[0153] One of the problems to be solved is to provide tools which allow to appropriately choose the vertex quantization step size ΔV and the residual quantization step size ΔC to optimize the trade-off between bitrate and distortion and / or to obtain a bitrate compatible with the transmission channel.

[0154] Vertex V i (with dequantized position p dec ) and the decoding determination point C dec Approximately on the surface ( Fig.10 The surface is defined locally by the points of the point cloud contained in the cuboid volume. The surface is approximated by the vertices V i and the decoding determination point C dec This is a first-order approximation of the surface, which results in a local error ε that is proportional to the distance between the interpolated points; this distance itself is proportional to the node size N. Therefore, we have

[0155] ε≈βN(β<1).

[0156] By setting the vertex quantization step size and the residual quantization step size equal to ΔV≈ΔC, uniform accuracy of the first-order approximation of the surface is obtained.

[0157] Making the vertex quantization step size and / or the residual quantization step size smaller than the error ε wastes bitrate because the surface approximation will not be significantly improved. On the other hand, instead of making the vertex quantization step size and / or the residual quantization step size larger than the error ε, a larger node size should be used to further reduce the bitrate.

[0158] Therefore, the logical trade-off is ε≈ΔV≈ΔC.

[0159] Thus, the number of bits b required to encode the vertex position is

[0160] b=log2(N / ΔV)≈log2(N / ε)≈log2(1 / β)

[0161] And this number b does not depend on the node size N, but on the local prior of the point cloud (parameter β).

[0162] In one embodiment, the vertex quantization step size ΔV is defined by a vertex quantization parameter QP V The quantization parameter definition.

[0163] In the variant, the vertex quantization parameter QP V Indicates the number of bits b used to uniformly encode / decode the dequantized vertex position p:

[0164] QP V = b = log2(N / ΔV)

[0165] Then the vertex quantization step size ΔV is given by:

[0166] ΔV = N*2 -QPv (1)

[0167] In the variant, the vertex quantization parameter QP V with 2 -QPv / 6 Then the vertex quantization step size ΔV is given by:

[0168] ΔV = N*2-QPv / 6 (2)

[0169] In the variant, the vertex quantization parameter QP V is the first vertex quantization parameter QP v,base and the second vertex quantization parameter QP v,offset The sum of:

[0170] QP v =QP v,base +QP v,offset .

[0171] First vertex quantization parameter QP v,base and the second vertex quantization parameter QP v,offset is the quantization parameter.

[0172] In a variant, the vertex quantization step size ΔV and the residual quantization step size ΔC are equal.

[0173] Since the quantized vertex position p Q and the quantized residual value α Q The cost of encoding may vary due to differences in the efficiency of prediction and entropy coding, so assuming ΔV ≈ ΔC may not provide the best result for optimizing the trade-off between bitrate and distortion.

[0174] In order to improve the coding efficiency and thus improve the trade-off between bit rate and distortion, in one embodiment, the residual quantization step size ΔC is determined by the residual quantization parameter QP c definition.

[0175] Residual quantization parameter QP c is the quantization parameter.

[0176] In the variant, the residual quantization parameter QP c Equal to the vertex quantization parameter QP V and QP offset parameter QP offset The sum of QP offset parameter QP offset Represents the quantized difference between the vertex position p and the residual value α:

[0177] QP c =QPv+QP offset .

[0178] QP offset parameter QP offset is the quantization parameter.

[0179] Then the quantization step size ΔC is proportional to the quantization step size ΔV, and the ratio of the quantization step size ΔC to the quantization step size ΔV depends on the QP offset parameter QP offset .

[0180] For example, there may be ΔC = ΔV * 2 -QPoffset =N*2-(QPv+QPoffset) (3)

[0181] For example, there may be ΔC = ΔV * 2 -QPoffset / 6 =N*2 -(QPv+QPoffset) / 6 (4)

[0182] In a variant, the QP offset parameter QP offset is the first QP offset parameter QP offset,base and the second QP offset parameter QP offset,offset The sum of:

[0183] QP offset =QP offset,base +QP offset,offset

[0184] First QP offset parameter QP offset,base and the second QP offset parameter QP offset,offset is the quantization parameter.

[0185] In one embodiment, the vertex quantization parameter QP V and / or QP offset parameter QP offset It can depend on the size N of the cuboid volume (node).

[0186] This improves the vertex quantization parameter QP V and the trade-off between bit rate and distortion, since the optimization between bit rate and distortion depends on N.

[0187] In one embodiment, the vertex quantization parameter QP is signaled in the bitstream B V A list of (N) where each element of the list corresponds to a specific vertex quantization parameter QP of a specific size N. V (N).

[0188] Vertex quantization parameter QP V (N) is the quantization parameter.

[0189] For example, N may be equal to 4, 8, 16, and 32, resulting in four vertex quantization parameters QP V (N) list.

[0190] Per-vertex quantization parameter QP V (N) may be the vertex quantization parameter QP obtained as discussed above V .

[0191] In the variant, the vertex quantization parameter QP V (N) is the first vertex quantization parameter QP v,base (N) and the second vertex quantization parameter QP v,offset (N) and: QP v (N) = QPv,base (N)+QP v,offset (N).

[0192] First vertex quantization parameter QP v,base (N) and the second vertex quantization parameter QP v,offset (N) is the quantization parameter.

[0193] In one embodiment, the vertex quantization parameter QP V (N) can be equal to the residual value quantization parameter QP c (N) and / or residual value quantization parameter QP c (N) can be determined by the vertex quantization parameter QP V (N) and QP offset parameter QP offset Infer.

[0194] In one variant, the vertex quantization parameter QP V (N) Vertex quantization parameter QP in the list V (N1) can be related to the vertex quantization parameter QP V Another vertex quantization parameter QP in the list of (N) V (N2) Perform differential encoding.

[0195] For example, a vertex quantization parameter QP in a list of vertex quantization parameters may be signaled first V (4) Then another vertex quantization parameter QP in the list of vertex quantization parameters can be V (8) and vertex quantization parameter QP V (4) The difference between them, i.e., QP V (8)-QP V (4) Encoding, rather than the vertex quantization parameter QP V (8) Encode to save some bits.

[0196] At least one quantization parameter used for quantizing / dequantizing the positions of vertices located on the edges of the cuboid volumes and the residual values ​​α of these cuboid volumes may be locally adapted to achieve a better trade-off between bitrate and distortion.

[0197] In order to perform such an adaptation, at least one quantization parameter may be dynamically provided and signaled (encoded / decoded) during encoding / decoding of the tree representing the point cloud geometry.

[0198] Signaling at least one quantization parameter refers to encoding at least one quantization parameter into the bitstream B (step 130) and decoding at least one quantization parameter from the bitstream B (step 220).

[0199] In one embodiment, at least one quantization parameter may be signaled for each TriSoup coded cuboid volume of the tree representing the point cloud geometry (ie the positions of the points of the point cloud contained in these cuboid volumes encoded according to the principles of the TriSoup coding scheme).

[0200] Then, at least one quantization parameter may be applied to a single TriSoup encoded cuboid volume (node) and all vertices (with dequantized positions) of the TriSoup encoded cuboid volume (node).

[0201] In one embodiment, at least one quantization parameter may be signaled for each TriSoup coded cuboid volume at a particular depth in a tree representing point cloud geometry, the signaled quantization parameters being applied to each TriSoup coded cuboid volume descending from the node from which they were signaled.

[0202] Then, at least one quantization parameter may be applied to all TriSoup coded cuboid volumes (nodes) at a particular depth in the tree and to all vertices (with dequantized positions) of said all TriSoup coded cuboid volumes (nodes) of the tree representing the point cloud geometry.

[0203] In one embodiment, at least one quantization parameter may be signaled in a geometry parameter set commonly used for G-PCC to signal geometry parameters used for encoding a point cloud.

[0204] Then, at least one quantization parameter may be applied to all TriSoup coded cuboid volumes (nodes) of the tree representing the point cloud geometry and all vertices (with dequantized positions) of said all TriSoup coded cuboid volumes (nodes).

[0205] In one embodiment, at least one quantization parameter may be signaled in a slice, tile or sub-picture header.

[0206] Then, at least one quantization parameter may be applied to all TriSoup coded cuboid volumes (nodes) relative to a slice, tile or sub-image and all vertices (with dequantized positions) of said all TriSoup coded cuboid volumes (nodes).

[0207] As defined in VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), a tile is a sequence of coding tree units (CTUs) covering a rectangular picture area of ​​a video image. A slice consists of an integer number of tiles or an integer number of consecutive complete CTU rows within a tile. A sub-image contains one or more slices that together cover a rectangular picture area.

[0208] When at least one first quantization parameter is signaled in a geometry parameter set and at least one second quantization parameter is signaled in a slice, tile or sub-image header, then the at least one first quantization parameter may be applied to all TriSoup coded cuboid volumes (nodes) of the tree and all TriSoup vertices of said all TriSoup coded cuboid volumes (nodes), and then at least one second quantization parameter may be applied to all TriSoup coded cuboid volumes (nodes) of the slice, tile or sub-image header and all TriSoup vertices of said all TriSoup coded cuboid volumes (nodes).

[0209] In one embodiment, signaling at least one quantization parameter may include signaling a vertex quantization parameter QP V .

[0210] In a variant, signaling at least one quantization parameter may include signaling a first vertex quantization parameter QP in two different syntax elements. v,base and the second vertex quantization parameter QP v,offset .

[0211] For example, the first vertex quantization parameter QP is signaled in the geometry parameter set v,base , signaling the second vertex quantization parameter QP in the slice, tile or sub-image header v,offset .

[0212] In one embodiment, signaling at least one quantization parameter may include signaling a residual quantization parameter QP c .

[0213] In one embodiment, signaling at least one quantization parameter may include signaling a QP offset parameter QP offset .

[0214] In one embodiment, signaling at least one quantization parameter may include signaling a first QP offset parameter QP in two different syntax elements offset,base and the second QP offset parameter QP offset,offset .

[0215] For example, the first QP offset parameter QP is signaled in the geometry parameter set offset,base , signaling a second QP offset parameter QP in the slice, tile or sub-picture header offset,offset .

[0216] In one embodiment, signaling at least one quantization parameter may include signaling a vertex quantization parameter QP in a syntax element representing a geometry parameter set. V (N) list.

[0217] In one embodiment, signaling at least one quantization parameter may include signaling a vertex quantization parameter QP in a syntax element representing a slice, tile, or sub-picture header. V (N) list.

[0218] In one embodiment, signaling at least one quantization parameter may include signaling a first vertex quantization parameter QP in two different syntax elements. v,base (N) and the second vertex quantization parameter QP v,offset (N).

[0219] For example, the first vertex quantization parameter QP is signaled in the geometry parameter set v,base (N) Signaling the second vertex quantization parameter QP in the slice, tile or sub-image header v,offset (N).

[0220] In one variant, signaling at least one quantization parameter may include signaling a vertex quantization parameter QP V (N) The first vertex quantization parameter QP in the list V (N1) and vertex quantization parameter QP V Another vertex quantization parameter QP in the list of (N) V The difference between (N2).

[0221] Fig.11 A schematic block diagram illustrating an example of a system in which various aspects and embodiments are implemented is shown.

[0222] System 300 may be embedded as one or more devices, including various components described below. In various embodiments, system 300 may be configured to implement one or more aspects described in the present disclosure.

[0223] Examples of equipment that may constitute all or part of system 300 include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMD, perspective glasses), projectors (projectors), "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing outputs from video decoders, pre-processors for providing inputs to video encoders, web servers, set-top boxes, and any other equipment for processing point clouds, videos or images, or other communication devices. The elements of system 300 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 300 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 300 may be coupled to other similar systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports.

[0224] The system 300 may include at least one processor 310 configured to execute instructions loaded therein for implementing, for example, various aspects described in the present disclosure. The processor 310 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 300 may include at least one memory 320 (e.g., a volatile memory device and / or a non-volatile memory device). The system 300 may include a storage device 340, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 340 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0225] The system 300 may include an encoder / decoder module 330, which is configured to, for example, process data to provide encoded / decoded point cloud geometry data, and the encoder / decoder module 330 may include its own processor and memory. The encoder / decoder module 330 may represent a module (or modules) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 330 may be implemented as a separate element of the system 300, or may be incorporated into the processor 310 as a combination of hardware and software known to those skilled in the art.

[0226] Program code to be loaded onto the processor 310 or the encoder / decoder 330 to perform various aspects described in the present disclosure may be stored in the storage device 340 and subsequently loaded onto the memory 320 to be executed by the processor 310. According to various embodiments, during the execution of the processes described in the present disclosure, one or more of the processor 310, the memory 320, the storage device 340, and the encoder / decoder module 330 may store one or more of various items. Such stored items may include, but are not limited to, point cloud frames, encoded / decoded geometry / attribute videos / images or portions of encoded / decoded geometry / attribute videos / images, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0227] In several embodiments, memory internal to the processor 310 and / or encoder / decoder module 330 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0228] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 310 or the encoder / decoder module 330) is used for one or more of these functions. The external memory may be a memory 320 and / or a storage device 340, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, such as for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), HEVC (High Efficiency Video Coding), VVC (Versatile Video Coding), or MPEG-I Part 5 or Part 9.

[0229] As indicated in block 390, input to the elements of system 300 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0230] In various embodiments, the input device of block 390 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting the signal band to within a band), (ii) down-converting the selected signal, (iii) limiting the band again to a narrower band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0231] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0232] Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0233] Adding an element may include inserting an element between existing elements, such as, for example, inserting an amplifier and an analog-to-digital converter.In various embodiments, the RF portion may include an antenna.

[0234] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 300 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 310 when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 310 when necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 310 and the encoder / decoder 330, which operate in conjunction with memory and storage elements to process the data streams for presentation on output devices when necessary.

[0235] The various elements of system 300 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 390, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0236] System 300 may include a communication interface 350 that enables communication with other devices via a communication channel 700. Communication interface 350 may include, but is not limited to, a transceiver configured to send and receive data over communication channel 700. Communication interface 350 may include, but is not limited to, a modem or a network card, and communication channel 700 may be implemented, for example, within a wired and / or wireless medium.

[0237] In various embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 300. The Wi-Fi signals of these embodiments may be received through a communication channel 700 suitable for Wi-Fi communications and a communication interface 350. The communication channel 700 of these embodiments may typically be connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0238] Other embodiments may provide streamed data to system 300 using a set top box that delivers the data through an HDMI connection to input block 390 .

[0239] Still other embodiments may use the RF connection of input block 390 to provide streaming data to system 300 .

[0240] The streamed data may be used as a means of signaling information used by the system 300. The signaling information may include a bit stream B and / or information such as the number of points of a point cloud, coordinates and / or sensor setting parameters.

[0241] It should be appreciated that signaling may be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0242] The system 300 may provide output signals to various output devices, including a display 400, speakers 500, and other peripheral devices 600. In various examples of embodiments, the other peripheral devices 600 may include one or more of a stand-alone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 300.

[0243] In various embodiments, control signals may be communicated between the system 300 and the display 400, speakers 500, or other peripheral devices 600 using signaling such as AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention.

[0244] Output devices may be communicatively coupled to system 300 via dedicated connections through respective interfaces 360 , 370 , and 380 .

[0245] Optionally, output devices may be connected to system 300 using communication channel 700 via communication interface 350. Display 400 and speaker 500 may be integrated in a single unit with other components of system 300 in an electronic device such as, for example, a television.

[0246] In various embodiments, the display interface 360 ​​may include a display driver, such as, for example, a timing controller (TCon) chip.

[0247] For example, if the RF portion of input 390 is part of a separate set-top box, then display 400 and speaker 500 may optionally be separate from one or more of the other components. In various embodiments where display 400 and speaker 500 may be external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0248] exist Figures 1 to 11 In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0249] Some examples are described about block diagrams and / or operational flow charts. Each square block represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the square block may not occur in the order indicated. For example, depending on the functions involved, two square blocks shown in succession can actually be executed substantially concurrently, or sometimes these square blocks can be executed in reverse order.

[0250] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0251] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device.

[0252] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Considering the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein can be considered as a non-transient storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. It should be appreciated that although more specific examples of computer-readable storage media to which the present embodiment can be applied are provided below, as those of ordinary skill in the art will readily recognize, it is merely an illustrative and non-exhaustive list: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0253] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0254] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0255] Device can be realized in suitable hardware, software and firmware, for example.Examples of such devices include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, head-mounted display devices (HMD, perspective glasses), projectors (projectors), "cave" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing outputs from video decoders, pre-processors for providing inputs to video encoders, web servers, set-top boxes, and any other equipment for processing point clouds, videos or images, or other communication devices.It should be clear that equipment can be mobile and even installed in a mobile vehicle.

[0256] The computer software may be implemented by the processor 310 or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments may also be implemented by one or more integrated circuits. The memory 320 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). The processor 310 may be of any type suitable for the technical environment and may encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0257] As will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0258] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor. The processor is configured to execute any of the methods described above.

[0259] The terms used herein are only used to describe the purpose of specific embodiments and are not intended to be limited. As used herein, the singular forms "one", "a kind of" and "the" may also be intended to include plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "response" or "connection" to another element, it may directly respond or be connected to another element, or there may be an intermediate element. On the contrary, when an element is referred to as "direct response" or "direct connection" to another element, there is no intermediate element.

[0260] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0261] Various numerical values ​​may be used in the present disclosure. Specific values ​​may be used for example purposes and the described aspects are not limited to these specific values.

[0262] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of the present disclosure, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0263] References to "one embodiment" or "an embodiment" or "one implementation" or "an implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with the embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in various places in this disclosure are not necessarily all referring to the same embodiment.

[0264] Similarly, references herein to "according to an embodiment / example / implementation" or "in an embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an embodiment / example / implementation) may be included in at least one embodiment / example / implementation. Therefore, the expressions "according to an embodiment / example / implementation" or "in an embodiment / example / implementation" appearing in various places in the specification do not necessarily all refer to the same embodiment / example / implementation, nor do separate or alternative embodiments / examples / implementations necessarily exclude other embodiments / examples / implementations.

[0265] Reference numerals appearing in the claims are for illustration only and have no limiting effect on the scope of the claims.The present embodiments / examples and variants may be employed in any combination or sub-combination although not explicitly described.

[0266] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0267] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0268] Various embodiments relate to decoding. As used in this disclosure, "decoding" may encompass, for example, all or part of a process performed on a received point cloud frame (which may include a received bitstream encoding one or more point cloud frames) to produce a final output suitable for display or further processing in a reconstructed point cloud domain. In various embodiments, such processes include one or more of the processes typically performed by a decoder. In various embodiments, for example, such processes also or alternatively include processes performed by a decoder of various embodiments described in this disclosure.

[0269] As a further example, in one embodiment "decoding" may refer only to dequantization, in one embodiment "decoding" may refer to entropy decoding, in another embodiment "decoding" may refer only to differential decoding, and in another embodiment "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. Based on the context of the specific description, whether the phrase "decoding process" may be intended to specifically refer to a subset of operations, or generally refer to a broader decoding process will be clear and is believed to be well understood by those skilled in the art.

[0270] Various embodiments are related to encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in the present disclosure may encompass, for example, all or part of a process performed on an input point cloud frame to produce an encoded bitstream. In various embodiments, such a process includes one or more of the processes typically performed by an encoder. In various embodiments, such a process also includes or optionally includes a process performed by an encoder of the various embodiments described in the present disclosure.

[0271] As a further example, in one embodiment "encoding" may refer only to quantization, in one embodiment "encoding" may refer only to entropy coding, in another embodiment "encoding" may refer only to differential coding, and in another embodiment "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Based on the context of the particular description, whether the phrase "encoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader encoding process will be clear and is believed to be well understood by those skilled in the art.

[0272] Additionally, this disclosure may refer to “obtaining” various information. Obtaining information may include, for example, one or more of: estimating information, calculating information, predicting information, or retrieving information from a memory.

[0273] Additionally, this disclosure may refer to "accessing" various information. Accessing information may include, for example, one or more of: receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0274] Furthermore, the present disclosure may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information may include one or more of: for example, accessing information or retrieving information (e.g., from a memory). Additionally, in one way or another, during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is typically involved.

[0275] Moreover, as used herein, the word "signal" refers in particular to indicating something to a corresponding decoder, etc. For example, in some embodiments, the encoder signals specific information, such as the number or coordinates of points of a point cloud or sensor setting parameters. In this way, in an embodiment, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit savings are achieved in various embodiments. It should be appreciated that signaling can be accomplished in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" can also be used as a noun in this article.

[0276] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present disclosure contemplates these and other embodiments.

Claims

1. A method for encoding into a bitstream the positions of points of a point cloud contained in a cuboid volume, said positions being expressed in a three-dimensional coordinate system, at least three vertices being located on the edges of said cuboid volume, wherein said method include: - obtaining (110) quantized positions (p) of the at least three vertices by quantizing the positions of the at least three vertices based on the vertex quantization step size (ΔV); Q ), wherein the vertex quantization step size (ΔV) depends on at least one quantization parameter; - defining the quantized positions (p Q ) into the bitstream; as well as - encoding (130) said at least one quantization parameter into said bitstream.

2. The method according to claim 1, wherein the method further include: - obtaining (140) dequantized positions (p) of the at least three vertices by dequantizing the positions of the at least three vertices based on the vertex quantization step size (ΔV) dec ); - By adding the centroid point (C mean ) and the one-dimensional residual position (C res ) to determine (150) a point (C) in the volume of the cuboid, the centroid point (C mean ) is composed of the dequantized positions (p dec ) is defined by the average value of the one-dimensional residual position (C res ) is defined as the normal vector The product of the residual value (α); - quantizing the residual value (α) based on the residual quantization step size (ΔC) to obtain (160) a quantized residual value (α) Q ), the residual quantization step size (ΔC) depends on the at least one quantization parameter; as well as - The quantized residual value (α Q ) is encoded (170) into the bitstream.

3. A method for decoding from a bit stream the positions of points of a point cloud contained in a cuboid volume, said positions being expressed in a three-dimensional coordinate system, wherein said method include: - decoding (210) vertex information from the bitstream, the vertex information defining at least three vertices located on the edges of the cuboid volume, one vertex for each edge; - decoding (220) at least one quantization parameter from said bitstream; - obtaining (140) dequantized positions (p) of the at least three vertices by dequantizing the positions of the at least three vertices based on the vertex quantization step size (ΔV) dec ), wherein the vertex quantization step size (ΔV) depends on the at least one quantization parameter; as well as - the dequantized positions p of the at least three vertices dec constructing (180) at least one triangle; - the positions of the rendered points obtained by voxelizing the at least one triangle represent the positions of the points of the point cloud contained in the cuboid volume.

4. The method according to claim 3, wherein the method further include: - decoding (230) the quantized residual value (α Q ); - by quantizing the residual value (α Q ) is dequantized to obtain (240) the dequantized residual value (IQ(α Q )), wherein the residual quantization step size (ΔC) depends on the at least one quantization parameter; - By adding the centroid point (C mean ) and the one-dimensional dequantized residual position to determine (250) a decoded determination point (C dec ), the centroid point (C mean ) is composed of the dequantized positions (p) of at least three vertices dec ), the one-dimensional dequantized residual position is defined as the normal vector and the dequantized residual value (IQ(α Q )); as well as - the dequantized positions (p) of the at least three vertices dec ) and the decoding determination point (C dec ) construct (180) the at least one triangle.

5. The method according to any one of claims 1 to 4, wherein the vertex quantization step size (ΔV) is determined by a vertex quantization parameter (QP V ), the residual quantization step size (ΔC) is defined by the residual quantization parameter (QP c ) definition, and the residual quantization parameter (QP c ) is equal to the vertex quantization parameter (QP V ) and at least one parameter called QP offset (QP offset ) and the sum of other quantization parameters.

6. The method of claim 5, wherein the vertex quantization parameter (QP V ) and / or the QP offset parameter (QP offset ) depends on the size of the volume of the cuboid (N).

7. The method according to claim 6, wherein the size (N) of the cuboid volume belongs to a list of vertex quantization parameters, each element of the list corresponding to a specific vertex quantization parameter (QP) of a specific size (N). V (N)), wherein the list of vertex quantization parameters is encoded into / decoded from the bitstream.

8. The method according to claim 6 or 7, wherein the quantization parameter is a first quantization parameter (QP v,base , QP offset,base ) and the second quantization parameter (QP v,offset , QP offset,offset ) and.

9. The method according to claim 8, wherein the first quantization parameter (QP v,base , QP offset,base ) and the second quantization parameter (QP v,offset , QP offset,offset ) belong to two different syntax elements that are encoded into / decoded from the bitstream.

10. The method according to any one of claims 1 to 9, wherein at least one quantization parameter in a geometry parameter set is encoded into / decoded from the bitstream.

11. The method according to any one of claims 1 to 10, wherein at least one quantization parameter of a slice header, a tile header or a sub-picture header is encoded into / decoded from the bitstream.

12. A bitstream formatted to include encoded point cloud geometry video data obtained by the method of any one of claims 1, 2, 5 to 11.

13. An apparatus comprising means for performing one of the methods as claimed in any one of claims 1 to 11.

14. A computer program product comprising instructions which, when executed by one or more processors, cause the one or more processors to perform the method as claimed in any one of claims 1 to 11.

15. A non-transitory storage medium carrying program code instructions for executing the method claimed in any one of claims 1 to 11.

16. An electronic device, include: processor; as well as a memory for storing instructions executable by the processor; The processor is configured to perform one of the methods claimed in any one of claims 1 to 11.

Citation Information

Patent Citations

  • Method and apparatus for point cloud compression

    CN113632142A

  • Improved encoding and decoding of geometry data in 3d mesh models

    GB2551387B

  • Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device

    US20210368186A1

  • Scale factor for quantization parameter values in geometry-based point cloud compression

    US20210409714A1

  • Vertex correction method and apparatus for rotated three-dimensional (3D) components

    WO2013155687A1