Coding method and apparatus, point cloud coder, code stream, device, storage medium

Abstract

The application discloses a coding method and device, a point cloud coder, a code stream, equipment and a storage medium, wherein the decoding method comprises the following steps: decoding a code stream, determining residual values of transform coefficients of a current transform block; determining multiple frames or multiple pieces of reference point clouds of the current transform block; determining a reference transform block of the current transform block from the multiple frames or the multiple pieces of reference point clouds; determining a prediction value of the transform coefficients of the current transform block according to transform coefficients of the reference transform block of the current transform block; and determining the transform coefficients of the current transform block according to the residual values and the prediction value of the transform coefficients of the current transform block.

Description

Coding and decoding method and device, point cloud codec, code stream, device, storage medium Technical Field

[0001] The embodiments of the present application relate to point cloud compression technology, including but not limited to encoding and decoding methods and devices, point cloud codecs, bit streams, devices, and storage media. Background Art

[0002] A point cloud is a collection of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Point cloud data typically includes both geometric and attribute information about the sampling points. Geometric information includes the three-dimensional position (x, y, z) of the sampling points, while attribute information includes color information and / or one-dimensional reflectance information.

[0003] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Moreover, since point clouds are obtained by directly sampling real objects, they can provide a strong sense of reality while ensuring accuracy. Therefore, they are widely used, including virtual reality games, computer-aided design, geographic information systems, automatic navigation systems, digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive remote presentation, and three-dimensional reconstruction of biological tissues and organs.

[0004] As application demands grow, the processing of massive amounts of three-dimensional (3D) point cloud data is facing bottlenecks in storage space and transmission bandwidth. To better manage data, conserve server storage space, and reduce transmission traffic and time between servers and clients, point cloud compression has become a key issue in promoting the development of the point cloud industry.

[0005] Summary of the Invention

[0006] The encoding and decoding methods and apparatuses, point cloud codecs, bitstreams, devices, and storage media provided in the embodiments of this application are intended to reduce codeword overhead at the encoding end and improve the quality of point cloud attributes at the decoding end. The encoding and decoding processing methods and apparatuses, point cloud codecs, bitstreams, devices, and storage media provided in the embodiments of this application are implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, which is applied to a point cloud decoder, and the method includes: decoding a code stream to determine the residual value of the transformation coefficient of the current transformation block; determining a multi-frame or multi-slice reference point cloud of the current transformation block; determining a reference transformation block of the current transformation block from the multi-frame or multi-slice reference point cloud; determining a predicted value of the transformation coefficient of the current transformation block based on the transformation coefficient of the reference transformation block of the current transformation block; and determining the transformation coefficient of the current transformation block based on the residual value and predicted value of the transformation coefficient of the current transformation block.

[0008] It can be understood that in the decoding method provided in the embodiment of the present application, the number of reference point clouds participating in the prediction of the transform coefficients of the current transform block is the reference point clouds of multiple frames or multiple slices, rather than the reference point clouds of a single frame or a single slice; this is beneficial to improving the prediction accuracy of the transform coefficients of the current transform block, thereby improving the accuracy of the transform coefficients of the current transform block, and further improving the quality of the point cloud attributes at the decoding end.

[0009] In a second aspect, an embodiment of the present application provides an encoding method, which is applied to a point cloud encoder, and the method includes: determining multiple frames or multiple slices of reference point clouds for a current transform block; determining a reference transform block of the current transform block from the multiple frames or multiple slices of reference point clouds; determining a predicted value of the transform coefficient of the current transform block based on the transform coefficient of the reference transform block of the current transform block; determining a residual value of the transform coefficient of the current transform block based on the transform coefficient of the current transform block and the predicted value of the transform coefficient of the current transform block; and generating a code stream based on the residual value of the transform coefficient of the current transform block.

[0010] It can be understood that in the encoding method provided in the embodiment of the present application, the number of reference point clouds participating in the prediction of the transform coefficients of the current transform block is multiple frames or multiple slices, rather than a single frame or a single slice; this is beneficial to improving the prediction accuracy of the transform coefficients of the current transform block, thereby helping to reduce the residual value of the transform coefficients of the current transform block, and further helping to reduce the codeword overhead at the point cloud encoding end.

[0011] In a third aspect, an embodiment of the present application provides a decoding device, which is applied to a point cloud decoder, and the device includes: a decoding module, configured to decode a code stream and determine the residual value of the transformation coefficient of the current transformation block; a first determination module, configured to determine multiple frames or multiple pieces of reference point clouds of the current transformation block; a second determination module, configured to determine the reference transformation block of the current transformation block from the multiple frames or multiple pieces of reference point clouds; a third determination module, configured to determine the predicted value of the transformation coefficient of the current transformation block based on the transformation coefficient of the reference transformation block of the current transformation block; a fourth determination module, configured to determine the transformation coefficient of the current transformation block based on the residual value and predicted value of the transformation coefficient of the current transformation block.

[0012] In a fourth aspect, an embodiment of the present application provides a point cloud decoder comprising a first memory and a first processor; wherein the first memory is used to store a computer program that can be run on the first processor; and the first processor is used to execute the decoding method described in the embodiment of the present application when running the computer program.

[0013] In a fifth aspect, an embodiment of the present application provides a point cloud code stream, which is generated by the encoding method described in an embodiment of the present application.

[0014] In a sixth aspect, an embodiment of the present application provides an encoding device, which is applied to a point cloud encoder, and the device includes: a fifth determination module, configured to determine multiple frames or multiple pieces of reference point clouds of a current transform block; a sixth determination module, configured to determine the reference transform block of the current transform block from the multiple frames or multiple pieces of reference point clouds; a seventh determination module, configured to determine the predicted value of the transform coefficient of the current transform block based on the transform coefficient of the reference transform block of the current transform block; an eighth determination module, configured to determine the residual value of the transform coefficient of the current transform block based on the transform coefficient of the current transform block and the predicted value of the transform coefficient of the current transform block; and an encoding module, configured to generate a code stream based on the residual value of the transform coefficient of the current transform block.

[0015] In the seventh aspect, an embodiment of the present application provides a point cloud encoder, comprising a second memory and a second processor; wherein the second memory is used to store a computer program that can be run on the second processor; and the second processor is used to execute the encoding method described in the embodiment of the present application when running the computer program.

[0016] In an eighth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, suitable for executing a computer program; a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the decoding method described in the embodiment of the present application is implemented, or when the computer program is executed by the processor, the encoding method described in the embodiment of the present application is implemented.

[0017] In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the decoding method described in the embodiment of the present application, or implements the encoding method described in the embodiment of the present application.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, serve to illustrate the technical solutions of the present application. Obviously, the drawings described below are merely some embodiments of the present application. Those skilled in the art can, without inventive effort, derive other drawings from these drawings.

[0020] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0021] Figure 1 is an example of a three-dimensional point cloud image;

[0022] FIG2 is a partial enlarged view of the three-dimensional point cloud image;

[0023] Figure 3 is an example of a point cloud image;

[0024] FIG4 is a data storage format of the point cloud image shown in FIG3 ;

[0025] FIG5 is a schematic diagram of the structure of a point cloud encoder in the MPEG G-PCC attribute codec framework;

[0026] Figure 6 shows the structure of the point cloud decoder in the MPEG G-PCC attribute codec framework;

[0027] FIG7 is a schematic diagram of a transformation process of a Region Adaptive Hierarchical Transform (RAHT) along the x, y, and z directions;

[0028] FIG8 is a schematic diagram of a RAHT transformation structure;

[0029] FIG9 is a schematic diagram of a RAHT forward transformation process;

[0030] FIG10 is a schematic diagram of a RAHT inverse transformation process;

[0031] FIG11 is a schematic diagram of the structure of an attribute coding block;

[0032] FIG12 is a schematic diagram of the overall process of RAHT attribute prediction transform coding;

[0033] FIG13 is a schematic diagram of a neighborhood prediction relationship of a current block;

[0034] FIG14 is a schematic diagram of a calculation process of an attribute transformation coefficient;

[0035] FIG15 is a schematic diagram of the structure of a RAHT attribute inter-frame prediction coding;

[0036] FIG16 is a first schematic diagram of an implementation flow of a decoding method provided in an embodiment of the present application;

[0037] FIG17 is a schematic diagram of an implementation flow of the encoding method provided in an embodiment of the present application;

[0038] FIG18 is a second schematic diagram of the implementation flow of the decoding method provided in an embodiment of the present application;

[0039] FIG19 is a schematic diagram of multi-frame inter-frame prediction provided by an embodiment of the present application;

[0040] FIG20 is a schematic diagram of the structure of a decoding device provided in an embodiment of the present application;

[0041] FIG21 is a schematic diagram of the structure of an encoding device provided in an embodiment of the present application;

[0042] FIG22 is a schematic diagram of the structure of a point cloud decoder provided in an embodiment of the present application;

[0043] Figure 23 is a structural diagram of the point cloud encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application but do not limit the scope of the present application.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0046] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0047] In the RAHT codec mode, the terms "current block to be coded," "current block," "current node to be coded," and "current node" mentioned herein can all be understood as referring to the "current transform block." The terms "predicted value of the transform coefficient of the current transform block" and "predicted value of the attribute transform coefficient of the current transform block" can also be understood as referring to the "attribute transform prediction value of the current transform block" or the "predicted value of the attribute transform coefficient of the current transform block."

[0048] A point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Figure 1 is an example of a three-dimensional point cloud image, and Figure 2 is a partial magnified view of the three-dimensional point cloud image. It can be seen that the point cloud surface is composed of densely distributed points.

[0049] Two-dimensional images contain information at every pixel, and their distribution is regular, so there's no need to record their location information. However, the distribution of points in a point cloud in three-dimensional space is random and irregular, so recording the location of each point in space is necessary to fully represent a point cloud. Similar to two-dimensional images, each location in the acquisition process has corresponding attribute information, typically an RGB color value, which reflects the object's color. For point clouds, in addition to color, reflectance is also a common attribute for each point, reflecting the surface material of the object. Therefore, a point cloud typically includes geometric information (x, y, z) composed of three-dimensional position information, as well as attribute information such as three-dimensional color information (r, g, b) and one-dimensional reflectance information (r). Figure 3 shows an example of a point cloud image, and Figure 4 shows the data storage format of the point cloud image shown in Figure 3. Among them, Figure 3 provides six viewing angles of the point cloud image; Figure 4 consists of a file header information part and a data part. The header information includes the data format, data representation type, the total number of point cloud points, and the content represented by the point cloud. For example, the point cloud in this example is in the ".ply" format, represented by ASCII code, with a total number of 207242 points, and each point has three-dimensional position information (x, y, z) and three-dimensional color information (r, g, b).

[0050] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Moreover, since point clouds are obtained by directly sampling real objects, they can provide a strong sense of reality while ensuring accuracy. Therefore, they are widely used, including virtual reality games, computer-aided design, geographic information systems, automatic navigation systems, digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive remote presentation, and three-dimensional reconstruction of biological tissues and organs.

[0051] Point clouds are primarily collected through computer generation, 3D laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual 3D objects and scenes; 3D laser scanning can obtain point clouds of static real-world 3D objects or scenes, generating millions of point clouds per second; and 3D photogrammetry can obtain point clouds of dynamic real-world 3D objects or scenes, generating tens of millions of point clouds per second. These technologies reduce the cost and time required to acquire point cloud data while improving data accuracy. While changes in point cloud data acquisition methods have made it possible to acquire large amounts of point cloud data, the processing of this massive amount of 3D point cloud data is facing bottlenecks due to storage space and transmission bandwidth constraints, as application demands grow.

[0052] Taking a point cloud video with a frame rate of 30 fps (frames per second) as an example, each frame contains 700,000 points, each with float coordinate information (x, y, z) and uchar color information (r, g, b). Therefore, the data volume of a 10-second point cloud video is approximately 0.7 million (4 bytes 3 + 1 byte 3) 30 fps 10 seconds = 3.15 GB. For a 1280 x 720 2D video with a YUV sampling format of 4:2:0 and a frame rate of 24 fps, the data volume for 10 seconds is approximately 1280 x 720 12-bit 24 frames 10 seconds = 0.33 GB. A 10-second two-view 3D video has a data volume of approximately 0.33 x 2 = 0.66 GB. This shows that the data volume of a point cloud video far exceeds that of 2D and 3D videos of the same length. Therefore, in order to better realize data management, save server storage space, and reduce the transmission traffic and transmission time between the server and the client, point cloud compression has become a key issue in promoting the development of the point cloud industry.

[0053] In the MPEG G-PCC point cloud encoder framework, the geometric information of the point cloud and the attribute information corresponding to each point are encoded separately. The G-PCC reference software attribute encoding framework can be divided into a Pred branch, a PredLift branch, and / or a RAHT branch.

[0054] Figure 5 is a schematic diagram of the point cloud encoder architecture within the MPEG G-PCC attribute codec framework. As shown in Figure 5, during the geometry encoding process, the geometric information is transformed so that all point clouds are contained within a bounding box. Quantization then occurs. This quantization step primarily serves a scaling purpose. Due to quantization rounding, some point clouds have identical geometric information. Parameters are then used to determine whether to remove duplicate points. This process of quantization and removing duplicate points is also known as voxelization. The bounding box is then partitioned into an octree or a prediction tree is constructed. During this process, arithmetic coding is performed on the points in the leaf nodes of the partition to generate a binary geometry bitstream. Alternatively, arithmetic coding is performed on the intersection points (vertices) generated by the partition (surface fitting is performed based on the intersections) to generate a binary geometry bitstream. During the attribute encoding process, after the geometry encoding is completed and the geometric information is reconstructed, color conversion is performed to convert the color information (i.e., attribute information) from RGB color space to YUV color space. The reconstructed geometry is then used to recolor the point cloud, aligning the unencoded attribute information with the reconstructed geometry. Attribute encoding is mainly performed on color information. In the color information encoding process, there are two main transformation methods. One is the distance-based lifting transformation that relies on the level of detail (LOD) division, and the other is the direct region adaptive hierarchical transformation (RAHT). Both methods convert color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through transformation, and finally quantize the coefficients to obtain quantized coefficients; then, the quantized coefficients are arithmetically encoded to generate a binary attribute bit stream.

[0055] Figure 6 shows the structure of the point cloud decoder in the MPEG G-PCC attribute codec framework. As shown in Figure 6, for the acquired binary bit stream, the geometric bit stream and attribute bit stream in the binary bit stream are first decoded independently. When decoding the geometric bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-reconstruction of the octree / reconstruction of the prediction tree-reconstruction of the geometry-coordinate inverse conversion; when decoding the attribute bit stream, the attribute information of the point cloud is obtained through arithmetic decoding-inverse quantization-LOD partitioning / RAHT-color inverse conversion, and the point cloud data to be encoded (i.e., the output point cloud) is restored based on the geometric information and attribute information. The attribute bit stream and geometric bit stream shown in Figures 5 and 6 can be understood as point cloud code streams or code streams.

[0056] As shown in Figure 5 or Figure 6, the geometric coding and decoding of MPEG G-PCC can be divided into octree-based geometric coding and decoding (marked with a dotted box) and prediction tree-based geometric coding and decoding (marked with a dotted box). It should be noted that the framework of the point cloud encoder and point cloud decoder and the business scenario described in the embodiment of the present application are for the purpose of more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. It is known to those skilled in the art that with the evolution of point cloud encoders and point cloud decoders and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0057] The G-PCC encoding framework currently includes three attribute encoding methods: Predicting Transform (PT), Lifting Transform (LT), and Region Adaptive Hierarchical Transform (RAHT). The first two predictively encode point clouds based on the order in which LODs are generated, while RAHT adaptively transforms attribute information from the bottom up based on the construction hierarchy of a multi-tree (such as an octree).

[0058] Regional Adaptive Hierarchical Transform (RAHT) is a Haar wavelet transform that can transform point cloud attribute information from the spatial domain to the frequency domain, further reducing the correlation between point cloud attributes. Its main idea is to transform the nodes in each layer from the three dimensions of x, y, and z in a bottom-up manner according to the multi-tree (such as octree) structure (as shown in Figure 7), and iterate until the root node of the octree. As shown in Figure 8, its basic idea is to perform wavelet transform based on the hierarchical structure of the octree, associate attribute information with the octree nodes, and recursively transform the attributes of the occupied nodes in the same parent node in a bottom-up manner. For each layer, the nodes are transformed from the three dimensions of x, y, and z until the root node of the octree is reached. In the process of hierarchical transformation, the low-pass / low-frequency (DC) coefficients obtained after the transformation of the nodes in the same layer are passed to the nodes in the next layer for further transformation, and all high-pass / high-frequency (AC) coefficients are encoded by the arithmetic encoder. In Figure 8, g represents the low-frequency coefficient and h represents the high-frequency coefficient. At the encoding end, the low-frequency coefficient g needs to be converted into the low-frequency coefficient. 0,0 The residual and high-frequency coefficient h 0,0 As shown in Figure 8, RAHT forward transform and RAHT inverse transform are relative. Both RAHT forward transform and RAHT inverse transform can be called RAHT transform. The difference is that the transformation process of the two is opposite.

[0059] During the transformation process, the DC coefficients (direct current components) of the transformed nodes at the same layer are passed to the previous layer for further transformation, while the AC coefficients (alternating current components) of each layer are quantized and encoded. The following describes the main transformation processes.

[0060] Figure 9 is a schematic diagram of a RAHT forward transformation process, and Figure 10 is a schematic diagram of a RAHT inverse transformation process. Assume that g′ L,2x,y,z and g′L,2x+1,y,z are the DC coefficients of two neighboring points in the L layer. After linear transformation, the information of the L-1 layer is the AC coefficient f′ L-1,x,y,z and DC coefficient g′ L-1,x,y,z ; Then, f′ L-1,x,y,z No more transformation will be performed, and quantization coding will be performed directly, g′ L-1,x,y,z The nearest neighbor will continue to be searched for transformation. If no neighbor is found, it will be directly passed to the L-2 layer. That is, the RAHT transformation is only effective for nodes with neighbor points. Nodes without neighbor points will be directly passed to the previous layer. In the above transformation process, g′ L,2x,y,z The weights (the number of non-empty child nodes in the node) corresponding to g′L, 2x+2, y, and z are w′ respectively. L,2x,y,z and w′L,2x+1,y,z (abbreviated as w′0 and w′1), g′ L-1,x,y,z The weight is w′ L-1,x,y,z , then the general transformation formula is as follows:

[0061] Where T w0,w1 is the transformation matrix, and its expression is shown in the following formula (2):

[0062] The transformation matrix will be updated as the weights corresponding to each point change adaptively. The above process will be iterated and updated continuously according to the partitioning structure of the octree until the root node of the octree is reached.

[0063] In a specific implementation, for regional adaptive hierarchical intra-frame prediction transform coding, prediction can be performed based on RAHT transform coding. As shown in Figure 8, RAHT attribute transform is based on the order of the octree hierarchy, and the transformation is continuously performed from the voxel level until the root node is obtained, thereby completing the hierarchical transform coding of the entire attribute. In predictive transform coding, attribute prediction transform coding is also performed based on the hierarchical order of the octree, but the transformation is continuously performed from the root node to the voxel level. In each RAHT attribute transformation process, attribute prediction transform coding is performed based on a 2×2×2 block. Specifically, as shown in Figure 11, it can be seen that the grid filling block is the current block to be encoded, and the diagonal filling block is some neighboring blocks that are coplanar and colinear with the current block to be encoded. Among them, the attributes of the current block are normalized in the following way: Anode =∑ p∈node attribute(p); w node =∑ pεnode 1={p∈node}; a node =A node / w node .

[0064] First, the attributes of the current block can be obtained by the attributes of the points contained in the current block, namely: A node By simply adding the attributes of the points in the current block, and then normalizing the attributes of the current block with the number of points in the current block, the mean value a of the current block attribute is obtained. node The attribute transform coding is performed using the mean value of the current block attribute. The specific coding process is shown in Figure 12.

[0065] Figure 12 shows the overall process of RAHT attribute prediction transform coding. In Figure 12, (d) represents the current block attribute, and (e) represents the predicted block attribute obtained by linear weighted fitting using the neighborhood attributes of the current block. Ultimately, attribute transformations are performed on both to obtain DC and AC coefficients, and predictive coding is performed on the AC coefficients.

[0066] The predicted attributes of the current block are obtained by linear fitting as shown in Figure 13. As shown in Figure 13, the 19 neighboring blocks of the current block are first obtained. Then, the attributes of each sub-block are linearly weighted predicted using the spatial geometric distance between the neighboring blocks and each sub-block of the current block. Finally, the predicted block attributes obtained by linear weighting are transformed. The specific attribute transformation is shown in Figure 14, where (d) represents the original value of the attribute, and the corresponding attribute transformation coefficient is shown in the following formula (3):

[0067] In Figure 14, (e) represents the attribute prediction value, and the corresponding attribute transformation coefficient is shown in the following formula 4:

[0068] By performing a subtraction operation on the original value of the attribute and the predicted value of the attribute, the prediction residual can be obtained as shown in the following formula (5):

[0069] In another specific implementation, for region-adaptive hierarchical inter-frame prediction transform coding, the process for G-PCC attribute inter-frame prediction is similar to that of intra-frame prediction coding. First, a RAHT attribute transform coding structure is constructed based on geometric information. That is, transformations are continuously performed at the voxel level until the root node is obtained, thus completing the hierarchical transform coding of the entire attribute. In this way, the intra-frame coding structure and the inter-frame attribute coding structure are constructed, as shown in Figure 15.

[0070] As shown in FIG15 , the geometric information of the current node to be coded is used to obtain the co-located predicted node of the current node to be coded in the reference frame. Then, the geometric information and attribute information of the reference node are used to obtain the predicted attribute of the current node to be coded.

[0071] Secondly, the attribute prediction value of the current node to be encoded is obtained according to the following two different methods:

[0072] (1) The inter-frame prediction node of the current node to be coded is valid: that is, if the same-position node exists, the attribute of the inter-frame prediction node is directly used as the attribute prediction value of the current node to be coded;

[0073] (2) The inter-frame prediction node of the current node to be coded is invalid: that is, the co-located node does not exist, and the attribute prediction value of the adjacent node in the frame is used as the attribute prediction value of the current node to be coded;

[0074] Finally, the attribute prediction value is used to predict the attribute of the current node to be encoded, thereby completing the predictive coding of the entire attribute.

[0075] For adaptive region-adaptive hierarchical inter / intra prediction transform coding, for each layer of the RAHT transform, the encoding end will first calculate the codewords required for the current layer using only region-adaptive hierarchical intra prediction transform coding, as well as the codewords required for region-adaptive hierarchical inter prediction transform coding, select the mode with smaller rate distortion, and encode the flag.

[0076] At the decoding end, the decoding flag is used to obtain the coding mode (mode); if the selected coding mode is region-adaptive layered intra-frame prediction transform coding, the decoding end adopts region-adaptive layered intra-frame prediction transform coding; if the selected coding mode is region-adaptive layered inter-frame prediction transform coding, the decoding end adopts region-adaptive layered inter-frame prediction transform coding.

[0077] The aforementioned scheme, when performing region-adaptive hierarchical inter-frame predictive transform coding, only references the point cloud sequence frame immediately preceding the current frame. This limited reference range results in limited performance. Furthermore, the method for determining the reference transform block from a single frame still suffers from insufficient prediction accuracy when applied directly to multiple frames. There is a lack of a reasonable method for determining the reference transform block that is compatible with multiple frames.

[0078] Based on the above analysis, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0079] The present application provides a decoding method that can be applied to a point cloud decoder. FIG16 is a schematic diagram of an implementation flow of the decoding method provided in the present application. As shown in FIG16 , the method includes the following steps 1601 and 1605:

[0080] Step 1601: Decode the bitstream and determine the residual value of the transform coefficient of the current transform block;

[0081] Step 1602, determining multiple frames or multiple slices of reference point clouds of the current transform block;

[0082] Step 1603: determining a reference transform block of the current transform block from the multiple frames or multiple reference point clouds;

[0083] Step 1604: determining a predicted value of a transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block;

[0084] Step 1605: Determine the transform coefficient of the current transform block according to the residual value and the predicted value of the transform coefficient of the current transform block.

[0085] It can be understood that in the embodiment of the present application, the number of reference point clouds participating in the prediction of the transform coefficients of the current transform block is multiple frames or multiple slices, rather than a single frame or a single slice; this is beneficial to improving the prediction accuracy of the transform coefficients of the current transform block, thereby helping to improve the accuracy of the transform coefficients of the current transform block, and further helping to improve the quality of the point cloud attributes at the decoding end.

[0086] It can be understood that steps 1602 to 1604 describe the implementation process of transform coefficient prediction based on multi-frame or multi-slice reference point clouds (hereinafter referred to as the multi-frame / multi-slice reference algorithm). Whether to use the multi-frame / multi-slice reference algorithm can be without preconditions or with preconditions. The so-called no preconditions means that there is no need to indicate whether to turn on the multi-frame / multi-slice reference algorithm, and the point cloud decoder directly uses the algorithm. On the contrary, the so-called preconditions mean that the first syntax identification information needs to be obtained from the bitstream, and whether to use or turn on the multi-frame / multi-slice reference algorithm is determined based on the value of the first syntax identification information, or if the first syntax identification information is not obtained from the bitstream, the multi-frame / multi-slice reference algorithm is used or not by default.

[0087] Of course, for schemes with preconditions, the way of setting the first syntax identification information is not limited. The first syntax identification information can be set in the bitstream or not. For schemes where it is not set in the bitstream, the point cloud decoder can use or not use the multi-frame / multi-slice reference algorithm by default.

[0088] That is, in some embodiments, the method further includes: obtaining first syntax identification information, wherein the first syntax identification information is used to indicate whether the transformation coefficient prediction is based on multiple frames or multiple pieces of reference point clouds, that is, whether it is based on a multi-frame / multi-piece reference algorithm.

[0089] Exemplarily, in some embodiments, obtaining the first syntax identification information includes: decoding a code stream to determine the first syntax identification information.

[0090] In some embodiments, the code stream is decoded to determine a sequence parameter set, an attribute parameter set and / or attribute header information.

[0091] It can be seen that in the embodiment of the present application, there is no limitation on the manner in which the first syntax identification information is carried in the bitstream, and the first syntax identification information can be set in a sequence parameter set, an attribute parameter set, or an attribute header. It can be understood that a sequence parameter set refers to relevant parameters of a set of frame point clouds. Therefore, if the first syntax identification information is set in the sequence parameter set, then the first syntax identification information is valid for the prediction of the transform coefficients of each transform block of each slice (Slice) in the set of frames. That is, whether each transform block of each point cloud in the set of frames is based on the multi-frame / multi-slice reference algorithm is determined based on the first syntax identification information carried in the sequence parameter set.

[0092] The attribute parameter set refers to the relevant parameters of a certain frame point cloud. Therefore, the first syntax identification information is set in the attribute parameter set, and the first syntax identification information is valid for the prediction of the transformation coefficients of each transformation block of each slice (Slice) of this frame, that is, whether each transformation block of each point cloud in this frame is based on the multi-frame / multi-slice reference algorithm is determined based on the first syntax identification information carried by the attribute parameter set.

[0093] The attribute header information refers to the relevant parameters of a certain slice of point cloud. Therefore, if the first syntax identification information is set in the attribute header information, the first syntax identification information is valid for the prediction of the transformation coefficients of each transformation block of this slice. That is, whether each transformation block of each point cloud in the frame is based on a multi-frame / multi-slice reference algorithm is determined based on the first syntax identification information carried by the attribute header information.

[0094] For example, in some other embodiments, the method further includes: if the first syntax identification information is not present in the bitstream, by default, not performing transform coefficient prediction based on the multi-frame or multi-slice reference point cloud, or by default, performing transform coefficient prediction based on the multi-frame or multi-slice reference point cloud. In other words, if the first syntax identification information is not present in the bitstream, the point cloud decoder may, by default, not enable the multi-frame / multi-slice reference algorithm, or enable the multi-frame / multi-slice reference algorithm.

[0095] Of course, in another possible implementation, the code stream may carry an indication that the current transform block starts a K-frame / K-slice reference algorithm, where K can be any value. That is, in some embodiments, the method further includes: decoding the code stream and determining a first parameter; the first parameter is used to indicate the number of frames or slices of the reference point cloud of the current transform block (hereinafter referred to as K frames or K slices). Based on this, if the first parameter is greater than 1, for step 1602, the point cloud decoder can determine the K-frame or K-slice reference point cloud of the current transform block (that is, the number of reference point clouds indicated by the first parameter); if the first parameter is equal to 1, for step 1602, the point cloud decoder can search for more adjacent frames or adjacent slice point clouds of the current frame or current slice as reference point clouds.

[0096] In the embodiment of the present application, there is no restriction on the location where the first parameter is carried, and the first parameter can be set in a sequence parameter set, an attribute parameter set, or an attribute header information. It can be understood that if the first parameter is set in a sequence parameter set, the first parameter is valid for a group of frame point clouds corresponding to the sequence parameter set, and the number of reference point clouds of each transformation block of the group of frame point clouds is the number indicated by the first parameter. Similarly, if the first parameter is set in an attribute parameter set, the first parameter is valid for a frame point cloud corresponding to the attribute parameter set, and the number of reference point clouds of each transformation block of the frame point cloud is the number indicated by the first parameter. If the first parameter is set in the attribute header information, the first parameter is valid for a piece of point cloud corresponding to the attribute header information, and the number of reference point clouds of each transformation block of the piece of point cloud is the number indicated by the first parameter.

[0097] Of course, in another possible implementation, the bitstream may also indicate a frame index or slice index of the reference point cloud. That is, in some embodiments, the method further includes: decoding the bitstream and determining a second parameter; the second parameter is used to indicate the frame index or slice index of the reference point cloud of the current transform block.

[0098] It can be understood that when there are multiple frames or multiple reference point clouds, the second parameters obtained by decoding the code stream are the frame indices or slice indices corresponding to the multiple frames or multiple reference point clouds, that is, the code stream carries one or more second parameters.

[0099] For step 1602, the point cloud decoder can determine multiple frames or multiple pieces of reference point clouds of the current transform block based on one or more second parameters, and the point cloud specified by the one or more second parameters is the reference point cloud of the current transform block.

[0100] In the embodiment of the present application, there is no restriction on the position of the second parameter, and the second parameter can be set in a sequence parameter set, an attribute parameter set, or an attribute header information. It can be understood that if the second parameter is set in a sequence parameter set, the second parameter is valid for a group of frame point clouds corresponding to the sequence parameter set, and the reference point cloud of each transformation block of the group of frame point clouds is the point cloud specified by the second parameter. Similarly, if the second parameter is set in an attribute parameter set, the second parameter is valid for a frame point cloud corresponding to the attribute parameter set, and the reference point cloud of each transformation block of the frame point cloud is the point cloud specified by the second parameter. If the second parameter is set in the attribute header information, the second parameter is valid for a piece of point cloud corresponding to the attribute header information, and the reference point cloud of each transformation block of the piece of point cloud is the point cloud specified by the second parameter.

[0101] It should also be noted that the first syntax identification information, the first parameter and / or the second parameter can be used in the decoding method. That is, the implementation of the decoding method can include any one or more combinations of the above-mentioned embodiments of the first syntax identification information, the first parameter and the second parameter.

[0102] In combination scheme 1, the information obtained by the point cloud decoder includes the first syntax identification information, but does not include the first parameter and the second parameter. By default, the point cloud decoder uses the point clouds of the N nearest neighboring frames or N nearest neighboring slices of the current frame or current slice as the reference point clouds for the current transform block; where N is greater than 1.

[0103] In combination scheme 2, the information obtained by the point cloud decoder includes the first syntax identification information and the first parameter, but does not include the second parameter. The first syntax identification information and the first parameter can be in the same parameter set, for example, both in the sequence parameter set, both in the attribute parameter set, or both in the attribute header information.

[0104] In combination scheme 3, the information obtained by the point cloud decoder includes the first syntax identification information, the first parameter and the second parameter. This information can be in the same parameter set, such as in the sequence parameter set, or both in the attribute parameter set, or both in the attribute header information.

[0105] In combination scheme 4, the information obtained by the point cloud decoder includes the first parameter, but does not include the first syntax identification information and the second parameter. That is, the point cloud decoder does not need to determine whether to turn on the multi-frame / multi-slice reference algorithm based on the first syntax identification information, but directly determines the reference point cloud of the current transformation block based on the number of reference point clouds indicated by the first parameter.

[0106] In combination scheme 5, the information obtained by the point cloud decoder includes the second parameter, but does not include the first syntax identification information and the first parameter. That is, the point cloud decoder does not need to determine whether to turn on the multi-frame / multi-slice reference algorithm based on the first syntax identification information, but directly obtains the reference point cloud of the current transformation block based on the reference point cloud specified by the second parameter.

[0107] In combination scheme 6, the information obtained by the point cloud decoder includes the first parameter and the second parameter, but does not include the first syntax identification information.

[0108] In combination scheme 7, the information obtained by the point cloud decoder includes the first syntax identification information and the second parameter, but does not include the first parameter.

[0109] Optionally, in some embodiments, the method further includes: determining whether the current RAHT layer uses RAHT inter-frame prediction transform coding according to second syntax identification information.

[0110] In some embodiments, the second syntax identification information is used to indicate a coding mode used by the current RAHT layer, for example, indicating whether the current RAHT layer uses RAHT inter-frame prediction transform coding or RAHT intra-frame prediction transform coding. Alternatively, for another example, the second syntax identification information is used to indicate whether the current RAHT layer uses RAHT inter-frame prediction transform coding, RAHT intra-frame prediction transform coding, or RAHT direct transform coding.

[0111] It can be understood that if the second syntax identification information indicates that the current RAHT layer uses RAHT inter-frame prediction transform coding, the point cloud decoder predicts the transform coefficients of the current transform block based on the reference point cloud.

[0112] In some embodiments, the method further includes: decoding the code stream to determine the second syntax identification information.

[0113] The following describes further optional implementations, related terms, and additional implementations of each of the above steps.

[0114] In step 1601, a code stream is decoded to determine the residual value of the transform coefficient of the current transform block.

[0115] It can be understood that in the embodiment of the present application, the transform coefficients of the current transform block can be high-frequency coefficients or low-frequency coefficients.

[0116] In step 1602, multiple frames or multiple slices of reference point clouds of the current transform block are determined.

[0117] It can be understood that steps 1602 to 1604 can be understood as a method for predicting transform coefficients based on multiple frames or multiple slices of reference point clouds, which can also be understood as a multi-frame / multi-slice reference algorithm. The point cloud decoder can determine the multi-frame or multi-slice reference point cloud for the current transform block based on any of the above combination schemes 1-7.

[0118] In step 1603, a reference transform block of the current transform block is determined from the multiple frames or multiple reference point clouds.

[0119] In the embodiments of the present application, the method for finding the reference transform block of the current transform block from each frame or each reference point cloud is the same. Therefore, the following describes how to find the reference transform block of the current transform block using a frame or a reference point cloud as an example. That is, in some embodiments, the reference transform block of the current transform block includes at least one of the following (1) to (6):

[0120] (1) a transform block whose geometric position in the reference point cloud is the same as that of the current transform block;

[0121] (2) a transform block whose geometric position of the parent transform block in the reference point cloud is the same as the geometric position of the parent transform block of the current transform block;

[0122] (3) A transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and whose placeholder information in the reference point cloud differs from that of the current transform block by a number of bits less than or equal to a first threshold. That is, a transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and whose placeholder information differs from that of the current transform block by a number of bits less than or equal to the first threshold.

[0123] (4) A transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and whose number of bits of placeholder information of the parent transform block of the reference point cloud differs from that of the parent transform block of the current transform block is less than or equal to a second threshold. That is, a transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and whose number of bits of placeholder information of the parent transform block of the transform block differs from that of the parent transform block of the current transform block is less than or equal to the second threshold.

[0124] (5) A transform block whose geometric position in the reference point cloud is the same as that of the parent transform block of the current transform block, and whose placeholder information in the reference point cloud differs from that of the current transform block by a number of bits less than or equal to a third threshold. That is, a transform block whose geometric position in the reference point cloud is the same as that of the parent transform block of the current transform block, and whose placeholder information differs from that of the current transform block by a number of bits less than or equal to a third threshold.

[0125] (6) A transform block whose geometric position in the reference point cloud is the same as that of the parent transform block of the current transform block, and whose placeholder information of the parent transform block in the reference point cloud differs from that of the parent transform block of the current transform block by a number of bits less than or equal to a fourth threshold. That is, a transform block whose geometric position in the reference point cloud is the same as that of the parent transform block of the current transform block, and whose placeholder information of the parent transform block differs from that of the parent transform block of the current transform block by a number of bits less than or equal to a fourth threshold.

[0126] It can be understood that the number of different bits in the placeholder information refers to the number of different bit values ​​corresponding to the binary number representing the placeholder information, that is, the number of occupied nodes. Taking the placeholder information as an 8-bit binary number as an example, assuming that the placeholder information for transform block 1 is 00100100 and the placeholder information for transform block 2 is 00100101, the number of different bits in the placeholder information of transform block 1 and transform block 2 is 1.

[0127] In the embodiment of the present application, the first threshold, the second threshold, the third threshold, and the fourth threshold may be the same or different. For an embodiment in which the placeholder information is 8 bits, the first threshold, the second threshold, the third threshold, or the fourth threshold may be any value between 0 and 7.

[0128] Exemplarily, in some embodiments, at least one of the first to fourth thresholds is equal to 0. For example, if the first to fourth thresholds are all equal to 0, this means that the placeholder information of the two transform blocks is the same, and the corresponding value of each bit is equal, that is, the occupied nodes in the two transform blocks are the same.

[0129] In other embodiments, the first threshold to the fourth threshold are all equal to 1, which means that only the value of one bit of the occupancy information of the two transformation blocks is different, and the values ​​of the remaining bits are all correspondingly equal, that is, there is one difference in the occupied nodes in the two transformation blocks.

[0130] In step 1604, a prediction value of the transform coefficient of the current transform block is determined based on the transform coefficient of the reference transform block of the current transform block.

[0131] In some embodiments, step 1604 may be implemented as follows: performing an averaging operation on the transform coefficients of the reference transform block of the current transform block to obtain a predicted value of the transform coefficient of the current transform block.

[0132] In the embodiment of the present application, the averaging operation may be a weighted averaging operation or an arithmetic averaging operation. When the number of reference transform blocks for the current transform block is one, the predicted value of the transform coefficient of the current transform block is equal to the transform coefficient of the reference transform block. When the number of reference transform blocks for the current transform block is multiple, the predicted value of the transform coefficient of the current transform block is equal to the average operation result (such as the arithmetic mean or weighted mean) of the transform coefficients of the multiple reference transform blocks.

[0133] In step 1605, the transform coefficients of the current transform block are determined according to the residual values ​​and prediction values ​​of the transform coefficients of the current transform block.

[0134] It can be understood that the residual value of the transform coefficient of the current transform block obtained by decoding the code stream is the quantized residual value. Therefore, the point cloud decoder needs to inverse quantize the residual value, and then obtain the transform coefficient of the current transform block based on the inverse quantized residual value and the predicted value, that is, the transform coefficient of the current transform block is equal to the sum of the inverse quantized residual value and the predicted value.

[0135] In some embodiments, the method further includes: performing a RAHT transform based on all the transform coefficients to obtain all the attribute information of the current point cloud. It can be understood that the RAHT transform here is the inverse transform process of the encoding end, that is, the inverse RAHT transform. For details, see FIG10 and the lower half of the rectangular box shown in FIG8, that is, from f 0,0 , g 0,0 Transform to g′ 3,5 to g′ 3,0 process and understand it.

[0136] The present application provides an encoding method, which corresponds to the processing flow at the decoding end and can be applied to a point cloud encoder. FIG17 is a schematic diagram of the implementation flow of the encoding method provided in the present application embodiment. As shown in FIG17 , the method includes the following steps 1701 and 1705:

[0137] Step 1701, determining multiple frames or multiple slices of reference point clouds of the current transform block;

[0138] Step 1702: Determine a reference transform block of the current transform block from the multiple frames or multiple reference point clouds;

[0139] Step 1703: determining a predicted value of a transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block;

[0140] Step 1704: determining a residual value of the transform coefficient of the current transform block according to the transform coefficient of the current transform block and the predicted value of the transform coefficient of the current transform block;

[0141] Step 1705: Generate a code stream according to the residual value of the transformation coefficient of the current transformation block.

[0142] It can be understood that in the encoding method provided in the embodiment of the present application, the reference point cloud participating in the prediction of the transformation coefficient of the current transformation block is a reference point cloud of multiple frames or multiple slices, rather than a reference point cloud of a single frame or a single slice; this is beneficial to improving the prediction accuracy of the transformation coefficient of the current transformation block, thereby helping to reduce the residual value of the transformation coefficient of the current transformation block, and further helping to reduce the codeword overhead at the point cloud encoding end.

[0143] In some embodiments, the method further includes: generating a code stream according to first syntax identification information; wherein the first syntax identification information is used to indicate whether to perform transform coefficient prediction based on multiple frames or multiple reference point clouds.

[0144] In some embodiments, the first syntax identification information is set in a sequence parameter set, an attribute parameter set or an attribute header information.

[0145] In some embodiments, the method further includes: generating a code stream according to a first parameter; the first parameter is used to indicate the number of frames or slices of the reference point cloud of the current transform block.

[0146] In some embodiments, the method further includes: generating a code stream according to a second parameter; the second parameter is used to indicate a frame index or a slice index of the reference point cloud of the current transform block.

[0147] In some embodiments, the method further includes: generating a code stream according to second syntax identification information; the second syntax identification information is used to determine whether the current RAHT layer uses RAHT inter-frame prediction transform coding.

[0148] In some embodiments, the reference transform block of the current transform block comprises: a transform block having the same geometric position in the reference point cloud as that of the current transform block.

[0149] In some embodiments, the reference transform block of the current transform block comprises: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as that of the parent transform block of the current transform block.

[0150] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position in the reference point cloud is the same as that of the current transform block and whose number of bits of placeholder information in the reference point cloud that is different from that in the placeholder information of the current transform block is less than or equal to a first threshold.

[0151] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and a transform block whose placeholder information of a parent transform block of the reference point cloud differs from that of the parent transform block of the current transform block by a number of bits less than or equal to a second threshold.

[0152] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as the geometric position of the parent transform block of the current transform block, and a transform block whose placeholder information in the reference point cloud is different from that in the placeholder information of the current transform block by a number of bits less than or equal to a third threshold.

[0153] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as that of the parent transform block of the current transform block, and a transform block whose placeholder information of the parent transform block in the reference point cloud is different from that of the parent transform block of the current transform block by a number of bits less than or equal to a fourth threshold.

[0154] In some embodiments, at least one of the first to fourth thresholds is equal to 0.

[0155] In some embodiments, determining the predicted value of the transform coefficient of the current transform block based on the transform coefficient of the reference transform block of the current transform block includes: performing an averaging operation on the transform coefficients of the reference transform block of the current transform block to obtain the predicted value of the transform coefficient of the current transform block.

[0156] It should also be noted that the encoding method associated with FIG. 17 corresponds to the decoding method associated with FIG. 16 . Therefore, for technical details not disclosed in the encoding method, please refer to the corresponding descriptions and explanations of the decoding method. To save space and avoid repetition, some technical details of the encoding method will not be further described.

[0157] Exemplary applications of the embodiments of the present application will be described below, such as Embodiments 1 to 6 described below.

[0158] Example 1

[0159] The decoding end, as shown in FIG18 , includes the following steps 1801 to 1808:

[0160] Step 1801, decoding flag bit;

[0161] Step 1802: Determine whether the flag is true; if so, enable the multi-frame reference algorithm and execute step 1803; if not, do not enable the multi-frame reference algorithm and execute step 1806;

[0162] Step 1803: Determine whether the coding mode for each layer is inter-frame; if yes, execute step 1804; otherwise, execute step 1805;

[0163] Step 1804, using the K-frame reference point cloud for prediction;

[0164] Step 1805, using intra-frame prediction;

[0165] Step 1806: Determine whether the coding mode for each layer is inter; if yes, execute step 1807; otherwise, execute step 1808;

[0166] Step 1807, using the previous frame reference point cloud for prediction;

[0167] Step 1808: adopt intra-frame prediction.

[0168] Decode the sequence parameter set or attribute parameter set or attribute header information to obtain the flag bit described in step 1801;

[0169] This flag indicates whether the multi-frame reference algorithm is enabled for the current slice. If this flag does not appear in the bitstream, its default value is a fixed value; the default value can indicate that the multi-frame reference algorithm is enabled or disabled.

[0170] If this flag is true or defaults to true, the multi-frame reference algorithm is enabled, as shown in Figure 19, the reference frame of the current frame is a fixed value K frame;

[0171] The residual values ​​of the decoded transform coefficients.

[0172] For each layer of the RAHT transform:

[0173] On the decoding side, the decoding flag is used to obtain the mode, which is the mode selected for decoding the current RAHT layer:

[0174] If the mode selected by the current RAHT layer is regional adaptive hierarchical intra-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical intra-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical intra-frame prediction transform decoding; if the mode selected by the current RAHT layer is regional adaptive hierarchical inter-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical inter-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical inter-frame prediction transform decoding.

[0175] Decoding the residual values ​​of the transform coefficients;

[0176] For each block in the current RAHT layer, find its inter-frame prediction transform block, specifically including at least one of the following 1-6:

[0177] 1. Find a transform block in the reference frame / slice whose geometric position is the same as the geometric position of the current transform block;

[0178] 2. Find a transform block in the reference frame / slice whose parent transform block has the same geometric position as the parent transform block of the current transform block;

[0179] 3. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the current transform block and whose placeholder information in the reference frame / slice differs from the placeholder information in the current transform block by a number less than or equal to J (where J is any value from 0 to 7);

[0180] 4. Find a transform block whose geometric position in the reference frame / slice is the same as that of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0181] 5. Find a transform block whose parent transform block in the reference frame / slice has the same geometric position as the parent transform block of the current transform block and whose placeholder information in the reference frame / slice differs from that in the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0182] 6. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the parent transform block of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7).

[0183] If no inter-frame prediction transform block is found, the transform coefficients of the intra-frame adjacent transform blocks (i.e., the adjacent transform blocks of the current transform block in the same frame) are used as the prediction values ​​of the transform coefficients of the current transform block (i.e., using region-adaptive hierarchical intra-frame prediction transform coding);

[0184] If inter-frame prediction transform blocks can be found, the number of the found blocks is m, and the transform coefficients of the m inter-frame prediction transform blocks are subjected to weighted prediction to obtain the predicted value of the transform coefficient of the current transform block.

[0185] The transform coefficient of the current transform block=the residual value of the transform coefficient of the current transform block+the predicted value of the transform coefficient of the current transform block.

[0186] Example 2

[0187] Example 2 is a processing flow of the encoding end corresponding to Example 1:

[0188] Encode the flag bit into the sequence parameter set or attribute parameter set or attribute header information;

[0189] This flag indicates whether the multi-frame reference algorithm is enabled for the current slice. If this flag does not appear in the bitstream, its default value is a fixed value; the default value can indicate that the multi-frame reference algorithm is enabled or disabled.

[0190] If this flag is true or defaults to true, the multi-frame reference algorithm is enabled, and the reference frame of the current frame is the fixed value K frame;

[0191] For each block in the current RAHT layer, find its inter-frame prediction transform block, specifically including at least one of the following 1-6:

[0192] 1. Find a transform block in the reference frame / slice whose geometric position is the same as the geometric position of the current transform block;

[0193] 2. Find a transform block in the reference frame / slice whose parent transform block has the same geometric position as the parent transform block of the current transform block;

[0194] 3. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the current transform block and whose placeholder information in the reference frame / slice differs from the placeholder information in the current transform block by a number less than or equal to J (where J is any value from 0 to 7);

[0195] 4. Find a transform block whose geometric position in the reference frame / slice is the same as that of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0196] 5. Find a transform block whose parent transform block in the reference frame / slice has the same geometric position as the parent transform block of the current transform block and whose placeholder information in the reference frame / slice differs from that in the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0197] 6. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the parent transform block of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7).

[0198] If no inter-frame prediction transform block is found, the transform coefficients of the intra-frame adjacent transform blocks (i.e., the adjacent transform blocks of the current transform block in the same frame) are used as the prediction values ​​of the transform coefficients of the current transform block (i.e., using region-adaptive hierarchical intra-frame prediction transform coding);

[0199] If inter-frame prediction transform blocks can be found, the number of the found blocks is m, and the transform coefficients of the m inter-frame prediction transform blocks are subjected to weighted prediction to obtain the predicted value of the transform coefficient of the current transform block.

[0200] A residual value of a transform coefficient of a current transform block is calculated, where the residual value of the transform coefficient of the current transform block is equal to the transform coefficient of the current transform block minus the predicted value of the transform coefficient of the current transform block.

[0201] Encodes the residual values ​​of the transform coefficients of the current transform block.

[0202] Example 3

[0203] The tool proposed in this solution can use a 1-bit flag to indicate whether the multi-frame reference algorithm is enabled. This flag is placed in the header information of high-level syntax elements, such as the attribute header. This flag is conditionally enabled under certain conditions; if this flag does not appear in the bitstream, its default value is a fixed value.

[0204] The decoder needs to decode this flag bit. If this flag bit does not appear in the code stream, it will not be decoded. The default value is a fixed value.

[0205] This flag indicates whether to enable the following multi-frame reference algorithm:

[0206] Encoding side:

[0207] Encoding flag;

[0208] This flag represents the number K of predicted neighbors referenced by the current slice, that is, the number of reference point cloud slices;

[0209] Using a rate-distortion optimization method, the method is subjected to rate-distortion optimization;

[0210] According to the flag bit, select the corresponding number K reference slices (i.e., K reference point clouds) in the reference list;

[0211] For each block in the current RAHT layer, find its inter-frame prediction transform block, specifically including at least one of the following 1-6:

[0212] 1. Find a transform block in the reference frame / slice whose geometric position is the same as the geometric position of the current transform block;

[0213] 2. Find a transform block in the reference frame / slice whose parent transform block has the same geometric position as the parent transform block of the current transform block;

[0214] 3. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the current transform block and whose placeholder information in the reference frame / slice differs from the placeholder information in the current transform block by a number less than or equal to J (where J is any value from 0 to 7);

[0215] 4. Find a transform block whose geometric position in the reference frame / slice is the same as that of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0216] 5. Find a transform block whose parent transform block in the reference frame / slice has the same geometric position as the parent transform block of the current transform block and whose placeholder information in the reference frame / slice differs from that in the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0217] 6. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the parent transform block of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7).

[0218] If no inter-frame prediction transform block is found, the transform coefficients of the intra-frame adjacent transform blocks (i.e., the adjacent transform blocks of the current transform block in the same frame) are used as the prediction values ​​of the transform coefficients of the current transform block (i.e., using region-adaptive hierarchical intra-frame prediction transform coding);

[0219] If inter-frame prediction transform blocks can be found, the number of the found blocks is m, and the transform coefficients of the m inter-frame prediction transform blocks are subjected to weighted prediction to obtain the predicted value of the transform coefficient of the current transform block.

[0220] Calculating a residual value of a transform coefficient of a current transform block, where the residual value of the transform coefficient of the current transform block is equal to the transform coefficient of the current transform block minus the predicted value of the transform coefficient of the current transform block;

[0221] Encodes the residual values ​​of the transform coefficients of the current transform block.

[0222] Example 4

[0223] Example 4 is a processing flow of the decoding end corresponding to Example 3:

[0224] Decoding flag;

[0225] This flag represents the number K of predicted neighbors referenced by the current slice;

[0226] According to the flag bit, select the corresponding number K reference slices in the reference list;

[0227] The residual values ​​of the decoded transform coefficients.

[0228] For each layer of the RAHT transform:

[0229] At the decoding end, the decoding flag is used to obtain the mode (mode), which is the mode selected for decoding of the current RAHT layer;

[0230] If the mode selected by the current RAHT layer is regional adaptive hierarchical intra-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical intra-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical intra-frame prediction transform decoding; if the mode selected by the current RAHT layer is regional adaptive hierarchical inter-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical inter-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical inter-frame prediction transform decoding.

[0231] Decoding the residual values ​​of the transform coefficients;

[0232] For each block in the current RAHT layer, find its inter-frame prediction transform block, specifically including at least one of the following 1-6:

[0233] 1. Find a transform block in the reference frame / slice whose geometric position is the same as the geometric position of the current transform block;

[0234] 2. Find a transform block in the reference frame / slice whose parent transform block has the same geometric position as the parent transform block of the current transform block;

[0235] 3. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the current transform block and whose placeholder information in the reference frame / slice differs from the placeholder information in the current transform block by a number less than or equal to J (where J is any value from 0 to 7);

[0236] 4. Find a transform block whose geometric position in the reference frame / slice is the same as that of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0237] 5. Find a transform block whose parent transform block in the reference frame / slice has the same geometric position as the parent transform block of the current transform block and whose placeholder information in the reference frame / slice differs from that in the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7);

[0238] 6. Find a transform block whose geometric position in the reference frame / slice is the same as the geometric position of the parent transform block of the current transform block and whose placeholder information of the parent transform block in the reference frame / slice differs from that of the parent transform block of the current transform block by a number of bits less than or equal to J (where J is any value from 0 to 7).

[0239] If no inter-frame prediction transform block is found, the transform coefficients of the intra-frame adjacent transform blocks (i.e., the adjacent transform blocks of the current transform block in the same frame) are used as the prediction values ​​of the transform coefficients of the current transform block (i.e., using region-adaptive hierarchical intra-frame prediction transform coding);

[0240] If inter-frame prediction transform blocks can be found, the number of the found blocks is m, and the transform coefficients of the m inter-frame prediction transform blocks are subjected to weighted prediction to obtain the predicted value of the transform coefficient of the current transform block.

[0241] The transform coefficient of the current transform block=the residual value of the transform coefficient of the current transform block+the predicted value of the transform coefficient of the current transform block.

[0242] Example 5

[0243] Decoding end:

[0244] Decoding attribute parameter set multi-frame reference algorithm flag (aps);

[0245] If this flag is true, the multi-frame reference algorithm is enabled for the current frame, and the reference frame of the current frame is a fixed value of 2 frames;

[0246] The residual values ​​of the decoded transform coefficients.

[0247] For each layer of the RAHT transform:

[0248] For each layer of the RAHT transform:

[0249] On the decoding side, the decoding flag is used to obtain the mode, which is the mode selected for decoding the current RAHT layer:

[0250] If the mode selected by the current RAHT layer is regional adaptive hierarchical intra-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical intra-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical intra-frame prediction transform decoding; if the mode selected by the current RAHT layer is regional adaptive hierarchical inter-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical inter-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical inter-frame prediction transform decoding.

[0251] The residual values ​​of the decoded transform coefficients.

[0252] For each block in the current layer, we search for the inter-frame prediction transform block. Specifically:

[0253] Find a transform block in the reference frame / slice that has the same geometric position as the current transform block; or,

[0254] Find a transform block in a reference frame / slice whose parent transform block has the same geometric position as the parent transform block of the current transform block;

[0255] If no inter-frame prediction transform block is found, the transform coefficients of the adjacent transform blocks within the frame (i.e., the adjacent transform blocks of the current transform block in the same frame) are used as the prediction values ​​of the transform coefficients of the current transform block (i.e., using region-adaptive hierarchical intra-frame prediction transform coding);

[0256] If m inter-frame prediction transform blocks can be found, the transform coefficients of the m inter-frame prediction transform blocks are weighted predicted to obtain the predicted transform coefficients of the current transform block. The predicted transform value for reference frame 1 is ref1, and the predicted transform value for reference frame 2 is ref2.

[0257] If m=1, the predicted value of the transform coefficient of the current transform block is ref1 or ref2;

[0258] If m=2, the predicted value of the transform coefficient of the current transform block is (ref1+ref2+1)>>1 or (ref1+ref2) / 2; or,

[0259] If an inter-frame prediction transform block can be found, the number of which is m, and the m prediction values ​​are weighted prediction to obtain the predicted values ​​of the transform coefficients of the current transform block. The transform coefficients of the inter-frame prediction transform block found based on reference frame 1 are ref1, and the transform coefficients of the inter-frame prediction transform block found based on reference frame 2 are ref2.

[0260] The transform coefficient of the current transform block=the residual value of the transform coefficient of the current transform block+the predicted value of the transform coefficient of the current transform block.

[0261] Example 6

[0262] Decoding coefficients;

[0263] This coefficient represents the number of predicted neighbors referenced by the current frame, which is K; for example, K = 2;

[0264] The residual values ​​of the decoded transform coefficients.

[0265] For each layer of the RAHT transform:

[0266] At the decoding end, the decoding flag is used to obtain the mode, which is the mode selected for decoding of the current RAHT layer.

[0267] If the mode selected by the current RAHT layer is regional adaptive hierarchical intra-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical intra-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical intra-frame prediction transform decoding; if the mode selected by the current RAHT layer is regional adaptive hierarchical inter-frame prediction transform decoding, the decoding end adopts regional adaptive hierarchical inter-frame prediction transform decoding, that is, all transform blocks in the current RAHT transform layer adopt regional adaptive hierarchical inter-frame prediction transform decoding.

[0268] Decoding the residual values ​​of the transform coefficients;

[0269] For each block in the current RAHT layer, we search for the inter-frame prediction transform block. Specifically:

[0270] Find a transform block in the reference frame / slice that has the same geometric position as the current transform block; or,

[0271] Find a transform block in a reference frame / slice whose parent transform block has the same geometric position as the parent transform block of the current transform block;

[0272] If no inter-frame prediction transform block is found, the transform coefficients of the intra-frame adjacent transform blocks (i.e., the adjacent transform blocks of the current transform block in the same frame) are used as the prediction values ​​of the transform coefficients of the current transform block (i.e., using region-adaptive hierarchical intra-frame prediction transform coding);

[0273] If inter-frame prediction transform blocks can be found, the number found is m. The transform coefficients of the m inter-frame prediction transform blocks are weighted predicted to obtain the predicted values ​​of the transform coefficients of the current transform block. The transform coefficients of the inter-frame prediction transform block found based on reference frame 1 are ref1, and the transform coefficients of the inter-frame prediction transform block found based on reference frame 2 are ref2.

[0274] If m=1, the predicted value of the transform coefficient of the current transform block is ref1 or ref2;

[0275] If m=2, the predicted value of the transform coefficient of the current transform block is (ref1+ref2+1)>>1 or (ref1+ref2) / 2;

[0276] or,

[0277] If inter-frame prediction transform blocks can be found, the number of them is m, and the attribute transform coefficient prediction value is obtained by weighted prediction method for the m prediction values. The predicted transform value of reference frame 1 is ref1, and the predicted transform value of reference frame 2 is ref2.

[0278] The transform coefficient of the current transform block=the residual value of the transform coefficient of the current transform block+the predicted value of the transform coefficient of the current transform block.

[0279] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps; or steps in different embodiments may be combined to form a new technical solution.

[0280] Based on the above embodiments, the present application provides a decoding device for use in a point cloud decoder. FIG20 is a schematic structural diagram of the decoding device provided in the present application. As shown in FIG20 , the decoding device 200 includes:

[0281] The decoding module 2001 is configured to decode the code stream and determine the residual value of the transform coefficient of the current transform block;

[0282] A first determining module 2002 is configured to determine a multi-frame or multi-slice reference point cloud of the current transform block;

[0283] A second determining module 2003 is configured to determine a reference transform block of the current transform block from the multiple frames or multiple reference point clouds;

[0284] a third determining module 2004 configured to determine a predicted value of a transform coefficient of the current transform block according to transform coefficients of a reference transform block of the current transform block;

[0285] The fourth determination module 2005 is configured to determine the transform coefficient of the current transform block according to the residual value and the prediction value of the transform coefficient of the current transform block.

[0286] In some embodiments, the decoding device 200 further includes an acquisition module configured to acquire first syntax identification information, where the first syntax identification information is used to indicate whether transform coefficient prediction is performed based on multiple frames or multiple reference point clouds.

[0287] Furthermore, in some embodiments, obtaining the first syntax identification information includes: decoding the bitstream to determine the first syntax identification information. Alternatively, in other embodiments, when the first syntax identification information is not present in the bitstream, the decoding apparatus 200 defaults to not performing transform coefficient prediction based on multiple frames or multiple slices of reference point clouds, or defaults to performing transform coefficient prediction based on multiple frames or multiple slices of reference point clouds.

[0288] In some embodiments, the decoding of the code stream to determine the first syntax identification information includes: decoding the code stream to determine a sequence parameter set, an attribute parameter set and / or attribute header information; wherein the first syntax identification information is set in the sequence parameter set, the attribute parameter set or the attribute header information.

[0289] In some embodiments, the decoding module 2001 is further configured to decode the code stream and determine a first parameter; the first parameter is used to indicate the number of frames or slices of the reference point cloud of the current transform block.

[0290] In some embodiments, the decoding module 2001 is further configured to decode the code stream and determine a second parameter; the second parameter is used to indicate a frame index or a slice index of the reference point cloud of the current transform block.

[0291] In some embodiments, the first determining module 2002 is further configured to determine whether the current RAHT layer uses RAHT inter-frame prediction transform coding according to the second syntax identification information.

[0292] In some embodiments, the decoding module 2001 is further configured to decode the code stream and determine the second syntax identification information.

[0293] In some embodiments, the reference transform block of the current transform block comprises: a transform block having the same geometric position in the reference point cloud as that of the current transform block.

[0294] In some embodiments, the reference transform block of the current transform block comprises: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as that of the parent transform block of the current transform block.

[0295] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position in the reference point cloud is the same as that of the current transform block and whose number of bits of placeholder information in the reference point cloud that is different from that in the placeholder information of the current transform block is less than or equal to a first threshold.

[0296] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and a transform block whose placeholder information of a parent transform block of the reference point cloud differs from that of the parent transform block of the current transform block by a number of bits less than or equal to a second threshold.

[0297] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as the geometric position of the parent transform block of the current transform block, and a transform block whose placeholder information in the reference point cloud is different from that in the placeholder information of the current transform block by a number of bits less than or equal to a third threshold.

[0298] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as that of the parent transform block of the current transform block, and a transform block whose placeholder information of the parent transform block in the reference point cloud is different from that of the parent transform block of the current transform block by a number of bits less than or equal to a fourth threshold.

[0299] In some embodiments, at least one of the first to fourth thresholds is equal to 0.

[0300] In some embodiments, determining the predicted value of the transform coefficient of the current transform block based on the transform coefficient of the reference transform block of the current transform block includes: performing an averaging operation on the transform coefficients of the reference transform block of the current transform block to obtain the predicted value of the transform coefficient of the current transform block.

[0301] In some embodiments, the decoding device 200 may perform RAHT transformation according to all transformation coefficients to obtain all attribute information of the current point cloud.

[0302] The present application provides an encoding device for use in a point cloud encoder. FIG21 is a schematic diagram of the structure of the encoding device provided in the present application. As shown in FIG21 , the encoding device 210 includes:

[0303] A fifth determining module 2101 is configured to determine a multi-frame or multi-slice reference point cloud of a current transform block;

[0304] a sixth determining module 2102 configured to determine a reference transform block of the current transform block from the multiple frames or multiple reference point clouds;

[0305] a seventh determination module 2103 configured to determine a predicted value of a transform coefficient of the current transform block based on a transform coefficient of a reference transform block of the current transform block;

[0306] an eighth determining module 2104 configured to determine a residual value of the transform coefficient of the current transform block based on the transform coefficient of the current transform block and the predicted value of the transform coefficient of the current transform block;

[0307] The encoding module 2105 is configured to generate a code stream according to the residual value of the transformation coefficient of the current transformation block.

[0308] In some embodiments, the encoding module 2105 is further configured to generate a bitstream according to first syntax identification information; wherein the first syntax identification information is used to indicate whether to perform transform coefficient prediction based on multiple frames or multiple reference point clouds.

[0309] In some embodiments, the first syntax identification information is set in a sequence parameter set, an attribute parameter set or an attribute header information.

[0310] In some embodiments, the encoding module 2105 is further configured to generate a code stream according to a first parameter; the first parameter is used to indicate the number of frames or slices of the reference point cloud of the current transform block.

[0311] In some embodiments, the encoding module 2105 is further configured to generate a code stream according to a second parameter; the second parameter is used to indicate a frame index or a slice index of the reference point cloud of the current transform block.

[0312] In some embodiments, the encoding module 2105 is further configured to generate a bitstream according to second syntax identification information; the second syntax identification information is used to determine whether the current RAHT layer uses RAHT inter-frame prediction transform coding.

[0313] In some embodiments, the reference transform block of the current transform block comprises: a transform block having the same geometric position in the reference point cloud as that of the current transform block.

[0314] In some embodiments, the reference transform block of the current transform block comprises: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as that of the parent transform block of the current transform block.

[0315] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position in the reference point cloud is the same as that of the current transform block and whose number of bits of placeholder information in the reference point cloud that is different from that in the placeholder information of the current transform block is less than or equal to a first threshold.

[0316] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position in the reference point cloud is the same as that of the current transform block, and a transform block whose placeholder information of a parent transform block of the reference point cloud differs from that of the parent transform block of the current transform block by a number of bits less than or equal to a second threshold.

[0317] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as the geometric position of the parent transform block of the current transform block, and a transform block whose placeholder information in the reference point cloud is different from that in the placeholder information of the current transform block by a number of bits less than or equal to a third threshold.

[0318] In some embodiments, the reference transform block of the current transform block includes: a transform block whose geometric position of a parent transform block in the reference point cloud is the same as that of the parent transform block of the current transform block, and a transform block whose placeholder information of the parent transform block in the reference point cloud is different from that of the parent transform block of the current transform block by a number of bits less than or equal to a fourth threshold.

[0319] In some embodiments, at least one of the first to fourth thresholds is equal to 0.

[0320] In some embodiments, determining the predicted value of the transform coefficient of the current transform block based on the transform coefficient of the reference transform block of the current transform block includes: performing an averaging operation on the transform coefficients of the reference transform block of the current transform block to obtain the predicted value of the transform coefficient of the current transform block.

[0321] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0322] It should be noted that the division of modules in the device described in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present application can be integrated into a processing unit, or they can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. It can also be implemented in the form of a combination of software and hardware.

[0323] It should be noted that, in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0324] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, it implements an encoding method such as a point cloud encoder side, or implements a decoding method such as a point cloud decoder side.

[0325] An embodiment of the present application provides a point cloud decoder. As shown in FIG22 , the point cloud decoder 220 includes: a first communication interface 2201, a first memory 2202, and a first processor 2203; each component is coupled together through a first bus system 2204. It can be understood that the first bus system 2204 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 2204 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 2204 in FIG22. Among them,

[0326] The first communication interface 2201 is used to receive and send signals when sending and receiving information with other external network elements;

[0327] A first memory 2202 is used to store computer programs that can be run on the first processor 2203;

[0328] The first processor 2203 is configured to execute the decoding method described in the embodiment of the present application when running the computer program.

[0329] It is understood that the first memory 2202 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 2202 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0330] The first processor 2203 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 2203. The above-mentioned first processor 2203 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented as a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 2202 , and the first processor 2203 reads the information in the first memory 2202 and completes the steps of the above method in combination with its hardware.

[0331] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0332] Optionally, as another embodiment, the first processor 2203 is further configured to execute any one of the aforementioned 9 decoding method embodiments when running the computer program.

[0333] The present application implements a point cloud encoder, as shown in FIG23 , the point cloud encoder 230 includes: a second communication interface 2301, a second memory 2302 and a second processor 2303; each component is coupled together through a second bus system 2304. It can be understood that the second bus system 2304 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 2304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 2304 in FIG23. Among them,

[0334] The second communication interface 2301 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0335] The second memory 2302 is used to store computer programs that can be run on the second processor 2303;

[0336] The second processor 2303 is configured to, when running the computer program, execute:

[0337] Sorting the samples in the first image according to differences between the samples to obtain a first list;

[0338] A sample value of the first component having the same position coordinates as the sample in the second image is obtained according to the index number of the sample in the first list and the bit depth of the first component.

[0339] Optionally, as another embodiment, the second processor 2303 is further configured to execute the aforementioned encoding method embodiment when running the computer program.

[0340] It can be understood that the hardware functions of the second memory 2302 are similar to those of the first memory 2202, and the hardware functions of the second processor 2303 are similar to those of the first processor 2203; they will not be described in detail here.

[0341] An embodiment of the present application provides an electronic device, comprising: a processor adapted to execute a computer program; and a computer-readable storage medium storing the computer program. When executed by the processor, the computer program implements the encoding and / or decoding methods described in the embodiments of the present application. The electronic device can be any type of device capable of point cloud encoding and / or decoding, such as a mobile phone, tablet computer, laptop computer, personal computer, television, projection device, or monitoring device.

[0342] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0343] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0344] The term "and / or" in this article is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, object A and / or object B can mean: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0345] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0346] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0347] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed across multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of this embodiment.

[0348] In addition, all functional modules in the embodiments of the present application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the above-mentioned integrated modules can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0349] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0350] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0351] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined, if they do not conflict, to obtain new method embodiments. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined, if they do not conflict, to obtain new product embodiments. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined, if they do not conflict, to obtain new method embodiments or device embodiments.

[0352] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A decoding method, which is applied to a point cloud decoder, and the method includes: Decoding a bitstream to determine the residual value of the transform coefficients of the current transform block; Determining the multi-frame or multi-slice reference point cloud of the current transform block; Determining the reference transform block of the current transform block from the multi-frame or multi-slice reference point cloud; Determining the predicted value of the transform coefficients of the current transform block according to the transform coefficients of the reference transform block of the current transform block; Determining the transform coefficients of the current transform block according to the residual value and the predicted value of the transform coefficients of the current transform block.

2. The method according to claim 1, wherein, The method further includes: Obtaining first syntax identification information, where the first syntax identification information is used to indicate whether to perform transform coefficient prediction based on a multi-frame or multi-slice reference point cloud.

3. The method according to claim 2, wherein The obtaining of the first syntax identification information includes: Decoding a bitstream to determine the first syntax identification information.

4. The method according to claim 3, wherein The decoding of the bitstream to determine the first syntax identification information includes: Decoding a bitstream to determine a sequence parameter set, an attribute parameter set, and / or attribute header information; wherein, the first syntax identification information is set in the sequence parameter set, the attribute parameter set, or the attribute header information.

5. The method according to claim 3 or 4, wherein The method further includes: In the case where the first syntax identification information does not exist in the bitstream, by default, do not perform transform coefficient prediction based on a multi-frame or multi-slice reference point cloud, or by default, perform transform coefficient prediction based on a multi-frame or multi-slice reference point cloud.

6. The method according to any one of claims 1-5, wherein The method further includes: Decoding a bitstream to determine a first parameter; the first parameter is used to indicate the number of frames or slices of the reference point cloud of the current transform block.

7. The method according to any one of claims 1-6, wherein, The method further includes: Decoding a bitstream to determine a second parameter; the second parameter is used to indicate the frame index or slice index of the reference point cloud of the current transform block.

8. The method according to any one of claims 1-7, wherein, The method further includes: Determining whether the current RAHT layer uses RAHT inter prediction transform coding according to second syntax identification information.

9. The method according to claim 8, wherein The method further includes: Decoding a bitstream to determine the second syntax identification information.

10. The method according to any one of claims 1-9, wherein, The reference transform block of the current transform block includes: The transform block in the reference point cloud with the same geometric position as the current transform block.

11. According to the method according to any one of claims 1-10, wherein, The reference transform block of the current transform block includes: The transform block in the reference point cloud where the geometric position of the parent transform block is the same as that of the parent transform block of the current transform block.

12. According to the method of any one of claims 1-11, wherein, The reference transform block of the current transform block includes: The transform block in the reference point cloud with the same geometric position as the current transform block and the number of different bits in the occupancy information from that of the current transform block is less than or equal to a first threshold.

13. The method according to any one of claims 1-12, wherein, The reference transform block of the current transform block includes: The transform block in the reference point cloud with the same geometric position as the current transform block and the number of different bits in the occupancy information of the parent transform block of the reference point cloud from that of the parent transform block of the current transform block is less than or equal to a second threshold.

14. The method according to any one of claims 1-13, wherein, The reference transform block of the current transform block includes: The transform block in the reference point cloud where the geometric position of the parent transform block is the same as that of the parent transform block of the current transform block and the number of different bits in the occupancy information in the reference point cloud from that of the current transform block is less than or equal to a third threshold.

15. The method according to any one of claims 1-14, wherein, The reference transform block of the current transform block includes: A transform block in the reference point cloud whose geometric position of the parent transform block is the same as that of the parent transform block of the current transform block, and the number of different bits in the occupancy information of the parent transform block in the reference point cloud and the occupancy information of the parent transform block of the current transform block is less than or equal to a fourth threshold.

16. The method according to any one of claims 10-15, wherein, At least one of the first threshold to the fourth threshold is equal to 0.

17. The method according to any one of claims 1-16, wherein, Determining a predicted value of the transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block includes: Performing an averaging operation on the transform coefficients of the reference transform block of the current transform block to obtain a predicted value of the transform coefficient of the current transform block.

18. The method according to any one of claims 1 to 17, wherein, The method further includes: Performing a RAHT transform according to all the transform coefficients to obtain all the attribute information of the current point cloud.

19. An encoding method, which is applied to a point cloud encoder, and the method includes: Determining a multi-frame or multi-slice reference point cloud of the current transform block; Determining a reference transform block of the current transform block from the multi-frame or multi-slice reference point cloud; Determining a predicted value of the transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block; Determining a residual value of the transform coefficient of the current transform block according to the transform coefficient of the current transform block and the predicted value of the transform coefficient of the current transform block; Generating a bitstream according to the residual value of the transform coefficient of the current transform block.

20. The method according to claim 19, wherein, The method further includes: Generating a bitstream according to first syntax identification information; wherein, the first syntax identification information is used to indicate whether to perform transform coefficient prediction based on a multi-frame or multi-slice reference point cloud.

21. The method according to claim 20, wherein The first syntax identification information is set in a sequence parameter set, an attribute parameter set or attribute header information.

22. The method according to any one of claims 19-21, wherein, The method further includes: Generating a bitstream according to a first parameter; the first parameter is used to indicate the number of frames or slices of the reference point cloud of the current transform block.

23. The method according to any one of claims 19-22, wherein The method further includes: Generating a bitstream according to a second parameter; the second parameter is used to indicate the frame index or slice index of the reference point cloud of the current transform block.

24. The method according to any one of claims 19-23, wherein The method further includes: Generating a bitstream according to second syntax identification information; the second syntax identification information is used to determine whether the current RAHT layer uses RAHT inter prediction transform coding.

25. The method according to any one of claims 19 - 24, wherein The reference transform block of the current transform block includes: A transform block in the reference point cloud whose geometric position is the same as that of the current transform block.

26. The method according to any one of claims 19 - 25, wherein, The reference transform block of the current transform block includes: A transform block in the reference point cloud whose geometric position of the parent transform block is the same as that of the parent transform block of the current transform block.

27. The method according to any one of claims 19-26, wherein, The reference transform block of the current transform block includes: A transform block in the reference point cloud whose geometric position is the same as that of the current transform block and the number of different bits in the occupancy information in the reference point cloud and the occupancy information of the current transform block is less than or equal to a first threshold.

28. The method according to any one of claims 19-27, wherein, The reference transform block of the current transform block includes: A transform block in the reference point cloud whose geometric position is the same as that of the current transform block, and the number of different bits between the occupancy information of the parent transform block of the reference point cloud and the occupancy information of the parent transform block of the current transform block is less than or equal to a second threshold.

29. The method according to any one of claims 19-28, wherein, The reference transform block of the current transform block, including: A transform block in the reference point cloud whose geometric position of the parent transform block is the same as that of the parent transform block of the current transform block, and the number of different bits between the occupancy information in the reference point cloud and the occupancy information of the current transform block is less than or equal to a third threshold.

30. The method according to any one of claims 19 - 29, wherein, The reference transform block of the current transform block, including: A transform block in the reference point cloud whose geometric position of the parent transform block is the same as that of the parent transform block of the current transform block, and the number of different bits between the occupancy information of the parent transform block in the reference point cloud and the occupancy information of the parent transform block of the current transform block is less than or equal to a fourth threshold.

31. The method according to any one of claims 25 - 30, wherein At least one of the first threshold to the fourth threshold is equal to 0.

32. The method according to any one of claims 19 - 31, wherein, Determining a predicted value of the transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block, including: Performing an averaging operation on the transform coefficients of the reference transform block of the current transform block to obtain a predicted value of the transform coefficient of the current transform block.

33. A decoding device, applied to a point cloud decoder, the device includes: A decoding module, configured to decode a bitstream and determine a residual value of the transform coefficient of the current transform block; A first determination module, configured to determine a multi-frame or multi-slice reference point cloud of the current transform block; A second determination module, configured to determine a reference transform block of the current transform block from the multi-frame or multi-slice reference point cloud; A third determination module, configured to determine a predicted value of the transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block; A fourth determination module, configured to determine the transform coefficient of the current transform block according to the residual value and the predicted value of the transform coefficient of the current transform block.

34. A point cloud decoder, including a first memory and a first processor; wherein, The first memory is used to store a computer program that can run on the first processor; The first processor is used to execute the method according to any one of claims 1 to 18 when running the computer program.

35. An encoding device, applied to a point cloud encoder, the device includes: A fifth determination module, configured to determine a multi-frame or multi-slice reference point cloud of the current transform block; A sixth determination module, configured to determine a reference transform block of the current transform block from the multi-frame or multi-slice reference point cloud; A seventh determination module, configured to determine a predicted value of the transform coefficient of the current transform block according to the transform coefficient of the reference transform block of the current transform block; An eighth determination module, configured to determine a residual value of the transform coefficient of the current transform block according to the transform coefficient of the current transform block and the predicted value of the transform coefficient of the current transform block; An encoding module, configured to generate a bitstream according to the residual value of the transform coefficient of the current transform block.

36. A point cloud encoder, including a second memory and a second processor; wherein, The second memory is configured to store a computer program that can run on the second processor; The second processor is configured to execute the method according to any one of claims 19 to 32 when running the computer program.

37. A point cloud bitstream, which is generated by the encoding method according to any one of claims 19 to 32.

38. An electronic device, comprising: A processor adapted to execute a computer program; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the decoding method according to any one of claims 1 to 18, or which, when executed by the processor, implements the encoding method according to any one of claims 19 to 32.

39. A computer-readable storage medium, where The computer-readable storage medium stores a computer program, which, when executed, implements the decoding method according to any one of claims 1 to 18, or implements the encoding method according to any one of claims 19 to 32.

Images