Point cloud coding method, coder, bitstream and storage medium

Abstract

This application provides a point cloud encoding / decoding method, an encoder / decoder, a bitstream, and a storage medium. The point cloud decoding method includes: parsing the bitstream to determine first identification information; and decoding the attribute information of the current layer based on the first identification information. The current layer is any one of multiple Region Adaptive Layer Transform (RAHT) layers. The first identification information indicates whether the current layer uses a first decoding method or a second decoding method. Under the first decoding method, each node in the current layer corresponds to the same decoding mode. Under the second decoding method, the decoding mode corresponding to each node in the current layer needs to be determined individually.

Description

Point cloud encoding and decoding methods, codecs, bitstreams, and storage media Technical Field

[0001] The present application relates to the field of point cloud encoding and decoding technology, and in particular to a point cloud encoding and decoding method, codec, bit stream and storage medium. Background Art

[0002] In the point cloud encoding and decoding framework, a region adaptive hierarchical transform (RAHT)-based encoding and decoding scheme can be activated during the encoding and decoding of point cloud attribute information. However, the current RAHT-based encoding and decoding scheme has the problem of low encoding and decoding efficiency.

[0003] Summary of the Invention

[0004] The present invention provides a point cloud encoding and decoding method, codec, code stream, and storage medium. The following describes various aspects of the present invention.

[0005] In a first aspect, a point cloud decoding method is provided, which is applied to a decoder, including: parsing a code stream to determine first identification information; decoding attribute information of a current layer based on the first identification information; wherein the current layer is any one of a plurality of RAHT layers, and the first identification information is used to indicate that the current layer uses a first decoding method or a second decoding method, and under the first decoding method, each node in the current layer corresponds to the same decoding mode, and under the second decoding method, the decoding modes corresponding to the nodes in the current layer need to be determined separately.

[0006] In a second aspect, a point cloud coding method is provided, which is applied to an encoder, including: determining the costs of a first coding method and a second coding method; encoding attribute information of a current layer according to the costs of the first coding method and the second coding method; wherein the current layer is any one of a plurality of RAHT layers, and under the first coding method, each node in the current layer corresponds to the same coding mode, and under the second coding method, the coding modes corresponding to the nodes in the current layer need to be determined separately.

[0007] According to a third aspect, a decoder is provided, including: a first decoding unit configured to parse a code stream and determine first identification information; and a second decoding unit configured to decode attribute information of a current layer based on the first identification information; wherein the current layer is any one of a plurality of RAHT layers, and the first identification information is used to indicate that the current layer uses a first decoding method or a second decoding method, and under the first decoding method, each node in the current layer corresponds to the same decoding mode, and under the second decoding method, the decoding modes corresponding to the nodes in the current layer need to be determined separately.

[0008] In a fourth aspect, a decoder is provided, comprising: a memory for storing a computer program; and a processor for executing the method of the first aspect when running the computer program.

[0009] In a fifth aspect, an encoder is provided, comprising: a first encoding unit configured to determine the cost of a first encoding mode and a second encoding mode; a second encoding unit configured to encode attribute information of a current layer according to the cost of the first encoding mode and the second encoding mode; wherein the current layer is any one of multiple RAHT layers, and under the first encoding mode, each node in the current layer corresponds to the same encoding mode, and under the second encoding mode, the encoding modes corresponding to the nodes in the current layer need to be determined separately.

[0010] In a sixth aspect, an encoder is provided, comprising: a memory for storing a computer program; and a processor for executing the method of the second aspect when running the computer program.

[0011] In a seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method of the first aspect or the second aspect is implemented.

[0012] In an eighth aspect, a non-volatile computer-readable storage medium for storing a bit stream is provided, wherein the bit stream is generated by an encoding method using an encoder, or the bit stream is decoded by a decoding method using a decoder, wherein the decoding method is the method of the first aspect and the encoding method is the method of the second aspect.

[0013] According to a ninth aspect, a code stream is provided, comprising a code stream generated according to the method of the second aspect.

[0014] The reason why the RAHT-based encoding and decoding scheme provided by the related art is less efficient is that the scheme determines the encoding and decoding mode for each node in the RAHT layer. Although this scheme can make the encoding and decoding mode corresponding to each node the better (or even the best) encoding and decoding mode for the node, determining the encoding and decoding mode node by node will result in low encoding and decoding efficiency. Unlike the related art, the embodiment of the present application introduces an encoding and decoding scheme for the RAHT layer. If this encoding and decoding scheme is selected, each node in the RAHT layer will correspond to the same encoding and decoding mode. In other words, the embodiment of the present application regards the RAHT layer as a coding unit and determines the encoding and decoding mode at the RAHT layer as the granularity. Compared with the node, the RAHT layer granularity is slightly larger, but compared with the slice, the RAHT layer granularity is smaller. Therefore, the coding unit of the RAHT layer granularity can maximize the encoding and decoding efficiency of the point cloud attribute information while ensuring high encoding and decoding accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG1A is a schematic diagram of a three-dimensional point cloud image.

[0016] FIG1B is a partially enlarged view of a three-dimensional point cloud image.

[0017] FIG2A is a schematic diagram of six viewing angles of a point cloud image.

[0018] FIG2B is a schematic diagram of a data storage format corresponding to a point cloud image.

[0019] FIG3 is a schematic diagram of a network architecture for point cloud encoding and decoding.

[0020] FIG4A is a schematic diagram of a composition framework of a G-PCC encoder.

[0021] FIG4B is a schematic diagram of a composition framework of a G-PCC decoder.

[0022] FIG5A is a schematic diagram of a low plane position in the Z-axis direction.

[0023] FIG5B is a schematic diagram of a high plane position in the Z-axis direction.

[0024] FIG6 is a schematic diagram of a node encoding sequence.

[0025] FIG. 7A is a schematic diagram of plane identification information.

[0026] FIG. 7B is a schematic diagram of another type of planar identification information.

[0027] FIG8 is a schematic diagram of sibling nodes of a current node.

[0028] FIG. 9A is a schematic diagram showing three intersection points included in a sub-block.

[0029] FIG9B is a schematic diagram of a triangular facet set fitted using three intersection points.

[0030] FIG9C is a schematic diagram of upsampling of a triangle face set.

[0031] FIG10 is a schematic diagram of a distance-based level of detail (LOD) construction.

[0032] FIG11 is a schematic diagram of a distance-based LOD point cloud generation process.

[0033] FIG12 is a schematic diagram of a process for encoding attribute information of an LOD point cloud.

[0034] FIG13 is a schematic diagram of the structure of a refinement layer based on LOD division.

[0035] FIG14 is a schematic diagram of an inter-layer nearest neighbor search based on LOD.

[0036] FIG15A is a schematic diagram showing a spatial relationship between a child block and a parent block.

[0037] FIG15B is a schematic diagram of a neighbor block that is coplanar, colinear, and co-point with the current parent block.

[0038] FIG16 is a schematic diagram of a method for performing nearest neighbor search for a current point.

[0039] FIG17 is a schematic diagram of a method for searching the nearest neighbor within an attribute information layer.

[0040] FIG18 is a schematic diagram of a fast search method within an LOD layer.

[0041] FIG19 is a schematic diagram of a neighborhood search prediction structure based on Morton code.

[0042] FIG20 is a schematic diagram of an encoding process of a lifting transform.

[0043] FIG21 is an example diagram of a region adaptive hierarchical transform (RAHT) transformation process.

[0044] FIG22 is another example diagram of the RAHT transformation process.

[0045] FIG23 is a schematic diagram of RAHT transformation and inverse RAHT transformation.

[0046] FIG24 is a schematic diagram of the encoding block structure of attribute information.

[0047] FIG25 is a schematic diagram of the overall process of RAHT intra-frame prediction transform coding of attribute information.

[0048] FIG26 is an example diagram of a linear fitting method for the neighborhood attribute information of the current block.

[0049] FIG27 is a schematic diagram of RAHT intra-frame prediction transform coding of attribute information.

[0050] FIG28 is a schematic diagram of an inter-frame prediction process of attribute information of a RAHT layer.

[0051] FIG29 is a schematic structural diagram of a RAHT layer.

[0052] Figure 30 is a flowchart of the decoding method provided in an embodiment of the present application.

[0053] Figure 31 is a flow chart of the encoding method provided in an embodiment of the present application.

[0054] Figure 32 is a flow chart of an encoding method provided in an embodiment of the present application.

[0055] Figure 33 is a flow chart of another encoding method provided in an embodiment of the present application.

[0056] Figure 34 is a flowchart of a decoding method provided in an embodiment of the present application.

[0057] Figure 35 is a flow chart of another decoding method provided in an embodiment of the present application.

[0058] FIG36 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application.

[0059] Figure 37 is a schematic structural diagram of a decoder provided in another embodiment of the present application.

[0060] Figure 38 is a schematic diagram of the structure of the encoder provided in one embodiment of the present application.

[0061] Figure 39 is a schematic structural diagram of an encoder provided in another embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0064] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0065] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0066] A point cloud is a set of irregularly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. These points contain geometric information representing spatial location and attribute information representing the point cloud's appearance and texture. Figure 1A shows a 3D point cloud image, and Figure 1B shows a zoomed-in view of a 3D point cloud image. It can be seen that the point cloud surface is composed of densely distributed points.

[0067] In a two-dimensional image, each pixel contains information and is distributed regularly, so there's no need to record its location. However, the distribution of points in a point cloud in three-dimensional space is random and irregular, so recording the location of each point in space is necessary to fully represent the point cloud. Similar to a two-dimensional image, each location in the acquisition process has corresponding attribute information, typically an RGB color value, which reflects the object's color. For a point cloud, in addition to color information, each point's corresponding attribute information often includes reflectance values, which reflect the surface texture of the object. Therefore, point cloud data typically includes both point location information and point attribute information. Point location information can also be referred to as point geometric information. For example, point geometric information can be the point's three-dimensional coordinates (x, y, z). Point attribute information can include color information and / or reflectance. For example, reflectance can be one-dimensional reflectance information (r). Color information can be information in any color space, or it can be three-dimensional color information, such as RGB. Here, R represents red (red), G represents green (green), and B represents blue (blue). For another example, the color information may be luminance and chrominance (YCbCr, YUV) information, where Y represents brightness (luma), Cb (U) represents blue color difference, and Cr (V) represents red color difference.

[0068] For example, a point cloud generated using laser measurement principles can include both its 3D coordinate information and its reflectivity. For another example, a point cloud generated using photogrammetry principles can include both its 3D coordinate information and its 3D color information. For another example, a point cloud generated using a combination of laser measurement and photogrammetry principles can include both its 3D coordinate information, its reflectivity value, and its 3D color information.

[0069] Figures 2A and 2B show a point cloud image and its corresponding data storage format. Figure 2A provides six viewing angles of the point cloud image, while Figure 2B consists of a file header and data. The header includes the data format, data representation type, the total number of points in the point cloud, and the content represented by the point cloud. For example, the point cloud is in ".ply" format, represented by ASCII code, with a total of 207,242 points. Each point has 3D coordinate information (x, y, z) and 3D color information (r, g, b).

[0070] Point clouds can be divided into the following categories according to the acquisition method:

[0071] Static point cloud: the object is stationary and the device that acquires the point cloud is also stationary;

[0072] Dynamic point cloud: The object is moving, but the device that obtains the point cloud is stationary;

[0073] Dynamic point cloud acquisition: The device used to acquire the point cloud is in motion.

[0074] For example, point clouds can be divided into two categories according to their usage:

[0075] Category 1: Machine perception point cloud, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and disaster relief robots;

[0076] Category 2: Human eye perception point cloud, which can be used in point cloud application scenarios such as digital cultural heritage, free viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0077] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Moreover, since point clouds are obtained by directly sampling real objects, they can provide a strong sense of reality while ensuring accuracy. Therefore, they are widely used, including virtual reality games, computer-aided design, geographic information systems, automatic navigation systems, digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive remote presentation, and three-dimensional reconstruction of biological tissues and organs.

[0078] Point clouds are primarily collected through computer generation, 3D laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual 3D objects and scenes; 3D laser scanning can obtain point clouds of static real-world 3D objects or scenes, generating millions of point clouds per second; and 3D photogrammetry can obtain point clouds of dynamic real-world 3D objects or scenes, generating tens of millions of point clouds per second. These technologies reduce the cost and time required to acquire point cloud data while improving data accuracy. While changes in point cloud data acquisition methods have made it possible to acquire large amounts of point cloud data, the processing of this massive amount of 3D point cloud data is facing bottlenecks due to storage space and transmission bandwidth constraints, as application demands grow.

[0079] For example, taking a point cloud video with a frame rate of 30 frames per second (fps), each frame contains 700,000 points, and each point has coordinate information (xyz, float) and color information (RGB, uchar). Therefore, the data volume of a 10-second point cloud video is approximately 0.7 million × (4 bytes × 3 + 1 byte × 3) × 30 fps × 10 seconds = 3.15 GB, where 1 byte is 10 bits. For a 1280 × 720 2D video with a YUV sampling format of 4:2:0 and a frame rate of 30 fps, the data volume for 10 seconds is approximately 1280 × 720 × 12 bits × 30 frames × 10 seconds, which is approximately 0.39 GB. A 10-second two-view 3D video has a data volume of approximately 0.39 × 2 = 0.78 GB. This shows that the data volume of a point cloud video far exceeds that of 2D and 3D videos of the same length. Therefore, in order to better realize data management, save server storage space, and reduce the transmission traffic and transmission time between the server and the client, point cloud compression has become a key issue in promoting the development of the point cloud industry.

[0080] That is to say, since the point cloud is a collection of massive points, storing the point cloud not only consumes a lot of memory, but is also not conducive to transmission. There is also not enough bandwidth to support direct transmission of the point cloud at the network layer without compression. Therefore, the point cloud needs to be compressed.

[0081] Currently, point cloud coding frameworks that can compress point clouds can be the geometry-based point cloud compression (G-PCC) codec framework or the video-based point cloud compression (V-PCC) codec framework provided by the Moving Picture Experts Group (MPEG), or the AVS-PCC codec framework provided by AVS. The G-PCC codec framework can be used to compress the first type of static point clouds and the third type of dynamically acquired point clouds, and it can be based on the point cloud compression test platform (test model compression 13, TMC13). The V-PCC codec framework can be used to compress the second type of dynamic point clouds, and it can be based on the point cloud compression test platform (test model compression 2, TMC2). Therefore, the G-PCC codec framework is also called the point cloud codec TMC13, and the V-PCC codec framework is also called the point cloud codec TMC2.

[0082] An embodiment of the present application provides a network architecture of a point cloud encoding and decoding system including a decoding method and an encoding method. FIG3 is a schematic diagram of a network architecture of a point cloud encoding and decoding system provided by an embodiment of the present application. As shown in FIG3 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During the implementation process, the electronic device can be various types of devices with point cloud encoding and decoding functions. For example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., which is not limited by the embodiment of the present application. Among them, the decoder or encoder in the embodiment of the present application can be the above-mentioned electronic device.

[0083] Among them, the electronic device in the embodiment of the present application has a point cloud encoding and decoding function, generally including a point cloud encoder (ie, encoder) and a point cloud decoder (ie, decoder).

[0084] The following describes the related technologies using the G-PCC codec framework and the AVS codec framework as examples.

[0085] As you can understand, in the point cloud G-PCC codec framework, the point cloud data to be encoded is first divided into multiple slices (slices, also called strips). In each slice, the geometric information of the point cloud and the attribute information corresponding to each point are encoded separately.

[0086] Figure 4A shows a schematic diagram of the G-PCC encoder's architecture. As shown in Figure 4A, during the geometry encoding process, the geometric information is transformed so that the entire point cloud is contained within a bounding box. Quantization is then performed. This quantization step primarily serves a scaling purpose. Due to quantization rounding, the geometric information of some point clouds becomes identical. Parameters are then used to determine whether to remove duplicate points. This process of quantization and removing duplicate points is also known as voxelization. The bounding box is then partitioned into an octree or constructed as a prediction tree. During this process, arithmetic coding is performed on the points within the leaf nodes of the partition to generate a binary geometry bitstream. Alternatively, arithmetic coding is performed on the intersections (vertex) generated by the partition (surface fitting is performed based on the intersections) to generate a binary geometry bitstream. During the attribute encoding process, after the geometry encoding is completed and the geometric information is reconstructed, color conversion is performed to convert the color information (i.e., attribute information) from the RGB color space to the YUV color space. The reconstructed geometry information is then used to recolor the point cloud so that the unencoded attribute information corresponds to the reconstructed geometry information. Attribute coding is mainly performed on color information. In the process of color information coding, there are three main transformation methods. The first two methods rely on the level of detail (LOD) division, which are distance-based lifting transformation and prediction transformation respectively. The third method is to directly perform RAHT. All three methods will convert color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through transformation, and finally quantize the coefficients. Then, the quantized coefficients are arithmetically coded to generate a binary attribute bit stream.

[0087] Figure 4B shows a schematic diagram of the composition framework of a G-PCC decoder. As shown in Figure 4B, for the acquired binary bit stream, the geometric bit stream and attribute bit stream in the binary bit stream are first decoded independently. When decoding the geometric bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-reconstruction of the octree / reconstruction of the prediction tree-reconstruction of the geometry-coordinate inverse conversion; when decoding the attribute bit stream, the attribute information of the point cloud is obtained through arithmetic decoding-inverse quantization-LOD partitioning / RAHT-color inverse conversion, and the point cloud data to be encoded (i.e., the output point cloud) is restored based on the geometric information and attribute information.

[0088] It should be noted that, as shown in FIG4A or FIG4B , the current geometric coding and decoding of G-PCC can be divided into octree-based geometric coding and decoding (marked by a dotted box) and prediction tree-based geometric coding and decoding (marked by a dotted box).

[0089] For octree geometry encoding (OctGeomEnc), octree geometry encoding includes: first, coordinate conversion of the geometric information so that all point clouds are contained in a bounding box. Then quantization is performed. This step of quantization mainly plays a role of scaling. Due to the quantization rounding, the geometric information of some points is the same. The parameters are used to decide whether to remove duplicate points. The process of quantization and removal of duplicate points is also called voxelization. Next, the bounding box is continuously divided into trees (such as octrees, quadtrees, binary trees, etc.) in the order of breadth-first traversal, and the placeholder code of each node is encoded. In related technologies, a company proposed an implicit geometric division method. First, the bounding box of the point cloud is calculated. Assume d x >d y >d z , the bounding box corresponds to a cuboid. When geometrically partitioning, the binary tree partitioning will be performed based on the x-axis to obtain two child nodes; until d is satisfied x =d y >d z When the conditions are met, the quadtree partitioning will be performed based on the x and y axes to obtain four child nodes; when d is finally satisfied x =d y =d z When the conditions are met, the octree partitioning will continue until the leaf node obtained by the partitioning is a 1×1×1 unit cube. The partitioning will stop and the points in the leaf node will be encoded to generate a binary code stream. In the process of binary tree / quadtree / octree partitioning, two parameters are introduced: K and M. Parameter K indicates the maximum number of binary tree / quadtree partitions before octree partitioning; parameter M is used to indicate that the minimum block side length corresponding to binary tree / quadtree partitioning is 2 M . At the same time, K and M must meet the following conditions: Assume d max = max(d x ,d y ,d z ), d min =min(d x ,d y ,d z ), parameter K satisfies: K ≥ d max -d min ; Parameter M satisfies: M≥d minThe reason why the parameters K and M meet the above conditions is that in the current G-PCC geometric implicit partitioning process, the priority of the partitioning method is binary tree, quadtree and octree. When the node block size does not meet the conditions of binary tree / quadtree, the node will be divided into octree until the minimum unit of leaf node is 1×1×1. The octree-based geometric information coding mode can effectively encode the geometric information of the point cloud by utilizing the correlation between adjacent points in space. However, for some relatively flat nodes or nodes with planar characteristics, the encoding performance of the point cloud geometric information can be further improved by using plane coding.

[0090] Exemplarily, Figure 5A and Figure 5B provide a kind of plane position schematic diagram.Wherein, Figure 5A shows a kind of low plane position schematic diagram in the Z-axis direction, and Figure 5B shows a kind of high plane position schematic diagram in the Z-axis direction.As shown in Figure 5A, here (a), (a0), (a1), (a2), (a3) ​​all belong to the low plane position in the Z-axis direction. Taking (a) as an example, it can be seen that the four child nodes occupied in the current node are all located at the low plane position of the current node in the Z-axis direction, so it can be considered that the current node belongs to a Z plane and is a low plane in the Z-axis direction. Similarly, as shown in Figure 5B, here (b), (b0), (b1), (b2), (b3) all belong to the high plane position in the Z-axis direction. Taking (b) as an example, it can be seen that the four child nodes occupied in the current node are located at the high plane position of the current node in the Z-axis direction, so it can be considered that the current node belongs to a Z plane and is a high plane in the Z-axis direction.

[0091] Furthermore, taking (a) in Figure 5A as an example, the performance of octree encoding and plane encoding is compared. Figure 6 provides a schematic diagram of the node encoding sequence, that is, node encoding is performed in the order of 0, 1, 2, 3, 4, 5, 6, and 7 as shown in Figure 6. Here, if octree encoding is used for (a) in Figure 5A, the placeholder information of the current node is represented as: 10101010. However, if plane encoding is used, first, an identifier must be encoded to indicate that the current node is a plane in the Z-axis direction. Secondly, if the current node is a plane in the Z-axis direction, the plane position of the current node must also be represented. Finally, only the placeholder information of the lower plane node in the Z-axis direction needs to be encoded (i.e., the placeholder information of the four child nodes 0, 2, 4, and 6). Therefore, encoding the current node based on plane encoding only requires 6 bits, which can reduce the representation by 2 bits compared to the related art octree encoding. Based on this analysis, plane encoding has significantly better coding performance than octree encoding. Therefore, for an occupied node, if a plane encoding method is used in a certain dimension, it is first necessary to represent the plane identification (planarMode) and plane position (PlanePos) information of the current node in that dimension, and then encode the occupancy information of the current node based on the plane information of the current node. For example, FIG7A shows a schematic diagram of plane identification information. As shown in FIG7A, there is a low plane in the Z-axis direction; correspondingly, the value of the plane identification information is true (true) or 1, that is, planarMode_ Z = true; the plane position information is the low plane (low), that is, PlanePosition_ Z =low. FIG7B shows another schematic diagram of plane identification information. As shown in FIG7B, here it is not a plane in the Z-axis direction; correspondingly, the value of the plane identification information is false (false) or 0, that is, planarMode_ Z =false.

[0092] It should be noted that for PlaneMode_ i :0 means the current node is not a plane in the i-axis direction, 1 means the current node is a plane in the i-axis direction. If the current node is a plane in the i-axis direction, then for PlanePosition_ i : 0 means the current node is a plane in the i-axis direction and the plane position is low, 1 means the current node is a high plane in the i-axis direction. Here, i represents the coordinate dimension, which can be the X-axis direction, Y-axis direction, or Z-axis direction, so i = 0, 1, 2.

[0093] The octree-based geometric information coding mode only has an efficient compression rate for points that are correlated in space. For points that are isolated in the geometric space, the use of the direct coding model (DCM) can greatly reduce the complexity. For all nodes in the octree, the use of DCM is not indicated by flag information, but is inferred from the parent node and neighbor information of the current node. There are three ways to determine whether the current node is eligible for DCM coding, as follows:

[0094] (1) The current node has no sibling child nodes, that is, the parent node of the current node has only one child node, and the parent node of the parent node of the current node has only two occupied child nodes, that is, the current node has at most one neighbor node.

[0095] (2) The parent node of the current node has only one child node, the current node, and the six neighbor nodes that share a face with the current node are all empty nodes.

[0096] (3) The number of sibling nodes of the current node is greater than 1.

[0097] Exemplarily, FIG8 provides a schematic diagram of IDCM coding. If the current node does not have the DCM coding qualification, it will be divided into octrees. If it has the DCM coding qualification, the number of points contained in the node will be further judged. When the number of points is less than a threshold (for example, 2), the node will be DCM-encoded, otherwise the octree division will continue. When the DCM coding mode is applied, it is first necessary to encode whether the current node is a true isolated point, that is, IDCM_flag. When IDCM_flag is true, the current node adopts DCM coding, otherwise octree coding is still adopted. When the current node meets the DCM coding requirements, the DCM coding mode of the current node needs to be encoded. There are currently two DCM modes, namely: (a) there is only one point (or multiple points, but they are duplicate points); (b) there are two points. Finally, the geometric information of each point needs to be encoded. Assuming that the side length of the node is 2 d When encoding each component of the node's geometric coordinates, d bits are required, and these bits are directly encoded into the bitstream. It is important to note that when encoding LiDAR point clouds, the performance of geometric information encoding can be further improved by predictively encoding the three-dimensional coordinate information using LiDAR acquisition parameters.

[0098] It is important to note that when partitioning nodes into leaf nodes, the number of duplicate points in the leaf nodes must be encoded in the case of lossless geometric coding. Ultimately, the placeholder information for all nodes is encoded to generate a binary bitstream. Furthermore, G-PCC currently introduces a plane coding mode. During the geometric partitioning process, it determines whether the child nodes of the current node are in the same plane. If the child nodes of the current node meet the condition of being in the same plane, the child nodes of the current node are represented by that plane.

[0099] For octree-based geometric decoding, the decoder follows a breadth-first traversal. Before decoding each node's occupancy information, it first uses the reconstructed geometric information to determine whether the current node is for plane decoding or IDCM decoding. If the current node meets the requirements for plane decoding, it first decodes the plane identifier and plane position information of the current node. Then, based on the plane information, it decodes the current node's occupancy information. If the current node meets the requirements for IDCM decoding, it first decodes whether the current node is a true IDCM node. If so, it continues to parse the DCM decoding mode of the current node, then obtains the number of points in the current DCM node, and finally decodes the geometric information of each point. For nodes that do not meet either plane decoding or DCM decoding requirements, the current node's occupancy information is decoded. By continuously parsing in this way, the placeholder code of each node is obtained, and the node is continuously partitioned until a 1×1×1 unit cube is obtained. The number of points contained in each leaf node is parsed, and the geometrically reconstructed point cloud information is finally recovered.

[0100] For triangle soup (trisoup)-based geometric information coding, geometric partitioning must also be performed first in the trisoup-based geometric information coding framework. However, unlike geometric information coding based on binary trees, quadtrees, and octrees, this method does not require step-by-step partitioning of the point cloud into unit cubes with side lengths of 1×1×1. Instead, the partitioning stops when the sub-blocks (blocks) have a side length of W. Based on the surface formed by the distribution of the point cloud in each block, the surface and the twelve edges of the block are obtained. The vertex coordinates of each block are encoded in sequence to generate a binary code stream.

[0101] For trisoup-based point cloud geometry reconstruction, when performing point cloud geometry reconstruction at the decoding end, the vertex coordinates are first decoded to complete the triangle face reconstruction. This process is shown in Figures 9A, 9B, and 9C. Among them, there are three intersection points (v1, v2, v3) in the block shown in Figure 9A. The set of triangle faces formed by these three intersection points in a certain order is called triangle soup, or trisoup, as shown in Figure 9B. Afterwards, sampling is performed on the triangle face set, and the obtained sampling points are used as the reconstructed point cloud within the block, as shown in Figure 9C.

[0102] Predictive geometry coding (PredGeomTree) involves first sorting the input point cloud. Currently used sorting methods include unordered, Morton order, azimuth order, and radial distance order. At the encoding end, the prediction tree structure is established using two different methods: KD-Tree (high-latency slow mode) and low-latency fast mode (using lidar calibration information). When using lidar calibration information, each point is assigned to a different laser, and a prediction tree structure is established based on the different lasers. Next, based on the prediction tree structure, each node in the prediction tree is traversed, and the geometric position information of the node is predicted by selecting different prediction modes to obtain a prediction residual. The geometric prediction residual is then quantized using a quantization parameter. Finally, through continuous iteration, the prediction residual of the prediction tree node position information, the prediction tree structure, and the quantization parameters are encoded to generate a binary bitstream.

[0103] For geometric decoding based on the prediction tree, the decoding end continuously parses the bitstream to reconstruct the prediction tree structure. Secondly, the geometric position prediction residual information and quantization parameters of each prediction node are obtained through parsing, and the prediction residual is dequantized to restore the reconstructed geometric position information of each node, finally completing the geometric reconstruction at the decoding end.

[0104] After the geometric encoding is completed, the geometric information needs to be reconstructed. At present, attribute encoding is mainly performed on color information. First, the color information is converted from the RGB color space to the YUV color space. Then, the point cloud is recolored using the reconstructed geometric information so that the unencoded attribute information corresponds to the reconstructed geometric information. In color information encoding, there are two main transformation methods. One is the distance-based lifting transformation that relies on LOD partitioning, and the other is to directly perform RAHT transformation. Both methods will convert the color information from the spatial domain to the frequency domain, and obtain high-frequency coefficients and low-frequency coefficients through transformation. Finally, the coefficients are quantized and encoded to generate a binary code stream, as shown in Figures 4A and 4B.

[0105] Furthermore, when using geometric information to predict attribute information, Morton codes can be used to perform nearest neighbor search. The Morton code corresponding to each point in the point cloud can be obtained from the geometric coordinates of the point. The specific method for calculating the Morton code is described below. For a three-dimensional coordinate represented by a d-bit binary number for each component, its three components can be expressed as:

[0106] Among them, x l ,y l ,z l ∈{0,1} are the binary values ​​corresponding to the highest bit (l=1) to the lowest bit (l=d) of x, y, and z respectively. The Morton code M is to cross-arrange x, y, and z starting from the highest bit. l ,y l ,z l To the lowest bit, the calculation formula of M is as follows:

[0107] Among them, m l′ ∈{0,1} are the values ​​of the highest bit (l′=1) to the lowest bit (l′=3d) of M. After obtaining the Morton code M of each point in the point cloud, the points in the point cloud are arranged in ascending order of the Morton code, and the weight value w of each point is set to 1.

[0108] As mentioned earlier, there are three main transform methods for encoding attribute information: lifting transform, predictive transform, and RAHT transform. Lifting transform and predictive transform predict and transform the point cloud based on the LOD generation order, while RAHT transform adaptively transforms attribute information from the bottom up based on the octree construction hierarchy. The following sections describe these three transform encoding methods, starting with a detailed description of predictive transform encoding.

[0109] In the current attribute prediction module of G-PCC, a nearest neighbor attribute prediction coding scheme based on the LOD structure is adopted. The LOD construction methods include a distance-based LOD construction scheme, a fixed sampling rate-based LOD construction scheme, and an octree-based LOD construction scheme. In the distance-based LOD construction scheme, the point cloud is first Morton sorted before constructing the LOD to ensure that there is a strong attribute correlation between adjacent points. Figure 10 shows a schematic diagram of a distance-based LOD construction. As shown in Figure 10, the point cloud is divided into L different point cloud detail layers (Rl)l=0,1,…L-1 according to the pre-set L Manhattan distances (dl)l=0,1,…L-1, where (dl)l=0,1,…L-1 satisfies dl <dl-1。

[0110] The construction process of LOD is described as follows: (1) First, mark all points in the point cloud as unvisited, and establish a set V to store the set of points that have been visited; (2) In each iteration, traverse the points in the point cloud. If the current point has been visited, ignore it. Otherwise, calculate the minimum distance D from the current point to the point set V. If D < dl, ignore the point; if D ≥ dl, mark the current point as visited and add the current point to the refinement layer Rl and the point set V; (3) The points in the level of detail LODl are composed of the points in the refinement layers R0, R1, R2... Rl; (4) Continuously repeat the above steps until all points are marked as visited.

[0111] Based on the LOD structure, the attribute information of each point is linearly weighted predicted by using the reconstructed attribute information of points in the same or higher LOD level. Among them, the maximum number of reference prediction neighbor points is determined by the high-level syntax elements of the encoder. For the attribute of each point, at the encoding end, the rate-distortion optimization algorithm is used to select the attribute information of the N nearest neighbor points found for weighted prediction; alternatively, the attribute information of a single nearest neighbor point is selected for prediction, and finally, the selected prediction mode and prediction residual are encoded. In the LOD structure, the formula for predicting the attribute information of the current point can be as follows:

[0112] Among them, N represents the number of prediction points in the nearest neighbor point set of the current point i, Pi represents the sum of the N nearest neighbor points of the current point i, Dm represents the spatial geometric distance from the nearest neighbor point m to the current point i, Attrm represents the attribute information after reconstruction of the nearest neighbor point m, and Attr i ′ represents the attribute prediction information of the current point i, and the number of points N is a preset value in advance.

[0113] To balance the attribute coding performance and parallel processing between different LOD levels, a switch is introduced in the high-level syntax elements of the encoder. This switch can control whether to introduce in-LOD prediction. For example, if the switch is turned on, in-LOD prediction is started, and then points within the same LOD level can be used for prediction. It should be noted that when the number of LOD levels is 1, in-LOD prediction will always be used.

[0114] Figure 11 shows a schematic diagram of the generation process of the distance-based LOD point cloud. As shown in Figure 11, the first image on the left is the original point cloud, and the second image on the left represents the outer contour of the point cloud. In the following images, as the level of detail gradually increases, the detailed description of the point cloud becomes clearer. The following details the prediction process of the attribute information of the LOD-structured point cloud.

[0115] Figure 12 shows a schematic diagram of the encoding process of the attribute information of the LOD point cloud. After the LOD is constructed, according to the generation order of the LOD, the three nearest neighbor points of the current point to be encoded are first found from the encoded data points. The attribute reconstruction values ​​of these three nearest neighbor points are used as candidate prediction values ​​of the current point to be encoded; then, the optimal prediction value is selected from them according to the rate-distortion optimization algorithm. For example, as shown in Table 1, when encoding the attribute value of point P2 in Figure 10, the prediction variable index of the attribute value of the nearest neighbor point P4 can be set to 1; the attribute prediction variable index of the second nearest neighbor point P5 and the third nearest neighbor point P0 can be set to 2 and 3 respectively; the prediction variable index of the weighted average of points P0, P5 and P4 is set to 0; finally, the rate-distortion optimization algorithm is used to select the best prediction variable. Among them, the formula for weighted average is as follows:

[0116] In the formula Represents the spatial geometric weight of the neighboring point j to the current point i:

[0117] Represents the attribute prediction value of the current point i, j represents the index of the three neighboring points, Represents the attribute value after reconstruction of the neighboring points, x i ,y i ,z i is the geometric position coordinate of the current point i, x ij ,y ij ,z ij is the geometric coordinate of the neighboring point j.

[0118] Table 1

[0119] The attribute prediction value of the current point i is obtained through the above prediction (k is the total number of points in the point cloud). Let (a i ) i∈0…k-1 is the original attribute value of the current point, then the attribute residual (r i ) i∈0…k-1 Denoted as:

[0120] Further quantify the prediction residuals:

[0121] In formula (7), Q i represents the quantized attribute residual at the current point i. Qs is the quantization step (Qs), which can be calculated from the quantization parameter (QP). After quantization, the quantization coefficients are arithmetic coded to generate the attribute bit rate.

[0122] During the encoding process, the encoder reconstructs the attribute value of the current point i. The purpose of reconstruction is to predict the subsequent points. Before reconstructing the attribute value, the residual must be dequantized. is the residual after inverse quantization:

[0123] and predicted value Add up to get the reconstruction value of point i

[0124] As described above, based on LOD partitioning, predicting the attribute value of the current point requires a nearest neighbor search. Currently, there are two main types of nearest neighbor search methods: intra-frame nearest neighbor search and inter-frame nearest neighbor search. The following sections describe these two nearest neighbor search methods in detail.

[0125] Intra-frame nearest neighbor search can be divided into two methods: inter-layer nearest neighbor search and intra-layer nearest neighbor search. First, let's introduce inter-layer nearest neighbor search. Figure 13 shows a schematic diagram of the structure of the refinement layer based on LOD division. As shown in Figure 13, after LOD division, different refinement layers R will form a pyramid-like structure. The method of inter-layer nearest neighbor search can be shown in Figure 14. First, based on the method shown in Figure 10, the geometric information is divided into different LOD layers, and LOD0, LOD1 and LOD2 are obtained. In the process of inter-layer nearest neighbor search, the points in LOD0 are used to predict the attributes of the points in the next layer LOD. Next, the process of inter-layer nearest neighbor search is introduced in detail.

[0126] During the entire LOD partitioning process, there are three sets: O(k), L(k), and I(k). Among them, k is the index of the LOD layer during LOD partitioning, and I(k) is the input point set during the current LOD layer partitioning. After LOD partitioning, the O(k) set and L(k) set are obtained. The O(k) set stores the sampling point set, and L(k) is the point set in the current LOD layer. The entire LOD partitioning process is as follows:

[0127] (1) Initialization;

[0128] if k=0,L(k)←{}otherwise L(k)←k(k-1)

[0129] O(k)←{}

[0130] (2) Using the LOD partitioning algorithm, the sampling points are stored in O(k), and the remaining points are divided into L(k);

[0131] (3) When the next iteration is performed, I←O(k).

[0132] It's important to note that since the LOD partitioning process is based on Morton codes, O(k), L(k), and I(k) store the Morton code index corresponding to the point. When performing inter-layer nearest neighbor search, the nearest neighbor search for a point in the L(k) set is performed in the O(k) set. The specific search method is described in detail below.

[0133] First, the nearest neighbor search is performed based on the spatial relationship. As shown in Figure 15A, when predicting the current point P, a neighbor search is performed by using the parent block (Block B) corresponding to point P. Figure 15B shows a schematic diagram of neighbor blocks that are coplanar, colinear, and co-point with the current parent block. As shown in Figure 15B, points in the coplanar and co-linear neighbor blocks with the current parent block are searched in Figure 15B to perform attribute prediction. That is to say, the coordinates of the current point are used to obtain the corresponding spatial block, and then, the nearest neighbor search is performed in the previously encoded LOD layer to find the spatial blocks that are coplanar, colinear, and co-point with the current block to obtain the N nearest neighbors of the current point.

[0134] If the N nearest neighbors of the current point are still not obtained after the nearest neighbor search for coplanar, colinear and co-point points, then the N nearest neighbors of the current point will be obtained based on the fast search algorithm. The specific method can be seen in Figure 16. Figure 16 shows a schematic diagram of the method of performing the nearest neighbor search for the current point. As shown in Figure 16, when performing inter-layer prediction of attributes, the geometric coordinates of the current point can be used to obtain the Morton code corresponding to the current point. Then, based on the Morton code of the current point, the first reference point (j) that is larger than the Morton code of the current point is found in the reference frame, and the nearest neighbor search is performed within the range of [j-searchRange, j+searchRange]. The specific method of updating the nearest neighbor is the same as the method of the inter-frame nearest neighbor search, which will be described when introducing the inter-frame nearest neighbor search and will not be repeated here. The following is a detailed introduction to the intra-layer nearest neighbor search.

[0135] Figure 17 shows a schematic diagram of the method of nearest neighbor search within the attribute information layer. As shown in Figure 17, when the intra-layer prediction method is turned on, the nearest neighbor search will be performed in the encoded point set within the same layer LOD to obtain the N nearest neighbors of the current point (the inter-layer nearest neighbor search is also performed). The method of performing the nearest neighbor search can be based on a quick search. For example, as shown in Figure 18, assuming that the Morton code index of the current point is i, the nearest neighbor search will be performed in [i+1, i+searchRange]. The specific nearest neighbor search method is consistent with the inter-frame block-based quick search method, which will not be repeated here. The inter-frame nearest neighbor search method is introduced in detail below.

[0136] Continuing to refer to Figure 16, when performing attribute inter-frame prediction, the geometric coordinates of the current point are used to obtain the Morton code corresponding to the current point. Based on the Morton code of the current point, the first reference point (j) with a Morton code larger than the current point is found in the reference frame, and then the nearest neighbor search is performed within the range of [j-searchRange, j+searchRange].

[0137] Currently, when performing nearest neighbor searches within and between frames, the neighborhood search is performed based on blocks. FIG19 shows a schematic diagram of a neighborhood search prediction structure based on Morton codes. For example, as shown in FIG19 , when performing neighborhood search for the current point (Morton code index is i), the points in the reference frame are divided into N (N=3) layers according to the Morton code. The division method can be as follows:

[0138] First layer: Assuming that the number of points in the reference frame is numPoints, first divide the points in the reference frame into M (M=2 5 =32) points are divided into one block;

[0139] Second layer: Based on the first layer, the blocks of the first layer are also processed every M (M=2 5 =32) blocks are divided into one block;

[0140] The third layer: Based on the second layer, the blocks of the first layer are also processed every M (M=2 5 =32) blocks are divided into one block.

[0141] Finally, the predicted structure shown in Figure 19 is obtained.

[0142] When performing attribute prediction based on the prediction structure shown in Figure 19, assume that the Morton code index of the current point to be encoded is i, and the first point in the reference frame with a Morton code greater than or equal to the current point has an index of j. The block index of the reference point is calculated based on j, and the specific calculation method is as follows:

[0143] First layer: BucketSize_0 = 2 5 =32;

[0144] Second layer: BucketSize_1=2 5 =32×BucketSize_0=1024;

[0145] Third layer: BucketSize_2=2 5 =32×BucketSize_1=32768.

[0146] Assume that the reference range in the prediction frame for the current point is [j-searchRange, j+searchRange]. Use j-searchRange to calculate the starting index of the third layer, and j+searchRange to calculate the ending index of the third layer. First, determine whether some blocks in the second layer need to be searched for their nearest neighbors within the blocks in the third layer. Then, for each block in the first layer, determine whether a search is required. If some blocks in the first layer need to be searched for their nearest neighbors, a point-by-point search is performed on some of the blocks in the first layer to update the nearest neighbors. The following describes the method for calculating blocks based on indexes.

[0147] Assuming that the Morton code index corresponding to the current point is index, then the index of the corresponding third-layer block is: idx_2=index / BucketSize_2 (10)

[0148] After obtaining the block index idx_2 of the third layer, the start index and end index of the block corresponding to the current block in the second layer can be obtained using idx_2: startIdx1=idx_2×BucketSize_1 (11) endIdx=idx_2×BucketSize_1+BucketSize_1-1 (12)

[0149] Based on the same algorithm, the index of the first layer block is obtained according to the index of the second layer block.

[0150] When performing a block-based nearest neighbor search, it will determine whether the current block needs to be searched for the nearest neighbor, that is, the nearest neighbor search of the filtered block. Each spatial block can obtain minPos and maxPos through two variables. MinPos represents the minimum value of the block, and maxPos represents the maximum value of the block. Assume that the distance to the farthest point among the N nearest neighbors of the current point is Dist, the coordinates of the point to be encoded are (x, y, z), and the current block is represented by (minPos, maxPos), where minPos is the minimum value of the three dimensions of the bounding box and maxPos is the maximum value of the three dimensions of the bounding box. The distance D between the current point and the bounding box is calculated as follows: int dx=int(std::max(std::max(minPos[0]-point[0],0),point[0]-maxPos[0])) (13) int dy=int(std::max(std::max(minPos[1]-point[1],0),point[1]-maxPos[1])) (14) int dz=int(std::max(std::max(minPos[2]-point[2],0),point[2]-maxPos[2])) (15) D=dx+dy+dz (16)

[0151] When D is less than or equal to Dist, the points in the current block will be traversed.

[0152] The above describes the predictive transform encoding method for point cloud attribute information. Next, the lifting transform encoding method for point cloud attribute information will be described in detail.

[0153] Figure 20 shows a schematic diagram of the encoding process of the lifting transform. As shown in Figure 20, the lifting transform also predicts and encodes the point cloud attributes based on LOD. The difference from the predictive transform described above is that the lifting transform divides the LOD into high and low layers. Then, the prediction is performed in the reverse order of the LOD generation layer, and an update operator is introduced in the prediction process to update the quantization weights of the low-level LOD midpoints to improve the accuracy of the prediction. The attribute values ​​of the low-level LOD midpoints are frequently used to predict the attribute values ​​of the high-level LOD midpoints, so the points in the low-level LOD should have greater influence. Continuing to refer to Figure 20, the encoding method of the lifting transform can be divided into three steps, namely: segmentation process, prediction process and update process. The following will introduce these three steps in detail.

[0154] Step 1: Segmentation Process

[0155] The segmentation process is to divide the complete LOD layer into a low LOD layer L(N) and a high LOD layer H(N). If a point cloud has three LOD layers, namely (LOD l ) l＝0,1,2 , after segmentation, LOD2 is the high LOD layer, denoted as H(N), (LOD l ) l＝0,1 It is the low LOD layer, denoted as L(N).

[0156] Step 2: Prediction Process

[0157] The point in the high-level LOD selects the attribute information of the nearest neighbor point from the low-level LOD as the attribute prediction value P(N) of the current point to be coded, and the prediction residual D(N) is recorded as: D(N) = H(N) - P(N) (17)

[0158] Step 3: Update Process

[0159] Update the attribute prediction residual D(N) in the high-level LOD to obtain U(N), and use U(N) to improve the attribute value of the midpoint of the low-level LOD, as shown in the following formula: L′(N)=L(N)+U(N) (18)

[0160] The above process will iterate continuously until the lowest LOD according to the order of LOD from high to low.

[0161] The LOD-based prediction scheme gives points in the lower LOD layers greater influence. The lifting wavelet transform-based transformation introduces quantization weights and updates the prediction residual based on the prediction residual D(N) and the distance between the prediction point and its adjacent points. Finally, the prediction residual is adaptively quantized using the quantization weights from the transformation process. It should be noted that the quantization weight value for each point can be determined by geometric reconstruction at the decoder, so the quantization weights should not be encoded.

[0162] The RAHT transform uses the Haar wavelet transform, which can transform the attribute information of the point cloud from the spatial domain to the frequency domain, thereby further reducing the correlation between the attribute information of the point cloud. Figure 21 is an example diagram of the RAHT transform process. As shown in Figure 21, RAHT performs wavelet transform based on the hierarchical structure of the octree, thereby associating the attribute information with the octree nodes. The attribute information of the occupied nodes in the same parent node is recursively transformed in a bottom-up manner, and the nodes in each layer are transformed from the three dimensions of x, y, and z (see Figure 22) until they are transformed to the root node of the octree. In the process of hierarchical transformation, the direct current (DC) coefficient (or low-pass coefficient) obtained after the transformation of the nodes in the same layer is passed to the nodes in the upper layer for further transformation, and all alternating current (AC) coefficients (or high-pass coefficients) will be quantized and encoded.

[0163] Figure 23 is a schematic diagram of RAHT transformation and inverse RAHT transformation. Assume that g′ L,2x,y,z And g′L,2x+1,y,z are the DC coefficients of two neighboring points in the L layer. After RAHT transformation, the information of the L-1 layer is the AC coefficient f′ L-1,x,y,z and DC coefficient g′ L-1,x,y,z . f′ L-1,x,y,z No more transformation will be performed, and quantization coding will be performed directly, g′ L-1,x,y,z We will continue to search for neighboring points to transform. If no neighboring points are found, we can transform g′ L-1,x,y,z Directly passed to the L-2 layer. That is to say, RAHT transformation is only valid for nodes with neighboring points, and nodes without neighboring points will be directly passed to the previous layer. In the above transformation process, g′ L,2x,y,z The weights corresponding to g′L, 2x+2, y, and z (the weights can be determined based on the number of non-empty child nodes in the node) are w′ L,2x,y,z and w′L,2x+1,y,z (abbreviated as w′0 and w′1), g′ L-1,x,y,z The weight is w′ L-1,x,y,z , then the general transformation formula of RAHT transformation is:

[0164] In formula (19), T w0,w1 is the transformation matrix, which can be determined based on formula (20):

[0165] The transformation matrix will be updated adaptively as the weights corresponding to each point are transformed. The above transformation process will be iterated and updated continuously according to the partitioning structure of the octree until the root node of the octree is reached.

[0166] Based on the RAHT transform, RAHT intra-frame prediction can be performed on the attribute information, that is, RAHT intra-frame prediction combined with transform coding can be performed on the attribute information. This coding mode is described in detail below.

[0167] As shown in Figure 24, the RAHT transform is based on the order of the octree hierarchy, and continuously transforms from the voxel level until the root node is obtained, thereby completing the hierarchical transform coding of the entire attribute information. In RAHT intra-frame prediction combined with transform coding, the attribute information can also be predicted and transformed based on the hierarchical order of the octree. The difference is that the process of RAHT intra-frame prediction combined with transform coding can continuously transform from the root node to the voxel level. In each RAHT transform process, the attribute information can be predicted and transformed based on a 2×2×2 block.

[0168] The structure of the encoding block of attribute information can be seen in Figure 24. The dark gray block in Figure 24 is the current block to be encoded, and the light gray block is the neighboring block coplanar and colinear with the current block. The attribute information of the current block can be normalized based on equations (21) to (23): Node =∑ p∈node attribute(p) (21) w node =∑ p∈node 1=#{p∈node} (22) a node =A node / w node (twenty three)

[0169] Specifically, we can first obtain the attribute information of the current block based on the attribute information of the nodes in the current block, that is, A node For example, a simple sum operation can be performed on the attribute information of the nodes in the current block to determine A node Then, we can use the attribute information of the current block and the number of nodes in the current block (i.e., w node ) is normalized to obtain the mean value a of the attribute information of the current block node Then, the mean value of the attribute information of the current block can be used for transform coding.

[0170] Figure 25 shows the overall process of RAHT intra-frame prediction combined with transform coding of attribute information. (d) in Figure 25 shows the attribute information of the current block, and (e) in Figure 25 shows the attribute information of the predicted block obtained by linear weighted fitting using the neighborhood attribute information of the current block. Then, the attribute information of the current block and the attribute information of the predicted block can be attribute transformed respectively to obtain DC coefficients and AC coefficients. Then, the AC coefficients can be predictively encoded. Among them, the attribute information of the predicted block is obtained by linear fitting based on the method shown in Figure 26.

[0171] FIG26 is an example diagram of a linear fitting method for the neighborhood attribute information of the current block. As shown in FIG26 , first, the 19 neighborhood blocks of the current block can be determined. Secondly, the spatial geometric distance between the neighborhood block and each sub-block in the current block can be used to perform linear weighted prediction on the attribute information of each sub-block to obtain the attribute information of the predicted block. Then, the attribute information of the predicted block can be transformed. Exemplarily, equations (24) to (26) in FIG27 can be used to predict and transform the attribute information (Equation (24) represents the transformation method of the attribute information of the current block, equation (25) represents the transformation method of the attribute information of the predicted block, and equation (26) outputs the predicted residual information):

[0172] It can also be understood that for the G-PCC codec framework, the general test conditions are as follows:

[0173] (1) There are 4 test conditions:

[0174] Condition 1: The geometric position is limited and the attributes are lost;

[0175] Condition 2: Geometric position lossless, attribute lossy;

[0176] Condition 3: Geometric position lossless, attribute loss limited;

[0177] Condition 4: Geometric position and attributes are lossless.

[0178] (2) The general test sequence includes four categories: Cat1A, Cat1B, Cat3-fused, and Cat3-frame. Among them, the Cat3-frame point cloud only contains reflectance attribute information, the Cat1A and Cat1B point clouds only contain color attribute information, and the Cat3-fused point cloud contains both color and reflectance attribute information.

[0179] (3) Technical routes: There are two types in total, which are distinguished by the algorithm used for geometric compression.

[0180] Technical route 1: Octree encoding branch.

[0181] At the encoding end, the bounding box is divided into sub-cubes in sequence, and the non-empty sub-cubes (containing points in the point cloud) are divided again until the leaf node obtained is a 1×1×1 unit cube. In the case of geometric lossless coding, the number of points contained in the leaf node needs to be encoded, and finally the geometric octree encoding is completed to generate a binary code stream.

[0182] At the decoding end, the decoding end obtains the placeholder code of each node by continuously parsing in the order of breadth-first traversal, and continuously divides the nodes in turn until a 1×1×1 unit cube is obtained. In the case of geometric lossless decoding, it is necessary to parse the number of points contained in each leaf node and finally recover the geometrically reconstructed point cloud information.

[0183] Technical route 2: prediction tree encoding branch.

[0184] On the encoding side, the prediction tree structure is established using two different methods: based on KD-Tree (high-latency slow mode) and using lidar calibration information (low-latency fast mode). Using lidar calibration information, each point can be divided into different lasers, and the prediction tree structure is established according to different lasers. Next, based on the structure of the prediction tree, each node in the prediction tree is traversed, and the geometric position information of the node is predicted by selecting different prediction modes to obtain the prediction residual, and the geometric prediction residual is quantized using the quantization parameter. Finally, through continuous iteration, the prediction residual of the prediction tree node position information, the prediction tree structure, and the quantization parameters are encoded to generate a binary code stream.

[0185] At the decoding end, the decoding end reconstructs the prediction tree structure by continuously parsing the bit stream. Secondly, the geometric position prediction residual information and quantization parameters of each prediction node are obtained through parsing, and the prediction residual is dequantized to restore the reconstructed geometric position information of each node, finally completing the geometric reconstruction at the decoding end.

[0186] The following uses the geometry-based solid content test model (GES-TM) encoding and decoding framework as an example to illustrate the RAHT-based attribute information encoding and decoding scheme in more detail.

[0187] When inter-frame prediction coding of attribute information is performed under the GES-TM codec framework, if inter-frame prediction coding is started, the RAHT attribute transform coding structure will first be constructed based on the geometric information of the current node, that is, the nodes are continuously merged at the voxel level until the root node of the entire RAHT transform tree is obtained, thereby obtaining the transform coding hierarchical structure corresponding to the attribute information. Then, according to the RAHT transform structure, the root node can be divided to obtain N child nodes of each node (N is less than or equal to 8). The difference from the RAHT intra-frame prediction combined with transform coding mode is that the RAHT inter-frame prediction combined with transform coding mode uses the node information of the reference frame. For example, the attribute information of the N child nodes of the current node can be RAHT transformed to obtain DC and AC coefficients. Secondly, the AC coefficients of the N child nodes can be inter-frame predicted in the following way.

[0188] For example, if the inter-frame prediction node of the current node is valid (ie, the co-located node of the current node in the reference frame exists), the attribute information of the prediction node is directly used as the attribute prediction value of the current node.

[0189] For another example, if the current node can find a node with exactly the same position as the current node in the cache of the reference frame (that is, the current node exists in the same node in the reference frame), then the attribute prediction values ​​of the AC coefficients of the N child nodes of the current node can be determined based on the AC coefficients of the M child nodes contained in the same node. For example, if the AC coefficient of the inter-frame prediction node corresponding to a child node is not zero, the AC coefficient of the inter-frame prediction node is directly used as the prediction value of the child node; if the AC coefficient of the inter-frame prediction node corresponding to a child node is zero, the AC coefficient of the intra-frame prediction node corresponding to the child node can be used as the prediction value.

[0190] For another example, if the inter-frame prediction node of the current node is invalid (ie, the co-located node of the current node in the reference frame does not exist), the attribute prediction value of the adjacent node in the frame can be used as the attribute prediction value of the current node.

[0191] In addition, after RAHT inter prediction is enabled, the optimal RAHT prediction mode can be selected for each layer. The RAHT prediction mode can be either RAHT intra prediction mode or RAHT inter prediction mode. If the cost of the RAHT intra prediction mode is less than the cost of the RAHT inter prediction mode, RAHT intra prediction can be performed on the current layer; otherwise, RAHT inter prediction is performed.

[0192] Under the GES-TM codec framework, the related technology can select a coding mode for each node to encode the attribute information of the RAHT layer. The following will introduce in detail the way in which the related technology selects the coding mode for the node.

[0193] In related technologies, whether inter-frame prediction is performed on the current coding unit can be determined by a syntax element (e.g., attrInterPredictionEnabled) in a high-level attribute parameter set (APS). If inter-frame prediction is enabled for the current coding unit, the coding mode for different RAHT layer attributes needs to be determined by a syntax element (e.g., rahtUpperModeLevel) in the APS.

[0194] Figure 28 is a schematic diagram of the inter-frame prediction process based on the attribute information of the RAHT layer. As shown in Figure 28, in step S2802, the node is RAHT transformed to form a plurality of RAHT layers (as shown in Figure 29). The RAHT levels of the multiple RAHT layers can be determined based on the division depth of the RAHT layer. Exemplarily, as shown in Figure 29, if the division depth of the RAHT layer is within the range of the RAHT upper layer, it means that the current layer is in the RAHT upper layer; if the division depth of the RAHT layer is within the range of the RAHT middle layer, it means that the layer is in the RAHT middle layer; if the division depth of the RAHT layer is within the range of the RAHT lower layer, it means that the current layer is in the RAHT lower layer.

[0195] In step S2804 , it is determined whether the current layer is located in the RAHT upper layer; if the current layer is located in the RAHT upper layer, step S2808 is executed; otherwise, step S2806 is executed.

[0196] In step S2806, it is determined whether the current layer is located in the RAHT lower layer; if the current layer is located in the RAHT lower layer, step S2812 is executed; otherwise, the current layer is located in the RAHT middle layer, and step S2810 is executed.

[0197] In step S2808, it is determined whether the current node has a co-located node. If the current node is in a RAHT upper layer, the coding mode of the current node is determined based on whether the current node can find a co-located node. If a co-located node exists, step S2814 is executed, that is, RAHT inter-frame prediction combined with transform coding is used; otherwise, step S2812 is executed, and RAHT intra-frame prediction combined with transform coding is used.

[0198] In step S2810, if the current node is in the RAHT middle layer, the optimal coding mode for the current node is adaptively selected from three coding modes based on a rate-distortion optimization algorithm, and the determined coding mode is ultimately transmitted to the decoder. The three coding modes include: RAHT inter-frame prediction combined with transform coding, RAHT intra-frame prediction combined with transform coding, and RAHT transform coding.

[0199] In step S2812, if the current node is in the lower RAHT layer, the coding mode of the current node is adaptively determined based on the neighborhood mode of the current node. For example, if the number of neighboring nodes of the current node is greater than a certain threshold, RAHT intra prediction combined with transform coding is used; otherwise, no prediction is performed, that is, RAHT transform coding is performed.

[0200] From the above introduction, it can be seen that in the GES-TM-based coding and decoding framework, in the process of encoding point cloud attribute information, if RAHT inter-frame prediction is enabled, the related technology will select a coding mode for each node in the RAHT layer. However, this coding method of the related technology is often a locally optimal coding method, that is, it may be the best coding mode for a node, but it may not be the best coding mode for the entire RAHT layer. For example, after the related technology determines a coding mode for each node, it needs to encode the coding modes corresponding to all nodes into the bitstream, which will increase the coding bits of the coding mode information, thereby limiting the coding and decoding efficiency of the attribute information to a certain extent. Therefore, how to improve the coding and decoding efficiency of the attribute information of the point cloud is a problem that needs to be solved.

[0201] In response to the above problems, an embodiment of the present application provides a point cloud coding method, including: determining the costs of a first coding method and a second coding method; encoding the attribute information of the current layer according to the costs of the first coding method and the second coding method; wherein the current layer is any one of multiple RAHT layers, and under the first coding method, each node in the current layer corresponds to the same coding mode, and under the second coding method, the coding modes corresponding to the nodes in the current layer need to be determined separately.

[0202] An embodiment of the present application also provides a point cloud decoding method, including: parsing a code stream to determine first identification information; decoding attribute information of a current layer based on the first identification information; wherein the current layer is any one of a plurality of RAHT layers, and the first identification information is used to indicate that the current layer uses a first decoding method or a second decoding method. Under the first decoding method, each node in the current layer corresponds to the same decoding mode, and under the second decoding method, the decoding modes corresponding to the nodes in the current layer need to be determined separately.

[0203] The reason why the RAHT-based encoding and decoding scheme provided by the related art is less efficient is that the scheme determines the encoding and decoding mode for each node in the RAHT layer. Although this scheme can make the encoding and decoding mode corresponding to each node the better (or even the best) encoding and decoding mode for the node, determining the encoding and decoding mode node by node will result in low encoding and decoding efficiency. Unlike the related art, the embodiment of the present application introduces an encoding and decoding scheme for the RAHT layer. If this encoding and decoding scheme is selected, each node in the RAHT layer will correspond to the same encoding and decoding mode. In other words, the embodiment of the present application regards the RAHT layer as a coding unit and determines the encoding and decoding mode at the RAHT layer as the granularity. Compared with the node, the RAHT layer granularity is slightly larger, but compared with the slice, the RAHT layer granularity is smaller. Therefore, the coding unit of the RAHT layer granularity can maximize the encoding and decoding efficiency of the point cloud attribute information while ensuring high encoding and decoding accuracy.

[0204] The following will describe in detail the point cloud decoding method provided in the embodiment of the present application with reference to the accompanying drawings.

[0205] Figure 30 is a flow chart of the point cloud decoding method provided in an embodiment of the present application. The decoding method of Figure 30 can be applied to a decoder. The decoding method of Figure 30 can be used to decode the attribute information of a point cloud. In some implementations, the decoding method can be applied to G-PCC. Alternatively, in other implementations, the decoding method can be applied to a geometry-based solid content test model (GES-TM). GES-TM is a coding and decoding framework proposed for dense point clouds (such as point clouds collected in augmented reality (AR) or virtual reality (VR) scenes).

[0206] 30 , in step S3010 , the code stream is parsed to determine first identification information. The first identification information is used to indicate whether the current layer is decoded using the first decoding method or the second decoding method. The first identification information may be located in an attribute brick header (ABH).

[0207] In the first decoding mode, nodes in the previous layer can be decoded based on the same decoding mode; in the second decoding mode, nodes in the current layer can be decoded based on their respective decoding modes.

[0208] The first identification information may indicate a decoding mode based on the rate-distortion cost of the first decoding mode and the second decoding mode. For example, the decoding mode may be the decoding mode with the lowest rate-distortion cost among the first decoding mode and the second decoding mode. The calculation method of the rate-distortion cost can be found in the description of the encoding side and will not be described in detail here.

[0209] In some implementations, the first identification information may be represented by attr_code_mode[i] (of course, the first identification information may also be represented by any other letters and / or numbers). For example, the first identification information may include a first value (such as 1 or true) and a second value (such as 0 or false). If the value of the first identification information is the first value, it may indicate that the front layer is decoded using the first decoding method; if the value of the first identification information is the second value, it may indicate that the front layer is decoded using the second decoding method.

[0210] The current layer may also be referred to as a RAHT layer, a RAHT decoding layer, a RAHT attribute decoding layer, or a RAHT transform layer. The current layer may be determined based on at least one splitting (or downsampling) of a previous layer of the current layer in each of the X, Y, and Z directions.

[0211] Multiple RAHT layers can be divided into multiple hierarchical ranges. For example, the multiple RAHT layers can be divided into a first hierarchical range, a second hierarchical range, and a third hierarchical range. The first hierarchical range may be the highest layer among the multiple RAHT layers and, therefore, may also be referred to as the RAHT upper layer. The second hierarchical range may be an intermediate layer among the multiple RAHT layers and, therefore, may also be referred to as the RAHT middle layer. The third hierarchical range may be the lowest layer among the multiple RAHT layers and, therefore, may also be referred to as the RAHT lower layer. The current layer may be located at any of the multiple hierarchical ranges.

[0212] In step S3020, attribute information of the current layer is decoded according to the first identification information.

[0213] As mentioned above, the first identification information may indicate whether the decoding mode of the current layer is the first decoding mode or the second decoding mode. In some implementations, if the first identification information indicates the first decoding mode, decoding the attribute information of the current layer may include: parsing the bitstream to determine the second identification information, where the second identification information is used to indicate the target decoding mode; and then decoding the attribute information of each node in the current layer according to the target decoding mode.

[0214] The target decoding mode indicated by the second identification information can be referred to as the optimal decoding mode for the current layer. The target decoding mode can be determined based on the rate-distortion costs of multiple decoding modes. For example, the target decoding mode can be the decoding mode with the lowest rate-distortion cost among the multiple decoding modes. The calculation method of the rate-distortion cost can be found in the description of the encoding side and is not described in detail here.

[0215] In some implementations, the multiple decoding modes mentioned above may include: RAHT transform decoding mode, RAHT intra-frame prediction combined with transform decoding mode, RAHT inter-frame prediction combined with transform decoding mode, a combination of RAHT intra-frame prediction combined with transform decoding mode and RAHT inter-frame prediction combined with transform decoding mode (for example, if the node to be decoded has a co-located node in the reference frame, RAHT inter-frame prediction combined with transform is performed; if there is no co-located node, RAHT intra-frame prediction combined with transform decoding is performed).

[0216] In some implementations, the second identification information may be represented by attr_pred_mode[i] (of course, the second identification information may also be represented by any other letters and / or numbers). For example, the second identification information may include a first value (such as 1 or true) and a second value (such as 0 or false). If the value of the second identification information is the first value, it may indicate that the front layer is decoded using RAHT inter-frame prediction combined with transform decoding mode; if the value of the second identification information is the second value, it may indicate that the front layer is decoded using RAHT intra-frame prediction combined with transform decoding mode.

[0217] In some implementations, if the first identification information indicates the second decoding method, then the method for decoding the attribute information of the current layer may include: determining the decoding mode of the current node in the current layer; and then, decoding the attribute information of the current node according to the decoding mode of the current node.

[0218] In the second decoding method, this embodiment of the present application determines a decoding mode for each node in the RAHT layer. If the second decoding method is inefficient, this embodiment of the present application can perform decoding based on the first decoding method. The first decoding method introduces the decoding mode corresponding to the RAHT layer. If the second decoding method is selected, all nodes in the RAHT layer will correspond to the same decoding mode, thereby improving the decoding efficiency of attribute information.

[0219] The embodiments of the present application do not specifically limit the method for determining the decoding mode of the current node in the current layer. In some implementations, the method for determining the decoding mode of the current node in the current layer is related to the hierarchical range of the current layer. As mentioned above, the current layer can be located within any hierarchical range of the RAHT upper layer, the RAHT middle layer, and the RAHT lower layer. When the current layer is in the above-mentioned different hierarchical ranges, the method for determining the decoding mode of the current node in the current layer is also different, which is described in detail below.

[0220] In some implementations, if the current layer is located in a RAHT upper layer, then determining the decoding mode of the current node in the current layer may include determining the decoding mode of the current node based on whether a reference frame contains a co-located node of the current node. For example, if a co-located node of the current node exists, then the decoding mode of the current node may be RAHT inter-frame prediction combined with transform decoding. For another example, if a co-located node of the current node does not exist, then the decoding mode of the current node may be RAHT intra-frame prediction combined with transform decoding.

[0221] In some implementations, if the current layer is in a RAHT mid-layer, then determining the decoding mode of the current node in the current layer may include: third identification information obtained by parsing the code stream indicates the decoding mode of the current node. The above decoding modes may include a RAHT transform decoding mode, a RAHT intra-frame prediction combined with transform decoding mode, and a RAHT inter-frame prediction combined with transform decoding mode. For example, the decoding mode with the lowest rate-distortion cost among the RAHT transform decoding mode, the RAHT intra-frame prediction combined with transform decoding mode, and the RAHT inter-frame prediction combined with transform decoding mode may be used as the decoding mode indicated by the third identification information.

[0222] In some implementations, if the current layer is located in the RAHT lower layer, then the method for determining the decoding mode of the current node in the current layer may include: determining the decoding mode of the current node based on the neighboring nodes of the current node. For example, if the number of neighboring nodes of the parent node of the current node is greater than a first threshold and the number of neighboring nodes of the grandparent node of the current node is greater than a second threshold, then the decoding mode of the current node is a RAHT intra-frame prediction combined with a transform decoding mode. For another example, if the number of neighboring nodes of the parent node of the current node is less than or equal to a first threshold and / or the number of neighboring nodes of the grandparent node of the current node is less than or equal to a second threshold, then the decoding mode of the current node is a RAHT transform decoding mode. As mentioned above, multiple decoding modes may include RAHT inter-frame prediction combined with transform decoding. The reference frame of the RAHT inter-frame prediction combined with transform decoding may be a reference frame that has not undergone motion compensation or a reference frame that has undergone motion compensation. Furthermore, in some implementations, a first reference frame and a second reference frame can be introduced simultaneously for RAHT inter-frame prediction combined with transform decoding. The first reference frame is a reference frame that has undergone motion compensation (which can refer to local motion compensation or global motion compensation), and the second reference frame is a reference frame that has not undergone motion compensation (which can refer to local motion compensation or global motion compensation). At the encoder, inter-frame prediction can be performed based on the first and second reference frames, and an appropriate reference frame can be selected based on the rate-distortion cost. Then, fourth identification information can be written to the bitstream to indicate the reference frame corresponding to the RAHT inter-frame prediction combined with transform decoding. At the decoder, the fourth identification information can be obtained by parsing the bitstream to determine the reference frame corresponding to the RAHT inter-frame prediction combined with transform decoding. Considering both motion-compensated and non-motion-compensated reference frames helps improve encoding and decoding performance.

[0223] In some implementations, the bitstream may be parsed to determine fourth identification information, which is used to indicate whether a reference frame after motion compensation is used for RAHT inter-frame prediction.

[0224] For example, the fourth identification information may include a first value (such as 1 or true) and a second value (such as 0 or false). If the value of the fourth identification information is the first value, it indicates that the reference frame after motion compensation can be used for RAHT inter-frame prediction; if the value of the fourth identification information is the second value, it indicates that only the reference frame without motion compensation is used for RAHT inter-frame prediction.

[0225] The fourth identification information may correspond to a slice (also referred to as a stripe). That is, the fourth identification information may be slice-level information, indicating whether a slice can use a motion-compensated reference frame for RAHT inter-frame prediction. Alternatively, the fourth identification information may correspond to a RAHT layer. That is, the fourth identification information may be RAHT layer-level information, indicating whether a RAHT layer can use a motion-compensated reference frame for RAHT inter-frame prediction.

[0226] In some implementations, if the target decoding mode is a combination of RAHT intra prediction combined with transform decoding and RAHT inter prediction combined with transform decoding, decoding the attribute information of the current layer may include: determining intra prediction values ​​and inter prediction values ​​for AC coefficients of nodes in the current layer according to the target decoding mode; fusing the intra prediction values ​​and inter prediction values ​​to determine a target prediction value for the AC coefficient; and determining reconstructed attribute information for the nodes in the current layer based on the target prediction value. Fusing the intra prediction results and inter prediction results helps improve the accuracy of the prediction information, thereby improving the encoding and decoding performance of the attribute information.

[0227] For example, the arithmetic mean of the intra-frame prediction value and the inter-frame prediction value can be used as the target prediction value.

[0228] For another example, the intra-frame prediction value and the inter-frame prediction value can be weighted and summed to determine the target prediction value. Assuming that the intra-frame prediction value of the AC coefficient of the current node (which can be any node in the current layer) is predIntraVal and the inter-frame prediction value is predInterVal, the target prediction value predVal can be calculated using formula (27): predVal = w1*predIntraVal + w2*predIntraVal (27)

[0229] Where w1 and w2 represent the weights of predIntraVal and predIntraVal respectively.

[0230] The weights of the intra-frame prediction value and the inter-frame prediction value of the AC coefficient of the current node can be determined in a variety of ways, which are not specifically limited in the embodiments of the present application. For example, the weights of the intra-frame prediction value and the inter-frame prediction value can be determined based on the number of neighboring nodes during intra-frame prediction. The greater the number of neighboring nodes during intra-frame prediction, the higher the weight of the intra-frame prediction value. Alternatively, multiple weights can be predefined for the intra-frame prediction value and the inter-frame prediction value, and then the optimal weight can be determined based on a rate-distortion optimization algorithm.

[0231] In some implementations, determining the reconstructed attribute information of a node in the current layer based on a target prediction value may include determining a reconstructed value of the AC coefficient based on a target prediction value of the AC coefficient and a prediction residual value of the AC coefficient; then, performing an inverse transform on the reconstructed value of the AC coefficient and the DC coefficient of the current node to determine the reconstructed value of the attribute information of the current node.

[0232] In some implementations, if the target decoding mode is a combination of RAHT intra-frame prediction combined with transform decoding and RAHT inter-frame prediction combined with transform decoding, decoding the attribute information of the current layer may include: determining whether the inter-frame prediction value of the AC coefficient of the node in the current layer is equal to zero; if the inter-frame prediction value is not equal to zero, the prediction value of the AC coefficient may be determined as the inter-frame prediction value; if the inter-frame prediction value is equal to zero, the prediction value of the AC coefficient of the node in the current layer may be determined as the intra-frame prediction value.

[0233] In some implementations, whether to execute step S3010 may be determined based on certain conditions. For example, it may first be determined whether the attribute information of the nodes in the current layer meets the conditions for performing inter-frame prediction. If the attribute information meets the conditions for performing inter-frame prediction, step S3010 is executed; otherwise, step S3010 may not be executed. The inter-frame prediction condition may, for example, be whether the current frame is a P frame or a B frame.

[0234] In some implementations, before determining whether the attribute information of a node in the current layer satisfies the conditions for inter-frame prediction, it may be determined whether the current layer can employ attribute prediction decoding. For example, whether the current layer can employ attribute prediction decoding may be determined based on the number of neighboring nodes in the current layer and / or the number of neighboring nodes of the parent node of the node in the current layer. Exemplarily, if the number of neighboring nodes in the current layer is greater than or equal to a third threshold and / or the number of neighboring nodes of the parent node of the node in the current layer is greater than or equal to a fourth threshold, it is determined that the current layer can employ attribute prediction decoding.

[0235] In some implementations, if the attribute information of the node in the current layer does not meet the conditions for inter-frame prediction, RAHT transform decoding and / or RAHT intra-frame prediction combined with transform decoding are performed on the attribute information.

[0236] The above describes in detail the point cloud decoding method provided by the embodiment of the present application in conjunction with Figure 30. The following describes in detail the point cloud encoding method provided by the embodiment of the present application in conjunction with Figure 31.

[0237] Figure 31 is a flow chart of the point cloud encoding method provided in an embodiment of the present application. The encoding method of Figure 31 can be applied to an encoder. The encoding method of Figure 31 can be used to encode the attribute information of a point cloud. In some implementations, the encoding method can be applied to G-PCC. Alternatively, in other implementations, the encoding method can be applied to a geometry-based solid content test model (GES-TM). GES-TM is a coding and decoding framework proposed for dense point clouds (such as point clouds collected in augmented reality (AR) or virtual reality (VR) scenes).

[0238] 31 , in step S3110 , the costs of the first encoding method and the second encoding method are determined.

[0239] The cost of the first encoding mode and the second encoding mode may refer to the rate-distortion cost of the first encoding mode and the second encoding mode. The above-mentioned rate-distortion cost can be determined based on the distortion parameter, code rate parameter and Lagrange factor corresponding to the first encoding mode (or the second encoding mode). For example, for each encoding mode, the distortion parameter can be determined based on the reconstructed attribute information and the original attribute information of each encoding mode; then, the number of encoding bits required for each encoding mode can be determined, thereby determining the code rate parameter. Then, the rate-distortion cost of each encoding mode can be determined based on the following formula (28): J = D + λxR (28)

[0240] Where J represents the rate-distortion cost, D represents the distortion parameter, R represents the bit rate parameter, and λ represents the Lagrangian factor.

[0241] The Lagrangian factor can be determined based on the quantization parameter of the attribute information of the current layer node. For example, the value of λ can be determined based on the following formula (29):

[0242] Where QP represents the quantization parameter of the attribute information. N can be a predefined value. The value of N can be associated with the type of attribute information. For example, the value of N can be different for reflectivity and color information.

[0243] In step S3120 , attribute information of the current layer is encoded according to the costs of the first encoding mode and the second encoding mode.

[0244] The current layer may also be referred to as a RAHT layer, a RAHT coding layer, a RAHT attribute coding layer, or a RAHT transform layer. The current layer may be determined based on at least one splitting (or downsampling) of a previous layer of the current layer along the X direction, the Y direction, and the Z direction.

[0245] Multiple RAHT layers can be divided into multiple hierarchical ranges. For example, the multiple RAHT layers can be divided into a first hierarchical range, a second hierarchical range, and a third hierarchical range. The first hierarchical range may be the highest layer among the multiple RAHT layers and, therefore, may also be referred to as the RAHT upper layer. The second hierarchical range may be an intermediate layer among the multiple RAHT layers and, therefore, may also be referred to as the RAHT middle layer. The third hierarchical range may be the lowest layer among the multiple RAHT layers and, therefore, may also be referred to as the RAHT lower layer. The current layer may be located at any of the multiple hierarchical ranges.

[0246] In the first coding mode, nodes in the previous layer can be encoded based on the same coding mode; in the second coding mode, nodes in the current layer can be encoded based on their respective coding modes.

[0247] In step S3120, a coding mode for the current layer may be determined based on a rate-distortion cost when encoding the current layer using the first coding mode (or the second coding mode). For example, if the rate-distortion cost when encoding the current layer using the first coding mode is less than the rate-distortion cost when encoding the current layer using the second coding mode, the first coding mode is selected as the coding mode for the current layer.

[0248] In the second coding mode, the embodiment of the present application determines a coding mode for each node in the RAHT layer. If the efficiency of the second coding mode is low, the embodiment of the present application can encode based on the first coding mode. The first coding mode introduces the coding mode corresponding to the RAHT layer. If the second coding mode is selected, the nodes in the RAHT layer will correspond to the same coding mode, thereby improving the coding efficiency of the attribute information.

[0249] In some implementations, the first identification information may be written into the bitstream. The first identification information is used to indicate whether the current layer is encoded based on the first encoding method or the second encoding method. The first identification information may be located in an attribute brick header (ABH).

[0250] In some implementations, the first identification information may be represented by attr_code_mode[i] (of course, the first identification information may also be represented by any other letters and / or numbers). For example, the first identification information may include a first value (such as 1 or true) and a second value (such as 0 or false). If the value of the first identification information is the first value, it may indicate that the front layer is encoded using the first encoding method; if the value of the first identification information is the second value, it may indicate that the front layer is encoded using the second encoding method.

[0251] As mentioned above, in step S3120, the coding mode for the current layer can be determined based on the rate-distortion cost. In some implementations, if the rate-distortion cost of the first coding mode is less than the rate-distortion cost of the second coding mode, the coding mode for the current layer is the first coding mode. Under the first coding mode, a target coding mode can be determined from multiple coding modes; then, the attribute information of each node in the current layer is encoded according to the target coding mode.

[0252] The various coding modes mentioned above may include: RAHT transform coding mode, RAHT intra-frame prediction combined with transform coding mode, RAHT inter-frame prediction combined with transform coding mode, a combination of RAHT intra-frame prediction combined with transform coding mode and RAHT inter-frame prediction combined with transform coding mode (for example, if the node to be coded has a co-located node in the reference frame, RAHT inter-frame prediction combined with transform is performed; if there is no co-located node, RAHT intra-frame prediction combined with transform coding is performed).

[0253] In some implementations, the second identification information may be written into the bitstream. The second identification information is used to indicate the target coding mode.

[0254] In some implementations, the second identification information may be represented by attr_pred_mode[i] (of course, the second identification information may also be represented by any other letters and / or numbers). For example, the second identification information may include a first value (such as 1 or true) and a second value (such as 0 or false). If the value of the second identification information is the first value, it may indicate that the front layer is encoded using RAHT inter-frame prediction combined with transform coding mode; if the value of the second identification information is the second value, it may indicate that the front layer is encoded using RAHT intra-frame prediction combined with transform coding mode.

[0255] In some implementations, in step S3120, if the rate-distortion cost of the second encoding mode is less than the rate-distortion cost of the first encoding mode, then the encoding mode of the current layer is the second encoding mode; then, the attribute information of the current node is encoded according to the encoding mode of the current node.

[0256] The embodiment of the present application does not specifically limit the method for determining the coding mode of the current node in the current layer. In some implementations, the method for determining the coding mode of the current node in the current layer is related to the hierarchical range of the current layer. As mentioned above, the current layer can be located within any hierarchical range of the RAHT upper layer, the RAHT middle layer, and the RAHT lower layer. When the current layer is in the above-mentioned different hierarchical ranges, the method for determining the coding mode of the current node in the current layer is also different, which is described in detail below.

[0257] In some implementations, if the current layer is located in a RAHT upper layer, determining the coding mode of the current node in the current layer may include determining the coding mode of the current node based on whether a reference frame contains a co-located node of the current node. For example, if a co-located node of the current node exists, the coding mode of the current node may be RAHT inter-frame prediction combined with transform coding. For another example, if a co-located node of the current node does not exist, the coding mode of the current node may be RAHT intra-frame prediction combined with transform coding.

[0258] In some implementations, if the current layer is located in the RAHT middle layer, then the method for determining the coding mode of the current node in the current layer may include: determining the coding mode of the current node in multiple coding modes based on rate-distortion cost. For example, the current node can be encoded based on multiple coding modes, and the coding mode with the smallest rate-distortion cost is used as the coding mode of the current node. The multiple coding modes mentioned here may include RAHT transform coding mode, RAHT intra-frame prediction combined with transform coding mode, and RAHT inter-frame prediction combined with transform coding mode. Furthermore, third identification information can be written into the code stream. The above-mentioned third identification information is used to indicate the coding mode of the current node.

[0259] In some implementations, if the current layer is located in a RAHT lower layer, then the method for determining the coding mode of the current node in the current layer may include: determining the coding mode of the current node based on the neighboring nodes of the current node. For example, if the number of neighboring nodes of the parent node of the current node is greater than a first threshold and the number of neighboring nodes of the grandparent node of the current node is greater than a second threshold, then the coding mode of the current node is a RAHT intra-frame prediction combined with a transform coding mode. For another example, if the number of neighboring nodes of the parent node of the current node is less than or equal to a first threshold and / or the number of neighboring nodes of the grandparent node of the current node is less than or equal to a second threshold, then the coding mode of the current node is a RAHT transform coding mode. As mentioned above, multiple coding modes may include RAHT inter-frame prediction combined with transform coding. The reference frame of the RAHT inter-frame prediction combined with transform coding may be a reference frame that has not undergone motion compensation or a reference frame that has undergone motion compensation. Furthermore, in some implementations, a first reference frame and a second reference frame can be simultaneously introduced for RAHT inter-frame prediction combined with transform coding. The first reference frame is a reference frame that has undergone motion compensation (which can refer to local motion compensation or global motion compensation), and the second reference frame is a reference frame that has not undergone motion compensation (which can refer to local motion compensation or global motion compensation). At the encoder, inter-frame prediction can be performed based on the first and second reference frames, and an appropriate reference frame can be selected based on the rate-distortion cost. Then, fourth identification information can be written into the bitstream to indicate the reference frame corresponding to the RAHT inter-frame prediction combined with transform coding. At the decoder, the fourth identification information can be obtained by parsing the bitstream to determine the reference frame corresponding to the RAHT inter-frame prediction combined with transform decoding. Considering both motion-compensated and non-motion-compensated reference frames helps improve encoding and decoding performance.

[0260] In some implementations, fourth identification information may be written into the bitstream, where the fourth identification information is used to indicate whether a reference frame after motion compensation will be used for RAHT inter-frame prediction.

[0261] For example, the fourth identification information may include a first value (such as 1 or true) and a second value (such as 0 or false). If the value of the fourth identification information is the first value, it indicates that the reference frame after motion compensation can be used for RAHT inter-frame prediction; if the value of the fourth identification information is the second value, it indicates that only the reference frame without motion compensation is used for RAHT inter-frame prediction.

[0262] The fourth identification information may correspond to a slice (also referred to as a stripe). That is, the fourth identification information may be slice-level information, indicating whether a slice can use a motion-compensated reference frame for RAHT inter-frame prediction. Alternatively, the fourth identification information may correspond to a RAHT layer. That is, the fourth identification information may be RAHT layer-level information, indicating whether a RAHT layer can use a motion-compensated reference frame for RAHT inter-frame prediction.

[0263] In some implementations, if the target coding mode is a combination of RAHT intra-frame prediction combined with transform coding and RAHT inter-frame prediction combined with transform coding, encoding the attribute information of the current layer may include: determining intra-frame prediction values ​​and inter-frame prediction values ​​for AC coefficients of nodes in the current layer based on the target coding mode; fusing the intra-frame prediction values ​​and inter-frame prediction values ​​to determine a target prediction value for the AC coefficient; and determining reconstructed attribute information for the nodes in the current layer based on the target prediction value. Fusing the intra-frame prediction results and the inter-frame prediction results helps improve the accuracy of the prediction information, thereby improving the encoding and decoding performance of the attribute information.

[0264] For example, the arithmetic mean of the intra-frame prediction value and the inter-frame prediction value can be used as the target prediction value.

[0265] For another example, the intra-frame prediction value and the inter-frame prediction value can be weighted and summed to determine the target prediction value. Assuming that the intra-frame prediction value of the AC coefficient of the current node (which can be any node in the current layer) is predIntraVal and the inter-frame prediction value is predInterVal, the target prediction value predVal can be calculated using formula (30): predVal = w1*predIntraVal + w2*predIntraVal (30)

[0266] Where w1 and w2 represent the weights of predIntraVal and predIntraVal respectively.

[0267] The weights of the intra-frame prediction value and the inter-frame prediction value of the AC coefficient of the current node can be determined in a variety of ways, which are not specifically limited in the embodiments of the present application. For example, the weights of the intra-frame prediction value and the inter-frame prediction value can be determined based on the number of neighboring nodes during intra-frame prediction. The greater the number of neighboring nodes during intra-frame prediction, the higher the weight of the intra-frame prediction value. Alternatively, multiple weights can be predefined for the intra-frame prediction value and the inter-frame prediction value, and then the optimal weight can be determined based on a rate-distortion optimization algorithm.

[0268] In some implementations, determining the reconstructed attribute information of a node in the current layer based on a target prediction value may include determining a reconstructed value of the AC coefficient based on a target prediction value of the AC coefficient and a prediction residual value of the AC coefficient; then, performing an inverse transform on the reconstructed value of the AC coefficient and the DC coefficient of the current node to determine the reconstructed value of the attribute information of the current node.

[0269] In some implementations, if the target coding mode is a combination of RAHT intra-frame prediction combined with transform coding and RAHT inter-frame prediction combined with transform coding, encoding the attribute information of the current layer may include: determining whether the inter-frame prediction value of the AC coefficient of the node in the current layer is equal to zero; if the inter-frame prediction value is not equal to zero, the prediction value of the AC coefficient may be determined as the inter-frame prediction value; if the inter-frame prediction value is equal to zero, the prediction value of the AC coefficient of the node in the current layer may be determined as the intra-frame prediction value.

[0270] In some implementations, whether to execute step S3110 may be determined based on certain conditions. For example, it may first be determined whether the attribute information of the nodes in the current layer meets the conditions for performing inter-frame prediction. If the attribute information meets the conditions for performing inter-frame prediction, step S3110 is executed; otherwise, step S3110 may not be executed. The inter-frame prediction condition may, for example, be whether the current frame is a P frame or a B frame.

[0271] In some implementations, before determining whether the attribute information of a node in the current layer satisfies the conditions for inter-frame prediction, it may be determined whether the current layer can employ attribute prediction coding. For example, whether the current layer can employ attribute prediction coding may be determined based on the number of neighboring nodes in the current layer and / or the number of neighboring nodes of the parent node of the node in the current layer. Exemplarily, if the number of neighboring nodes in the current layer is greater than or equal to a third threshold and / or the number of neighboring nodes of the parent node of the node in the current layer is greater than or equal to a fourth threshold, it is determined that the current layer can employ attribute prediction coding.

[0272] In some implementations, if the attribute information of the node in the current layer does not meet the conditions for inter-frame prediction, RAHT transform coding and / or RAHT intra-frame prediction combined with transform coding are performed on the attribute information.

[0273] The following examples are used to describe the embodiments of the present application in more detail. It should be noted that the examples below are only intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. It is apparent that those skilled in the art can make various equivalent modifications or changes based on the examples given below, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0274] In the GES-TM encoding and decoding framework, when encoding point cloud attribute information, if RAHT inter-frame prediction is enabled, the related technology will select a coding mode for each node in the RAHT layer and encode all coding modes into the bitstream. However, this encoding method of the related technology is often a locally optimal coding method, that is, it may be the best coding mode for a node, but it may not be the best coding mode for the entire RAHT layer. For example, after determining a coding mode for each node, the related technology needs to encode the coding modes corresponding to all nodes into the bitstream, which will increase the coding bits of the coding mode information, thereby limiting the encoding and decoding efficiency of the attribute information to a certain extent.

[0275] To address the above issues, this example proposes a method based on RAHT layers as coding units, selecting the best coding mode layer by layer: RAHT layer coding or node coding. First, the following definitions are given:

[0276] RAHT layer coding (first coding mode): taking the RAHT layer as a coding unit, all nodes in the coding unit only need to encode one coding mode, for example, inter-frame prediction coding or intra-frame prediction coding.

[0277] Node coding (second coding mode): A different coding mode is selected for each node in the current RAHT layer, such as intra-frame prediction coding, inter-frame prediction coding, or no prediction coding. That is, each node needs to be coded in one mode.

[0278] After giving the above definition, for each RAHT coding layer, this example uses the RAHT layer as the coding unit, and the encoder selects the adaptive mode layer by layer, and finally passes the coding mode of the RAHT layer unit to the decoder. The decoder uses the decoding mode of the layer to be decoded to reconstruct and restore the attribute information of the current RAHT layer, which can ultimately improve the attribute information coding efficiency of the point cloud to a certain extent.

[0279] In this example, after obtaining different RAHT layers, adaptive mode selection is performed for each coding unit, taking the RAHT layer as a unit: the first coding mode or the second coding mode. RAHT layers can be divided into three categories: RAHT upper layer, RAHT middle layer, and RAHT lower layer. The coding schemes for the three RAHT layer attributes have been mentioned before and will not be repeated here. The coding method of this example is as follows:

[0280] When in the RAHT coding upper layer: (1) The first coding method uses the current RAHT layer as the coding unit and selects the optimal coding mode of the current RAHT layer: inter-frame prediction or intra-frame prediction, that is, all nodes in the entire RAHT layer correspond to one coding mode; (2) The second coding method adaptively selects the coding mode according to the geometric information of the node: inter-frame prediction or intra-frame prediction coding.

[0281] When in the RAHT coding middle layer: (1) The first coding method uses the current RAHT layer as the coding unit and selects the optimal coding mode of the current RAHT layer: inter-frame prediction or intra-frame prediction, that is, all nodes in the entire RAHT layer correspond to one mode; (2) The second coding method needs to adaptively select and encode a mode for each node: inter-frame prediction, intra-frame prediction, and no prediction, and encode the final selected mode node by node.

[0282] When in the RAHT coding lower layer: (1) The first coding method uses the current RAHT layer as the coding unit and selects the optimal coding mode for the current RAHT layer: inter-frame prediction or intra-frame prediction, that is, all nodes in the entire RAHT layer correspond to one mode; (2) The second coding method adaptively determines whether the current node uses intra-frame prediction coding or no prediction coding based on the number of neighboring nodes of the node.

[0283] When encoding the attribute information of three different RAHT layers, the attribute information is pre-encoded and reconstructed according to different prediction modes to obtain the comprehensive cost of each mode. Finally, the optimal coding mode for the current layer is obtained using the rate-distortion optimization algorithm, and the optimal coding mode is passed to the decoder. The decoder uses the predicted decoding mode obtained by analysis to reconstruct and restore the attribute information of the current layer point to be decoded. Among them, in the rate-distortion optimization algorithm, the distortion D between the reconstructed attribute and the original attribute of each prediction mode is first calculated. Secondly, the code stream R required for encoding of each prediction mode is obtained. The rate-distortion cost is calculated as follows:

[0284] J=D+λxR(31)

[0285] Where J represents the rate-distortion cost, D represents the distortion parameter, R represents the bit rate parameter, and λ represents the Lagrangian factor.

[0286] The Lagrangian factor can be determined based on the quantization parameter of the attribute information of the current layer node. For example, the value of λ can be determined based on the following formula (32):

[0287] Where QP represents the quantization parameter of the attribute information. N can be a predefined value. The value of N can be associated with the type of attribute information. For example, the value of N can be different for reflectivity and color information.

[0288] The following describes the method flow of this example in detail from the encoding end and the decoding end, respectively, with reference to Figures 32 to 35. Figure 32 corresponds to the encoding process of this example under the first encoding mode, Figure 33 corresponds to the encoding process of this example under the second encoding mode, Figure 34 corresponds to the decoding process of this example under the first decoding mode, and Figure 35 corresponds to the decoding process of this example under the second decoding mode.

[0289] FIG32 corresponds to the encoding process of this example under the first encoding mode. As shown in FIG32 , in step S3202 , RAHT transformation is performed on the node.

[0290] In step S3204 , it is determined whether the partition depth of the current layer is less than the maximum partition depth of the RAHT layer; if it is less than the maximum partition depth of the RAHT layer, step S3206 is executed.

[0291] In step S3206, determine whether the cost of the first encoding method is less than the cost of the second encoding method; if it is less than the cost of the second encoding method, then execute step S3208, otherwise, execute the encoding method in Figure 33.

[0292] Before executing step S3206, it can be adaptively determined whether the nodes in the current layer can use attribute prediction based on the number of neighboring nodes in the current layer and the number of neighboring nodes of the parent node; if the nodes in the current layer can use attribute prediction, then execute step S3206.

[0293] In step S3208, it is determined whether the inter-frame prediction cost is less than the intra-frame prediction cost; if the inter-frame prediction cost is less than the intra-frame prediction cost, then step S3210 is executed, otherwise, step S3214 is executed.

[0294] In step S3210, the residual of the AC coefficients is encoded based on the inter-frame prediction coding mode.

[0295] In step S3212, the value of the first identification information is set to 1. That is, it means that the attribute information of the current layer is encoded based on the inter-frame prediction coding mode.

[0296] In step S3214, the residual of the AC coefficients is encoded based on the intra-frame prediction coding mode.

[0297] In step S3216, the value of the first identification information is set to 0. This means that the attribute information of the current layer is encoded based on the intra-frame prediction coding mode.

[0298] Figure 33 corresponds to the encoding process of this example under the second encoding mode. As shown in Figure 33, the first three steps in Figure 33 are the same as the first three steps in Figure 32, and will not be repeated here.

[0299] In step S3308, it is determined whether the node is located in the upper layer of RAHT; if it is located in the upper layer of RAHT, step S3312 is executed, otherwise step S3310 is executed.

[0300] In step S3310, determine whether the node is located in the lower layer of RAHT; if it is located in the lower layer of RAHT, execute step S3314, otherwise execute step S3318.

[0301] In step S3312, it is determined whether the current node has a co-node; if so, step S3316 is executed. Otherwise, step S3314 is executed.

[0302] In step S3314, intra-frame prediction or no prediction is performed on the current node. Intra-frame prediction coding or no prediction coding (transform coding) is performed on the current node.

[0303] In step S3316, inter-frame prediction is performed on the current node.

[0304] In step S3318, determine whether the intra-frame prediction cost is less than the inter-frame prediction cost; if the intra-frame prediction cost is less than the inter-frame prediction cost, execute step S3322, otherwise execute step S3320.

[0305] In step S3320, it is determined whether the inter-frame prediction cost is less than the non-prediction cost; if the inter-frame prediction cost is less than the non-prediction cost, step S3324 is executed, otherwise step S3328 is executed.

[0306] In step S3322, determine whether the intra-frame prediction cost is less than the non-prediction cost; if the intra-frame prediction cost is less than the non-prediction cost, then execute step S3326, otherwise, execute step S3326.

[0307] In step S3324, the residual of the AC coefficients is encoded based on the inter-frame prediction coding mode.

[0308] In step S3326, the residual of the AC coefficient is encoded based on the intra-frame prediction coding mode.

[0309] In step S3328, the AC coefficients are encoded based on the transform coding mode.

[0310] In the encoding method of this example, if the nodes of the current layer can adopt attribute prediction and can perform attribute inter-frame prediction, the rate-distortion optimization algorithm is introduced for the current layer. By encoding each node of the current layer, the cost corresponding to each prediction coding mode (inter-frame prediction coding combined with transformation or intra-frame prediction combined with transformation) is calculated; the encoding mode of the current RAHT layer is adaptively determined according to whether the current layer is in the RAHT coding upper layer, RAHT coding middle layer, and RAHT coding lower layer, and the comprehensive costs under different coding modes are calculated; finally, the comprehensive costs of different coding modes are compared, and the best coding mode is selected as the coding mode of the current RAHT layer.

[0311] FIG34 corresponds to the decoding process of this example under the first decoding mode. As shown in FIG34 , in step S3402 , RAHT transformation is performed on the node.

[0312] In step S3404, it is determined whether it is less than the maximum partition depth; if it is less than the maximum partition depth, step S3406 is executed.

[0313] In step S3406, it is determined whether the decoding method is the first decoding method; if so, step S3408 is executed, otherwise the decoding method in Figure 35 is executed. For example, whether the first decoding method is the first decoding method can be confirmed by the indication of the first identification information.

[0314] Before executing step S3406, it is adaptively determined whether the nodes in the current layer can use attribute prediction based on the number of neighboring nodes in the current layer and the number of neighboring nodes of the parent node; if the nodes in the previous layer can use attribute prediction, step S3406 is executed.

[0315] If the nodes in the current layer can use attribute prediction and attribute inter-frame prediction, the optimal prediction decoding mode for the current layer is obtained based on attr_code_mode[i]. When attr_code_mode[i] is 1, the RAHT layer decoding scheme is used to decode the node attribute information of the current layer; otherwise, the existing GES-TM decoding scheme is used to decode the attribute information of the nodes in the current layer.

[0316] When the RAHT layer decoding scheme is used, it is necessary to continue parsing to obtain the current RAHT prediction decoding mode, intra-frame prediction combined with transform or inter-frame prediction combined with transform. For example, when attr_pred_mode[i] is 1, it specifies that the current RAHT layer uses the inter-frame prediction combined with transform scheme to reconstruct and restore the attribute information of the current layer node; for example, when attr_pred_mode[i] is 0, it specifies that the current RAHT layer uses the intra-frame prediction combined with transform scheme to reconstruct and restore the attribute information of the current layer node.

[0317] When the existing GEST-TM decoding scheme is used, the attribute information of the current layer node will be reconstructed and restored based on the existing GES-TM decoding scheme.

[0318] In step S3408, it is confirmed whether the prediction mode is intra-frame prediction; if it is intra-frame prediction, step S3410 is executed; otherwise, step S3412 is executed.

[0319] In step S3410 , the residual of the AC coefficients is decoded based on the intra prediction decoding mode.

[0320] In step S3412, the residual of the AC coefficients is decoded based on the inter-frame prediction decoding mode.

[0321] Figure 35 corresponds to the decoding process of this example under the second decoding mode. As shown in Figure 35, the first three steps in Figure 35 (step S3502, step S3504 and step S3506) are the same as the first three steps in Figure 34 and will not be repeated here.

[0322] In step S3508, it is confirmed whether the node is located in the upper layer of RAHT; if it is located in the upper layer of RAHT, step S3512 is executed; otherwise, step S3510 is executed.

[0323] In step S3510 , it is determined whether the node is located in the lower layer of the RAHT; if so, step S3516 is executed; otherwise, step S3514 is executed.

[0324] In step S3512, confirm whether there is a co-located node for the current node; if there is a co-located node, then execute step S3518, otherwise execute step S3516.

[0325] In step S3514, the prediction mode (inter / intra / no prediction) is identified. For example, the prediction mode may be determined by the indication of the second identification information.

[0326] In step S3516, intra-frame prediction or no prediction is performed on the current node.

[0327] In step S3518, inter-frame prediction is performed on the current node.

[0328] In this example, when performing inter-frame prediction encoding and decoding on attribute information, the point cloud information is divided into different RAHT layers, and the RAHT layer is used as the coding unit to select a mode: RAHT layer coding mode or original GES-TM coding mode, and finally the coding mode is passed to the decoding end. The decoding end uses the decoding mode of the RAHT layer and the corresponding prediction decoding scheme to reconstruct and restore the attribute information of the current layer node. In this scheme, the focus is on introducing a coding mode in each RAHT coding layer, obtaining the best coding mode by utilizing the rate-distortion optimization selection algorithm at the encoding end, and then using the decoding mode to reconstruct the attributes of the point cloud at the decoding end. Currently, the coding mode of each layer is stored in ABH, and the decoding end uses ABH to obtain the decoding mode of the RAHT coding layer. There is no restriction on how this parameter is encoded.

[0329] This embodiment of the present application uses RAHT layers as coding units when performing inter-frame predictive encoding and decoding of attribute information. Two coding modes are introduced for each RAHT layer: the first coding mode and the second coding mode. The optimal coding mode is selected for predictive coding based on rate-distortion optimization, thereby improving the coding efficiency of point cloud attributes. The coding efficiency of this embodiment of the present application is demonstrated below.

[0330] Table 2: Coding performance test results under lossless geometry and lossy attributes conditions

[0331] Table 3: Coding performance test results under lossless geometry and lossy attributes conditions

[0332] Table 4: Coding performance test results under lossless geometry and lossless attributes conditions

[0333] Table 5: Coding performance test results under lossless geometry and lossy attributes conditions

[0334] Table 6: Coding performance test results under lossless geometry and lossy attributes conditions

[0335] Table 7: Coding performance test results under lossless geometry and lossless attributes conditions

[0336] Table 8: Coding performance test results under lossy geometry and lossy attributes conditions

[0337] Luma in the table above represents brightness, and Cb and Cr represent chrominance. The test results in the table above show that for some sequences, the encoding efficiency of attribute information is significantly improved. In particular, under the C2 condition, the luma component performance of the queen sequence is improved by about 31%, significantly improving the encoding efficiency of point cloud attributes.

[0338] Syntax elements in the attribute data unit header (attribute data unit header syntax):

[0339] The 11th to 5th last rows in the above table are the syntax elements newly introduced in the embodiments of the present application. Among them, the syntax element attr_code_mode_cnt specifies the number of RAHT coding layer modes when inter-frame prediction coding is turned on for the current slice, and the default value is zero. The syntax element attr_code_mode[i] specifies the mode of each RAHT coding layer when inter-frame prediction coding is turned on for the current slice, that is, RAHT layer coding or original coding mode. A value of 1 indicates that the current layer adopts the RAHT layer coding scheme; a value of 0 indicates that the current layer adopts the existing GES-TM coding scheme. The syntax element attr_pred_mode[i] specifies the prediction decoding mode of the current RAHT layer, that is, inter-frame prediction combined with transform or intra-frame prediction combined with transform. A value of 1 indicates that the current layer adopts the inter-frame prediction combined with transform scheme; a value of 0 indicates that the current layer adopts the intra-frame prediction combined with transform scheme.

[0340] (1) The embodiment of the present application can further modify the attribute inter-frame prediction mode.

[0341] In one embodiment, a coding mode is introduced at the encoder end for the current RAHT coding layer by combining three existing prediction coding modes: inter-frame prediction coding mode combined with intra-frame prediction coding mode, intra-frame prediction coding mode, and transform coding mode. This mode indicates which prediction coding mode is used to recover the AC coefficients of the current RAHT coding layer. In another embodiment, the prediction coding mode can be further modified to include: inter-frame prediction coding mode + transform coding, intra-frame prediction coding mode + transform coding, and transform coding mode. The optimal coding mode for the current layer is determined in the same manner as the main scheme. The decoder also recovers the AC coefficients of the current layer based on the prediction coding mode of the current layer, thus completing the entire RAHT attribute coding.

[0342] (2) The embodiment of the present application can further modify the attribute prediction model.

[0343] In one embodiment, for any prediction coding mode, it is first determined whether the inter-frame attribute prediction value is equal to zero. If it is not equal to zero, the current prediction value will be directly used as the prediction value of the AC coefficient of the current node. Otherwise, the AC coefficient obtained by intra-frame prediction will be used as the AC coefficient prediction value of the current node. In another embodiment, the inter-frame prediction values ​​and intra-frame prediction values ​​of different RAHT transformation layers can be merged to finally obtain the best prediction value according to different weights, thereby further improving the RAHT coding efficiency of point cloud attributes. The specific prediction coding scheme is as follows: Assuming that the RAHT intra-frame prediction value of the current node is predIntraVal and the inter-frame prediction value is predInterVal, the final prediction value is predVal: predVal=w1*predIntraVal+w2*predIntraVal (33)

[0344] (3) The embodiment of the present application can further modify the attribute prediction model.

[0345] In one embodiment, a coding mode is introduced at the encoding end for the current RAHT coding layer to represent which prediction coding mode is used to restore the AC coefficients of the current RAHT coding layer by combining the three existing prediction coding modes: inter-frame prediction coding mode combined with intra-frame prediction coding mode, intra-frame prediction coding mode, and transform coding mode. In another embodiment, the prediction coding mode can be further changed to: by adding a coding mode at the slice level: inter-frame prediction coding is performed on the point cloud attributes of the current frame based on the reference frame point cloud after local motion compensation and after non-local motion compensation. The encoding end uses the RDO rate-distortion optimization algorithm to adaptively select the best coding mode, and the decoding end uses the slice-level decoding mode to decode the attribute information of the current slice.

[0346] The method embodiment of the present application is described in detail above in conjunction with Figures 1 to 35 , and the device embodiment of the present application is described in detail below in conjunction with Figures 36 to 39 . It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, and therefore, for portions not described in detail, reference can be made to the above method embodiment.

[0347] Figure 36 is a schematic diagram of the structure of a decoder provided by an embodiment of the present application. As shown in Figure 36, the decoder 3600 may include a first decoding unit and a second decoding unit.

[0348] The first decoding unit 3610 is configured to parse the code stream and determine the first identification information.

[0349] The second decoding unit 3620 is configured to decode the attribute information of the current layer according to the first identification information.

[0350] The current layer is any one of multiple RAHT layers, and the first identification information is used to indicate that the current layer uses a first decoding method or a second decoding method. Under the first decoding method, each node in the current layer corresponds to the same decoding mode. Under the second decoding method, the decoding modes corresponding to the nodes in the current layer need to be determined separately.

[0351] In some implementations, the second decoding unit 3620 is configured to: if the first identification information indicates the first decoding mode, parse the code stream to determine the second identification information, where the second identification information is used to indicate the target decoding mode; decode the attribute information of each node in the current layer according to the target decoding mode; wherein the target decoding mode is one of the following: RAHT transform decoding mode; RAHT intra-frame prediction combined with transform decoding mode; RAHT inter-frame prediction combined with transform decoding mode; a combination of RAHT intra-frame prediction combined with transform decoding mode and RAHT inter-frame prediction combined with transform decoding mode.

[0352] In some implementations, the second decoding unit 3620 is configured to: determine the decoding mode of the current node in the current layer if the first identification information indicates the second decoding mode; and decode the attribute information of the current node according to the decoding mode of the current node.

[0353] In some implementations, the second decoding unit 3620 is configured to: if the current layer is located in the first level range of the multiple RAHT layers, determine the decoding mode of the current node based on whether the reference frame includes a co-located node of the current node.

[0354] In some implementations, the second decoding unit 3620 is configured as follows: if the reference frame includes the co-located node, the decoding mode of the current node is a RAHT inter-frame prediction combined with a transform decoding mode; and / or if the reference frame does not include the co-located node, the decoding mode of the current node is a RAHT intra-frame prediction combined with a transform decoding mode.

[0355] In some implementations, the RAHT layer within the first level range is located at the highest layer among the multiple RAHT layers.

[0356] In some implementations, the second decoding unit 3620 is configured to: if the current layer is located in the second level range of the multiple RAHT layers, parse third identification information, where the third identification information is used to indicate a decoding mode of the current node.

[0357] In some implementations, the decoding mode indicated by the third identification information is one of the following: RAHT transform decoding mode; RAHT intra-frame prediction combined with transform decoding mode; RAHT inter-frame prediction combined with transform decoding mode.

[0358] In some implementations, the RAHT layer within the second level range is located in a middle layer among the multiple RAHT layers.

[0359] In some implementations, the second decoding unit 3620 is configured to: if the current layer is located in the third level range of the multiple RAHT layers, determine the decoding mode of the current node based on the neighboring nodes of the current node.

[0360] In some implementations, the second decoding unit 3620 is configured to: if the number of neighboring nodes of the parent node of the current node is greater than a first threshold and the number of neighboring nodes of the grandparent node of the current node is greater than a second threshold, then the decoding mode of the current node is RAHT intra prediction combined with transform decoding mode; and / or if the number of neighboring nodes of the parent node of the current node is less than or equal to a first threshold and / or the number of neighboring nodes of the grandparent node of the current node is less than or equal to a second threshold, then the decoding mode of the current node is RAHT transform decoding mode. In some implementations, the RAHT layer within the third hierarchical range is located at the lowest layer of the multiple RAHT layers.

[0361] In some implementations, the first identification information is located in an attribute block header.

[0362] In some implementations, the second decoding unit 3620 is configured to: determine the intra-frame prediction value and the inter-frame prediction value of the AC coefficient of the current node in the current layer; fuse the intra-frame prediction value and the inter-frame prediction value to determine the target prediction value of the AC coefficient; and determine the reconstruction attribute information of the current node based on the target prediction value.

[0363] In some implementations, the second decoding unit 3620 is configured to: determine the reconstructed value of the AC coefficient based on the target prediction value of the AC coefficient and the prediction residual value of the AC coefficient; and perform inverse transformation on the reconstructed value of the AC coefficient and the DC coefficient of the current node to determine the reconstructed value of the attribute information of the current node.

[0364] In some implementations, the decoder 3600 further includes a third decoding unit 3630 configured to parse the bitstream and determine fourth identification information, where the fourth identification information is used to indicate whether to use a motion-compensated reference frame for RAHT inter-frame prediction.

[0365] In some implementations, the fourth identification information is identification information corresponding to a slice; or, the fourth identification information is identification information corresponding to a RAHT layer.

[0366] In some implementations, the motion compensation is local motion compensation or global motion compensation.

[0367] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0368] If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0369] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 3600. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the decoding method described in any one of the aforementioned embodiments.

[0370] Based on the composition of the above-mentioned decoder 3600 and the computer-readable storage medium, refer to Figure 37, which shows a specific hardware structure diagram of the encoder 3700 provided in an embodiment of the present application. As shown in Figure 37, the encoder 3700 may include: a communication interface 3710, a memory 3720 and a processor 3730; each component is coupled together through a bus system 3740. It can be understood that the bus system 3740 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 3740 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as bus system 3740 in Figure 37. Among them,

[0371] Communication interface 3710, used for sending and receiving signals when sending and receiving information with other external network elements;

[0372] Memory 3720, for storing computer programs;

[0373] The processor 3730 is configured to, when running the computer program, execute:

[0374] Parse a bitstream to determine first identification information; and decode attribute information of a current layer based on the first identification information. The current layer is any one of a plurality of RAHT layers, and the first identification information is used to indicate whether the current layer uses a first decoding mode or a second decoding mode. Under the first decoding mode, each node in the current layer corresponds to the same decoding mode. Under the second decoding mode, each node in the current layer needs to determine its own decoding mode.

[0375] It is understood that the memory 3720 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM). The memory 3720 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0376] Processor 3730 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by hardware integrated logic circuits or software instructions in processor 3730. The above processor 3730 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 3720, and the processor 3730 reads the information in the memory 3720 and completes the steps of the above method in combination with its hardware.

[0377] It is understood that the embodiments described herein can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein or a combination thereof. For software implementation, the technology described herein can be implemented by a module (such as a process, a function, etc.) that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in a processor or outside a processor.

[0378] Optionally, as another embodiment, the processor 3730 is further configured to execute the decoding method described in any one of the aforementioned embodiments when running the computer program.

[0379] FIG38 is a schematic diagram of the structure of an encoder provided by an embodiment of the present application. As shown in FIG38 , the encoder 3800 includes a first encoding unit 3810 and a second encoding unit 3820 .

[0380] The first encoding unit 3810 is configured to determine the cost of the first encoding method and the second encoding method.

[0381] The second encoding unit 3820 is configured to encode the attribute information of the current layer according to the costs of the first encoding mode and the second encoding mode.

[0382] The current layer is any one of multiple RAHT layers. Under the first coding method, each node in the current layer corresponds to the same coding mode. Under the second coding method, the coding modes corresponding to the nodes in the current layer need to be determined separately.

[0383] In some implementations, the encoder 3800 further includes a third encoding unit 3830 configured to write first identification information into the bitstream, where the first identification information is used to indicate the first encoding mode or the second encoding mode.

[0384] In some implementations, the second encoding unit 3820 is configured to: if the cost of the first encoding mode is less than the cost of the second encoding mode, determine a target encoding mode from multiple encoding modes; encode the attribute information of each node in the current layer according to the target encoding mode; wherein the multiple encoding modes include at least one of the following: RAHT transform coding mode; RAHT intra-frame prediction combined with transform coding mode; RAHT inter-frame prediction combined with transform coding mode; a combination of RAHT intra-frame prediction combined with transform coding mode and RAHT inter-frame prediction combined with transform coding mode.

[0385] In some implementations, the encoder 3800 further includes a fourth encoding unit 3840 configured to write second identification information into the bitstream, where the second identification information is used to indicate the target encoding mode.

[0386] In some implementations, the second encoding unit 3820 is configured to: determine the encoding mode of the current node in the current layer if the cost of the second encoding mode is less than the cost of the first encoding mode; and encode the attribute information of the current node according to the encoding mode of the current node.

[0387] In some implementations, the second encoding unit 3820 is configured to: if the current layer is located in the first level range of the multiple RAHT layers, determine the encoding mode of the current node based on whether the reference frame includes the co-located node of the current node.

[0388] In some implementations, the second encoding unit 3820 is configured as: if the reference frame includes the co-located node, the encoding mode of the current node is RAHT inter-frame prediction combined with transform coding mode; and / or if the reference frame does not include the co-located node, the encoding mode of the current node is RAHT intra-frame prediction combined with transform coding mode.

[0389] In some implementations, the RAHT layer within the first level range is located at the highest layer among the multiple RAHT layers.

[0390] In some implementations, the second encoding unit 3820 is configured to: if the current layer is located in the second level range of the multiple RAHT layers, write third identification information into the codestream, where the third identification information is used to indicate the coding mode of the current node.

[0391] In some implementations, the coding mode indicated by the third identification information is one of the following: RAHT transform coding mode; RAHT intra-frame prediction combined with transform coding mode; RAHT inter-frame prediction combined with transform coding mode.

[0392] In some implementations, the RAHT layer within the second level range is located in a middle layer among the multiple RAHT layers.

[0393] In some implementations, the second encoding unit 3820 is configured to: if the current layer is located in the third level range of the multiple RAHT layers, determine the encoding mode of the current node based on the neighboring nodes of the current node.

[0394] In some implementations, the second coding unit 3820 is configured to: if the number of neighboring nodes of the parent node of the current node is greater than a first threshold and the number of neighboring nodes of the grandparent node of the current node is greater than a second threshold, then the coding mode for the current node is RAHT intra prediction combined with transform coding mode; and / or if the number of neighboring nodes of the parent node of the current node is less than or equal to a first threshold and / or the number of neighboring nodes of the grandparent node of the current node is less than or equal to a second threshold, then the coding mode for the current node is RAHT transform coding mode. In some implementations, the RAHT layer within the third hierarchical range is located at the lowest layer of the multiple RAHT layers.

[0395] In some implementations, the first identification information is located in an attribute block header.

[0396] In some implementations, the second encoding unit 3820 is configured to: determine the intra-frame prediction value and the inter-frame prediction value of the AC coefficient of the current node in the current layer; fuse the intra-frame prediction value and the inter-frame prediction value to determine the target prediction value of the AC coefficient; and determine the reconstruction attribute information of the current node based on the target prediction value.

[0397] In some implementations, the second encoding unit 3820 is configured to: determine the reconstructed value of the AC coefficient based on the target prediction value of the AC coefficient and the prediction residual value of the AC coefficient; and perform inverse transformation on the reconstructed value of the AC coefficient and the DC coefficient of the current node to determine the reconstructed value of the attribute information of the current node.

[0398] In some implementations, the encoder 3800 further includes a fifth encoding unit 3850 configured to write fourth identification information into the bitstream, where the fourth identification information is used to indicate whether to use a motion-compensated reference frame for RAHT inter-frame prediction.

[0399] In some implementations, the fourth identification information is identification information corresponding to a slice; or, the fourth identification information is identification information corresponding to a RAHT layer.

[0400] In some implementations, the motion compensation is local motion compensation or global motion compensation.

[0401] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0402] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, ROM, RAM, a magnetic disk, or an optical disk.

[0403] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 3800. The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the decoding method described in any one of the aforementioned embodiments.

[0404] Based on the composition of the above-mentioned encoder 3800 and the computer-readable storage medium, refer to Figure 39, which shows a specific hardware structure diagram of the encoder 3900 provided in an embodiment of the present application. As shown in Figure 39, the encoder 3900 may include: a communication interface 3910, a memory 3920 and a processor 3930; each component is coupled together through a bus system 3940. It can be understood that the bus system 3940 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 3940 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as bus systems 3940 in Figure 39. Among them,

[0405] Communication interface 3910, used for sending and receiving signals when sending and receiving information with other external network elements;

[0406] Memory 3920, for storing computer programs;

[0407] Processor 3930 is configured to, when running the computer program, execute:

[0408] Determine costs of a first encoding mode and a second encoding mode; and encode attribute information of a current layer according to the costs of the first encoding mode and the second encoding mode; wherein the current layer is any one of a plurality of RAHT layers, and under the first encoding mode, each node in the current layer corresponds to the same encoding mode, and under the second encoding mode, the encoding modes corresponding to the nodes in the current layer need to be determined separately.

[0409] It will be appreciated that the memory 3920 in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a ROM, PROM, EPROM, EEPROM, or flash memory. The volatile memory may be a RAM, which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as SRAM, DRAM, SDRAM, DDRSDRAM, ESDRAM, SLDRAM, and DRRAM. The memory 3920 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0410] Processor 3930 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method may be performed by hardware integrated logic circuits within processor 3930 or by software instructions. Processor 3930 may be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. It may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules may be located in a storage medium known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 3920. Processor 3930 reads information from memory 3920 and, in conjunction with its hardware, completes the steps of the above method.

[0411] It is understood that the embodiments described herein can be implemented with hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, DSPDs, PLDs, FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof. For software implementation, the technology described herein can be implemented by modules (e.g., processes, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0412] Optionally, as another embodiment, the processor 3930 is further configured to execute the encoding method described in any one of the aforementioned embodiments when running the computer program.

[0413] An embodiment of the present application also provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium for storing a bit stream. The bit stream can be generated by an encoding method of an encoder, or the bit stream can be decoded by a decoding method of a decoder, wherein the decoding method can be the decoding method described in any of the foregoing embodiments, and the encoding method can be the encoding method described in any of the foregoing embodiments.

[0414] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0415] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0416] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0417] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0418] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0419] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A point cloud decoding method, applied to a decoder, comprising: Analyzing a bitstream to determine first identification information; Decoding the attribute information of the current layer according to the first identification information; Wherein, the current layer is any one of multiple Region Adaptive Hierarchical Transform (RAHT) layers, and the first identification information is used to indicate that the current layer uses a first decoding method or a second decoding method. In the first decoding method, each node in the current layer corresponds to the same decoding mode. In the second decoding method, the decoding mode corresponding to each node in the current layer needs to be determined respectively.

2. The method according to claim 1, wherein, The decoding the attribute information of the current layer according to the first identification information includes: If the first identification information indicates the first decoding method, analyzing the bitstream to determine second identification information, where the second identification information is used to indicate a target decoding mode; Decoding the attribute information of each node in the current layer according to the target decoding mode; Wherein, the target decoding mode is one of the following: RAHT transform decoding mode; RAHT intra prediction combined with transform decoding mode; RAHT inter prediction combined with transform decoding mode; A combination of the RAHT intra prediction combined with transform decoding mode and the RAHT inter prediction combined with transform decoding mode.

3. The method according to claim 1, wherein The decoding the attribute information of the current layer according to the first identification information includes: If the first identification information indicates the second decoding method, determining the decoding mode of the current node in the current layer; Decoding the attribute information of the current node according to the decoding mode of the current node.

4. The method according to claim 3, wherein, The determining the decoding mode of the current node in the current layer includes: If the current layer is within the first hierarchical range of the multiple RAHT layers, determining the decoding mode of the current node according to whether the reference frame contains a co-located node of the current node.

5. The method according to claim 4, wherein: If the reference frame contains the co-located node, the decoding mode of the current node is the RAHT inter prediction combined with transform decoding mode; and / or If the reference frame does not contain the co-located node, the decoding mode of the current node is the RAHT intra prediction combined with transform decoding mode.

6. The method according to claim 4 or 5, wherein The RAHT layers within the first hierarchical range are located at the highest layer of the multiple RAHT layers.

7. The method according to claim 3, wherein The determining the decoding mode of the current node in the current layer includes: If the current layer is within the second hierarchical range of the multiple RAHT layers, analyzing third identification information, where the third identification information is used to indicate the decoding mode of the current node.

8. The method according to claim 7, wherein The decoding mode indicated by the third identification information is one of the following: RAHT transform decoding mode; RAHT intra prediction combined with transform decoding mode; RAHT inter prediction combined with transform decoding mode.

9. The method according to claim 7 or 8, wherein The RAHT layers within the second hierarchical range are located at the middle layer of the multiple RAHT layers.

10. The method according to claim 3, wherein, The determining the decoding mode of the current node in the current layer includes: If the current layer is within the third hierarchical range of the plurality of RAHT layers, determine the decoding mode of the current node according to the neighboring nodes of the current node.

11. The method according to claim 10, wherein: If the number of neighboring nodes of the parent node of the current node is greater than a first threshold and the number of neighboring nodes of the grandparent node of the current node is greater than a second threshold, the decoding mode of the current node is the RAHT intra prediction combined with transform decoding mode; and / or, If the number of neighboring nodes of the parent node of the current node is less than or equal to the first threshold and / or the number of neighboring nodes of the grandparent node of the current node is less than or equal to the second threshold, the decoding mode of the current node is the RAHT transform decoding mode.

12. The method according to claim 10 or 11, wherein The RAHT layer within the third hierarchical range is the lowest layer among the plurality of RAHT layers.

13. The method according to claim 1, wherein, The first identification information is located in the attribute block header.

14. The method according to claim 1 or 2, wherein The decoding of the attribute information of the current layer includes: Determine the intra prediction value and the inter prediction value of the alternating current (AC) coefficients of the current node in the current layer; Fuse the intra prediction value and the inter prediction value to determine the target prediction value of the AC coefficients; Determine the reconstructed attribute information of the current node according to the target prediction value.

15. The method according to claim 14, wherein, The determining the reconstructed attribute information of the current node according to the target prediction value includes: Determine the reconstructed value of the AC coefficients according to the target prediction value of the AC coefficients and the prediction residual value of the AC coefficients; Perform an inverse transform on the reconstructed value of the AC coefficients and the direct current (DC) coefficients of the current node to determine the reconstructed value of the attribute information of the current node.

16. The method according to claim 1, wherein, The method further includes: Parse the bitstream to determine fourth identification information, where the fourth identification information is used to indicate whether to use a motion-compensated reference frame for RAHT inter prediction.

17. The method according to claim 16, wherein: The fourth identification information is the identification information corresponding to the slice; or, The fourth identification information is the identification information corresponding to the RAHT layer.

18. The method according to claim 16 or 17, wherein The motion compensation is local motion compensation or global motion compensation.

19. A point cloud encoding method, applied to an encoder, includes: Determine the costs of a first encoding method and a second encoding method; Encode the attribute information of the current layer according to the costs of the first encoding method and the second encoding method; Wherein, the current layer is any one of a plurality of region adaptive hierarchical transform (RAHT) layers. Under the first encoding method, each node in the current layer corresponds to the same encoding mode. Under the second encoding method, the encoding modes corresponding to each node in the current layer need to be determined respectively.

20. The method according to claim 19, wherein, The method further includes: Write first identification information into the bitstream, where the first identification information is used to indicate the first encoding method or the second encoding method.

21. The method according to claim 19, wherein, The encoding the attribute information of the current layer according to the costs of the first encoding method and the second encoding method includes: If the cost of the first encoding method is less than the cost of the second encoding method, determine a target encoding mode from multiple encoding modes; Encoding attribute information of each node in the current layer according to the target coding mode; The multiple encoding modes include at least one of the following: RAHT transform coding mode; RAHT intra prediction combined with transform coding mode; RAHT inter-frame prediction combined with transform coding mode; A combination of RAHT intra prediction combined with transform coding mode and RAHT inter prediction combined with transform coding mode.

22. The method according to claim 21, wherein The method further comprises: The second identification information is written into the bitstream, where the second identification information is used to indicate the target encoding mode.

23. The method according to claim 19, wherein, The encoding of the attribute information of the current layer according to the cost of the first encoding mode and the second encoding mode includes: If the cost of the second encoding mode is less than the cost of the first encoding mode, determining the encoding mode of the current node in the current layer; The attribute information of the current node is encoded according to the encoding mode of the current node.

24. The method according to claim 23, wherein, The determining of the encoding mode of the current node in the current layer includes: If the current layer is located in the first level range of the multiple RAHT layers, the coding mode of the current node is determined according to whether the reference frame includes the co-located node of the current node.

25. The method of claim 24, wherein: If the reference frame includes the co-located node, the coding mode of the current node is a RAHT inter-frame prediction combined with a transform coding mode; and / or If the reference frame does not include the co-located node, the coding mode of the current node is the RAHT intra prediction combined with transform coding mode.

26. The method according to claim 24 or 25, wherein The RAHT layer within the first level range is located at the highest level among the plurality of RAHT layers.

27. The method according to claim 23, wherein The determining of the encoding mode of the current node in the current layer includes: If the current layer is located in the second level range of the multiple RAHT layers, third identification information is written into the bitstream, where the third identification information is used to indicate the coding mode of the current node.

28. The method according to claim 27, wherein, The encoding mode indicated by the third identification information is one of the following: RAHT transform coding mode; RAHT intra prediction combined with transform coding mode; RAHT inter-frame prediction combined with transform coding mode.

29. The method according to claim 27 or 28, wherein, The RAHT layer within the second level range is located in a middle layer among the plurality of RAHT layers.

30. The method according to claim 23, wherein The determining of the encoding mode of the current node in the current layer includes: If the current layer is located in a third level range among the multiple RAHT layers, a coding mode of the current node is determined according to neighboring nodes of the current node.

31. The method of claim 30, wherein: If the number of neighboring nodes of the parent node of the current node is greater than a first threshold and the number of neighboring nodes of the grandparent node of the current node is greater than a second threshold, the coding mode of the current node is a RAHT intra prediction combined with a transform coding mode; and / or, If the number of neighboring nodes of the parent node of the current node is less than or equal to a first threshold and / or the number of neighboring nodes of the grandparent node of the current node is less than or equal to a second threshold, the coding mode of the current node is a RAHT transform coding mode.

32. The method according to claim 30 or 31, wherein The RAHT layer within the third layer range is located at the lowest layer among the multiple RAHT layers.

33. The method according to claim 20, wherein The first identification information is located in the attribute block header.

34. The method according to claim 19 or 21, wherein Encoding the attribute information of the current layer includes: Determining the intra-prediction value and the inter-prediction value of the AC coefficients of the current node in the current layer; Fusing the intra-prediction value and the inter-prediction value to determine the target prediction value of the AC coefficients; Determining the reconstructed attribute information of the current node according to the target prediction value.

35. The method according to claim 34, wherein The determining the reconstructed attribute information of the current node according to the target prediction value includes: Determining the reconstructed value of the AC coefficients according to the target prediction value of the AC coefficients and the prediction residual value of the AC coefficients; Performing an inverse transform on the reconstructed value of the AC coefficients and the DC coefficients of the current node to determine the reconstructed value of the attribute information of the current node.

36. The method according to claim 19, wherein, The method further includes: Writing fourth identification information into the bitstream, where the fourth identification information is used to indicate whether to use a motion-compensated reference frame for RAHT inter-prediction.

37. The method according to claim 36, wherein: The fourth identification information is the identification information corresponding to a slice; or, The fourth identification information is the identification information corresponding to a RAHT layer.

38. The method according to claim 36 or 37, wherein, The motion compensation is local motion compensation or global motion compensation.

39. A decoder, comprising: A first decoding unit configured to parse the bitstream and determine the first identification information; A second decoding unit configured to decode the attribute information of the current layer according to the first identification information; wherein the current layer is any one of multiple Region Adaptive Hierarchical Transform (RAHT) layers, and the first identification information is used to indicate that the current layer uses a first decoding method or a second decoding method. In the first decoding method, each node in the current layer corresponds to the same decoding mode, and in the second decoding method, the decoding mode corresponding to each node in the current layer needs to be determined separately.

40. A decoder, comprising: A memory for storing a computer program; A processor for executing the method according to any one of claims 1 to 18 when running the computer program.

41. An encoder, comprising: A first encoding unit configured to determine the costs of a first encoding method and a second encoding method; A second encoding unit configured to encode the attribute information of the current layer according to the costs of the first encoding method and the second encoding method; wherein the current layer is any one of multiple Region Adaptive Hierarchical Transform (RAHT) layers. In the first encoding method, each node in the current layer corresponds to the same encoding mode, and in the second encoding method, the encoding mode corresponding to each node in the current layer needs to be determined separately.

42. An encoder, comprising: A memory for storing a computer program; A processor for executing the method according to any one of claims 19 to 38 when running the computer program.

43. A non-volatile computer-readable storage medium storing a bitstream, the bitstream being generated by an encoding method using an encoder or being decoded by a decoding method using a decoder, wherein, The decoding method is the method according to any one of claims 1 to 18, and the encoding method is the method according to any one of claims 19 to 38.

44. A bitstream, the bitstream including a bitstream generated by the method according to any one of claims 19 to 38.

45. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 18 or 19 to 38 is implemented.

Images