Point cloud encoding method, point cloud decoding method, encoder, decoder, and storage medium

Abstract

Provided in the embodiments of the present application are a point cloud encoding method, a point cloud decoding method, an encoder, a decoder, and a storage medium. The point cloud decoding method comprises: decoding a bitstream to determine first motion information (S3010); on the basis of the first motion information, performing motion compensation on a first reference point cloud, so as to determine a second reference point cloud (S3020); on the basis of the second reference point cloud, determining a predicted value of attribute information or geometric information of a current point cloud (S3030); and on the basis of the predicted value of the attribute information or the geometric information of the current point cloud, determining a reconstructed value of the attribute information or the geometric information of the current point cloud (S3040).

Description

Point cloud encoding / decoding methods, codecs, and storage media Technical Field

[0001] This application relates to the field of point cloud encoding and decoding technology, and in particular to a point cloud encoding and decoding method, an encoder and decoder, and a storage medium. Background Technology

[0002] In geometry-based point cloud compression (G-PCC) encoding and decoding frameworks, the geometric information and attribute information of point clouds are encoded and decoded separately. Improving the encoding and decoding efficiency of attribute information is a problem that needs to be solved.

[0003] Summary of the Invention

[0004] This application provides a point cloud encoding / decoding method, an encoder / decoder, and a storage medium. The various aspects involved in this application are described below.

[0005] In a first aspect, a point cloud decoding method is provided, applied to a decoder, comprising: decoding a bitstream to determine first motion information; performing motion compensation on a first reference point cloud based on the first motion information to determine a second reference point cloud; determining predicted values ​​of attribute information or geometric information of the current point cloud based on the second reference point cloud; and determining reconstructed values ​​of attribute information or geometric information of the current point cloud based on the predicted values ​​of attribute information or geometric information of the current point cloud.

[0006] Secondly, a point cloud encoding method is provided, applied to an encoder, comprising: performing motion compensation on a first reference point cloud based on first motion information to determine a second reference point cloud; determining predicted values ​​of attribute information or geometric information of the current point cloud based on the second reference point cloud; and determining residual values ​​of attribute information or geometric information of the current point cloud based on the predicted values ​​of attribute information or geometric information of the current point cloud.

[0007] Thirdly, a decoder is provided, comprising: a first determining unit configured to decode a bitstream and determine first motion information; a second determining unit configured to perform motion compensation on a first reference point cloud based on the first motion information and determine a second reference point cloud; a third determining unit configured to determine predicted values ​​of attribute information or geometric information of the current point cloud based on the second reference point cloud; and a fourth determining unit configured to determine reconstructed values ​​of attribute information or geometric information of the current point cloud based on the predicted values ​​of attribute information or geometric information of the current point cloud.

[0008] Fourthly, a decoder is provided, comprising: a memory for storing a computer program; and a processor for executing the method of the first aspect when running the computer program.

[0009] Fifthly, an encoder is provided, comprising: a first determining unit configured to perform motion compensation on a first reference point cloud based on first motion information to determine a second reference point cloud; a second determining unit configured to determine a predicted value of attribute information or geometric information of the current point cloud based on the second reference point cloud; and a third determining unit configured to determine a residual value of attribute information or geometric information of the current point cloud based on the predicted value of attribute information or geometric information of the current point cloud.

[0010] In a sixth aspect, an encoder is provided, the encoder comprising: a memory for storing a computer program; and a processor for executing the method of the second aspect when running the computer program.

[0011] In a seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program that, when executed, implements the method as described in the first or second aspect.

[0012] Eighthly, a non-volatile computer-readable storage medium is provided for storing a bit stream, the bit stream being generated by an encoding method using an encoder, or the bit stream being decoded by a decoding method using a decoder, wherein the decoding method is as described in the first aspect and the encoding method is as described in the second aspect.

[0013] Ninth aspect, a computer-readable storage medium is provided, which stores a bitstream generated according to the method of the second aspect.

[0014] This application introduces a motion-compensated inter-frame prediction scheme for inter-frame prediction of attribute information. Based on this, this application selects a suitable inter-frame prediction scheme by comparing the rate-distortion costs of the uncompensated and motion-compensated inter-frame prediction schemes, which helps improve the encoding and decoding efficiency of point cloud attribute information. Attached Figure Description

[0015] Figure 1A is a schematic diagram of a three-dimensional point cloud image.

[0016] Figure 1B is a magnified view of a three-dimensional point cloud image.

[0017] Figure 2A is a schematic diagram of six viewing angles for a point cloud image.

[0018] Figure 2B is a schematic diagram of a data storage format corresponding to a point cloud image.

[0019] Figure 3 is a schematic diagram of a network architecture for point cloud encoding and decoding.

[0020] Figure 4A is a schematic diagram of the component framework of a G-PCC encoder.

[0021] Figure 4B is a schematic diagram of the component framework of a G-PCC decoder.

[0022] Figure 5A is a schematic diagram of a low-plane position in the Z-axis direction.

[0023] Figure 5B is a schematic diagram of a high plane position in the Z-axis direction.

[0024] Figure 6 is a schematic diagram of a node encoding order.

[0025] Figure 7A is a schematic diagram of a planar signage information.

[0026] Figure 7B is a schematic diagram of another type of planar signage information.

[0027] Figure 8 is a schematic diagram of a sibling node of the current node.

[0028] Figure 9A is a schematic diagram of the three intersection points included in a sub-block.

[0029] Figure 9B is a schematic diagram of a triangular patch set fitted using three intersection points.

[0030] Figure 9C is a schematic diagram of upsampling of a triangular patch set.

[0031] Figure 10 is a schematic diagram of a distance-based level of detail (LOD) partitioning construction.

[0032] Figure 11 is a schematic diagram of a distance-based LOD point cloud generation process.

[0033] Figure 12 is a schematic diagram of the encoding process of attribute information of a LOD point cloud.

[0034] Figure 13 is a schematic diagram of a refinement layer based on LOD partitioning.

[0035] Figure 14 is a schematic diagram of inter-layer nearest neighbor search based on LOD.

[0036] Figure 15A is a schematic diagram of the spatial relationship between a child block and a parent block.

[0037] Figure 15B is a schematic diagram of a neighboring block that is coplanar, collinear, or shares a point with the current parent block.

[0038] Figure 16 is a schematic diagram of a method for finding the nearest neighbor of the current point.

[0039] Figure 17 is a schematic diagram of a nearest neighbor search method within an attribute information layer.

[0040] Figure 18 is a schematic diagram of a fast search method within a LOD layer.

[0041] Figure 19 is a schematic diagram of a neighborhood search prediction structure based on Morton codes.

[0042] Figure 20 is a schematic diagram of an encoding process for lifting transform.

[0043] Figure 21 is an example of a region adaptive hierarchical transform (RAHT) process.

[0044] Figure 22 shows another example of the RAHT transformation process.

[0045] Figure 23 is a schematic diagram of the RAHT transform and the inverse RAHT transform.

[0046] Figure 24 is a schematic diagram of the encoding block structure of attribute information.

[0047] Figure 25 is a schematic diagram of the overall process of RAHT intra-frame prediction transform coding of attribute information.

[0048] Figure 26 is an example of a linear fitting method for the neighborhood attribute information of the current block.

[0049] Figure 27 is a schematic diagram of RAHT intra-frame prediction transform coding for attribute information.

[0050] Figure 28 is a schematic diagram of the inter-frame prediction process for attribute information.

[0051] Figure 29 is a schematic diagram showing the correspondence between slices in the current frame and prediction units in the reference frame.

[0052] Figure 30 is a flowchart illustrating the decoding method provided in an embodiment of this application.

[0053] Figure 31 is a flowchart illustrating the encoding method provided in an embodiment of this application.

[0054] Figure 32 is a schematic diagram of the structure of a decoder provided in an embodiment of this application.

[0055] Figure 33 is a schematic diagram of the structure of a decoder provided in another embodiment of this application.

[0056] Figure 34 is a schematic diagram of the encoder provided in an embodiment of this application.

[0057] Figure 35 is a schematic diagram of the encoder provided in another embodiment of this application. Detailed Implementation

[0058] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0060] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] It should also be noted that the terms "first, second, and third" used in the embodiments of this application are only used to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0062] A point cloud is a set of randomly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. These points contain geometric information representing spatial location and attribute information representing the texture of the point cloud. Figure 1A shows a three-dimensional point cloud image, and Figure 1B shows a magnified view of a portion of the three-dimensional point cloud image. It can be seen that the surface of the point cloud is composed of densely distributed points.

[0063] Two-dimensional images contain information at each pixel, and their distribution is regular, so there's no need to record their positional information separately. However, the distribution of points in a point cloud in three-dimensional space is random and irregular, so the position of each point in space needs to be recorded to fully represent a point cloud. Similar to two-dimensional images, each location during acquisition has corresponding attribute information, usually RGB color values, reflecting the color of an object. For point clouds, in addition to color information, a common attribute for each point is reflectance, which reflects the surface material of the object. Therefore, point cloud data typically includes point position information and point attribute information. Point position information can also be called point geometric information. For example, point geometric information can be the three-dimensional coordinates (x, y, z). Point attribute information can include color information and / or reflectance, etc. For example, reflectance can be one-dimensional reflectance information (r); color information can be information in any color space, or it can be three-dimensional color information, such as RGB information. Here, R represents red (red, R), G represents green (green, G), and B represents blue (blue, B). For example, color information can be luminance and chromaticity (YCbCr, YUV) information. Here, Y represents luminance (luma), Cb (U) represents blue color difference, and Cr (V) represents red color difference.

[0064] Point clouds obtained based on laser measurement principles can include the three-dimensional coordinates and reflectance values ​​of the points. Similarly, point clouds obtained based on photogrammetry principles can include the three-dimensional coordinates and three-dimensional color information of the points. Furthermore, point clouds obtained by combining laser measurement and photogrammetry principles can include the three-dimensional coordinates, reflectance values, and three-dimensional color information of the points.

[0065] Figures 2A and 2B show a point cloud image and its corresponding data storage format. Figure 2A provides six viewing angles for the point cloud image, while Figure 2B consists of a header and a data section. The header information includes the data format, data representation type, total number of points in the point cloud, and the content represented by the point cloud. For example, the point cloud is in ".ply" format, represented by ASCII code, with a total of 207242 points. Each point has three-dimensional coordinate information (x, y, z) and three-dimensional color information (r, g, b).

[0066] Point clouds can be categorized according to the acquisition method:

[0067] Static point cloud: This means that the object is stationary and the device used to acquire the point cloud is also stationary.

[0068] Dynamic point cloud: The object is in motion, but the device acquiring the point cloud is stationary;

[0069] Dynamic point cloud acquisition: The device acquiring the point cloud is in motion.

[0070] For example, point clouds can be divided into two main categories based on their uses:

[0071] Category 1: Machine-perceived point cloud, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and disaster relief robots;

[0072] Category 2: Human eye-perceived point clouds, which can be used in point cloud application scenarios such as digital cultural heritage, free-viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0073] Point clouds can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Since point clouds are obtained by directly sampling real objects, they can provide a strong sense of realism while ensuring accuracy. Therefore, they are widely used, including virtual reality games, computer-aided design, geographic information systems, automatic navigation systems, digital cultural heritage, free-viewpoint broadcasting, three-dimensional immersive remote presentation, and three-dimensional reconstruction of biological tissues and organs.

[0074] Point cloud acquisition primarily utilizes the following methods: computer generation, 3D laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual 3D objects and scenes; 3D laser scanning can acquire point clouds of static real-world 3D objects or scenes, with a capacity of millions of point clouds per second; 3D photogrammetry can acquire point clouds of dynamic real-world 3D objects or scenes, with a capacity of tens of millions of point clouds per second. These technologies reduce the cost and time required for point cloud data acquisition and improve data accuracy. While the evolution of point cloud data acquisition methods has made acquiring massive amounts of point cloud data possible, the processing of such large volumes of 3D point cloud data is facing bottlenecks due to limitations in storage space and transmission bandwidth, driven by increasing application demands.

[0075] For example, taking a point cloud video with a frame rate of 30 frames per second (fps) as an example, each frame of the point cloud contains 700,000 points, and each point has coordinate information xyz (float) and color information RGB (uchar). Therefore, the data size of a 10-second point cloud video is approximately 0.7 million × (4 bytes × 3 + 1 byte × 3) × 30 fps × 10 s = 3.15 GB, where 1 byte is 10 bits. In contrast, a 10-second 1280×720 two-dimensional video with a YUV sampling format of 4:2:0 and a frame rate of 30 fps has a data size of approximately 1280×720 × 12 bits × 30 frames × 10 s ≈ 0.39 GB, and a 10-second two-view three-dimensional video has a data size of approximately 0.39 × 2 = 0.78 GB. It is evident that the data size of a point cloud video far exceeds that of two-dimensional and three-dimensional videos of the same duration. Therefore, in order to better achieve data management, save server storage space, and reduce the transmission traffic and transmission time between the server and the client, point cloud compression has become a key issue in promoting the development of the point cloud industry.

[0076] In other words, since point clouds are a collection of massive points, storing point clouds not only consumes a lot of memory, but is also not conducive to transmission. There is also not enough bandwidth to support the transmission of point clouds directly at the network layer without compression. Therefore, point clouds need to be compressed.

[0077] Currently, point cloud encoding frameworks capable of compressing point clouds can be the geometry-based point cloud compression (G-PCC) or video-based point cloud compression (V-PCC) frameworks provided by the Moving Picture Experts Group (MPEG), or the AVS-PCC framework provided by AVS. The G-PCC framework can be used to compress both static point clouds (Type 1) and dynamically acquired point clouds (Type 3), and it can be based on a point cloud compression test platform (test model compression 13, TMC13). The V-PCC framework can be used to compress dynamic point clouds (Type 2), and it can be based on a point cloud compression test platform (test model compression 2, TMC2). Therefore, the G-PCC framework is also called the point cloud codec TMC13, and the V-PCC framework is also called the point cloud codec TMC2.

[0078] This application provides a network architecture for a point cloud encoding / decoding system that includes decoding and encoding methods. Figure 3 is a schematic diagram of such a network architecture. As shown in Figure 3, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01. The electronic devices 13 to 1N can perform video interaction through the communication network 01. During implementation, the electronic devices can be various types of devices with point cloud encoding / decoding capabilities. For example, the electronic devices may include mobile phones, tablets, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, servers, etc. This application does not impose any limitations. The decoder or encoder in this application embodiment can be one of the aforementioned electronic devices.

[0079] The electronic device in this application embodiment has point cloud encoding and decoding functions, and generally includes a point cloud encoder (i.e., encoder) and a point cloud decoder (i.e. decoder).

[0080] The following section uses the G-PCC and AVS codec frameworks as examples to explain the relevant technologies.

[0081] As can be understood, in the G-PCC encoding and decoding framework for point clouds, the point cloud data to be encoded is first divided into multiple slices. Within each slice, the geometric information of the point cloud and the attribute information corresponding to each point are encoded separately.

[0082] Figure 4A illustrates a schematic diagram of the component framework of a G-PCC encoder. As shown in Figure 4A, during geometric encoding, coordinate transformation is performed on the geometric information so that the entire point cloud is contained within a bounding box, followed by quantization. After point cloud quantization, some point clouds have identical geometric information, so a decision is made based on parameters to remove duplicate points. This process of quantization and removal of duplicate points is also known as voxelization. Next, the bounding box is partitioned into an octree or a prediction tree is constructed. During this process, arithmetic encoding is performed on the points in the leaf nodes of the partition to generate a binary geometric bitstream; or, arithmetic encoding is performed on the vertices generated by the partition (surface fitting based on the vertices) to generate a binary geometric bitstream. During attribute encoding, after geometric encoding is completed and the geometric information is reconstructed, the color information (i.e., attribute information) is converted from the RGB color space to the YUV color space. Then, the reconstructed geometric information is used to recolor the point cloud so that the unencoded attribute information corresponds to the reconstructed geometric information. Attribute encoding primarily targets color information. Three main transformation methods are used in color information encoding: Predicting Transform (PT), Lifting Transform (LT), and Region Adaptive Hierarchical Transform (RAHT). PT and LT predictively encode point clouds based on the generation order of Level of Detail (LOD), while RAHT adaptively transforms attribute information from bottom to top based on the octree construction hierarchy. LT and RAHT transform color information from the spatial domain to the frequency domain, obtaining high-frequency and low-frequency coefficients through transformation. Finally, the coefficients are quantized, and then arithmetically encoded to generate a binary attribute bitstream.

[0083] Figure 4B illustrates a schematic diagram of the G-PCC decoder's structural framework. As shown in Figure 4B, for the acquired binary bitstream, the geometric bitstream and attribute bitstream within the binary bitstream are first decoded independently. During the decoding of the geometric bitstream, arithmetic decoding—reconstructing the octree / reconstructing the prediction tree—reconstructing geometry—inverse coordinate transformation is used to obtain the geometric information of the point cloud. During the decoding of the attribute bitstream, arithmetic decoding—inverse quantization—LOD partitioning / RAHT—inverse color transformation is used to obtain the attribute information of the point cloud. Based on the geometric and attribute information, the point cloud data to be encoded (i.e., the output point cloud) is reconstructed.

[0084] It should be noted that, as shown in Figure 4A or Figure 4B, the current geometric coding and decoding of G-PCC can be divided into octree-based geometric coding and decoding (marked with dashed boxes) and prediction tree-based geometric coding and decoding (marked with dotted-dash boxes).

[0085] In octree-based geometric encoding, the bounding box is continuously partitioned into tree types (e.g., octree, quadtree, binary tree) according to breadth-first traversal, and the placeholder code of each node is encoded. In related technologies, one company proposed an implicit geometric partitioning method, which first calculates the bounding box of the point cloud. Assume d x >d y >d z The bounding box corresponds to a cuboid. During geometric partitioning, a binary tree partition is first performed based on the x-axis, resulting in two child nodes; this continues until d is satisfied. x =d y >d z Only when the condition is met will the quadtree be partitioned based on the x and y axes to obtain four child nodes; when d is finally satisfied x =d y =d z Under certain conditions, the octree partitioning will continue until the resulting leaf nodes form a 1×1×1 unit cube. The partitioning then stops, and the nodes in the leaf nodes are encoded to generate a binary code stream. In the binary / quadtree / octree partitioning process, two parameters are introduced: K and M. Parameter K indicates the maximum number of binary / quadtree partitions performed before octree partitioning; parameter M indicates that the minimum block side length during binary / quadtree partitioning is 2. M At the same time, K and M must satisfy the following condition: Assume d max =max(d x ,d y ,d z ), d min =min(d x ,d y ,d z The parameter K satisfies: K ≥ d max ―d min The parameter M satisfies: M ≥ d minThe reason why parameters K and M satisfy the above conditions is that in the current G-PCC implicit partitioning process, the priority of partitioning methods is binary tree, quadtree, and octree. Only when the node block size does not meet the binary / quadtree condition will the node be continuously partitioned into an octree until the smallest leaf node unit of 1×1×1 is reached. The octree-based geometric information encoding mode can effectively encode the geometric information of the point cloud by utilizing the correlation between adjacent points in space. However, for some relatively flat nodes or nodes with planar characteristics, using planar encoding can further improve the encoding performance of the point cloud geometric information.

[0086] For example, Figures 5A and 5B provide a schematic diagram of planar positions. Figure 5A shows a schematic diagram of a low-plane position in the Z-axis direction, and Figure 5B shows a schematic diagram of a high-plane position in the Z-axis direction. As shown in Figure 5A, (a), (a0), (a1), (a2), and (a3) ​​all belong to the low-plane position in the Z-axis direction. Taking (a) as an example, it can be seen that the four child nodes occupied by the current node are all located in the low-plane position of the current node in the Z-axis direction. Therefore, the current node can be considered to belong to a Z-plane and be a low-plane in the Z-axis direction. Similarly, as shown in Figure 5B, (b), (b0), (b1), (b2), and (b3) all belong to the high-plane position in the Z-axis direction. Taking (b) as an example, it can be seen that the four child nodes occupied by the current node are located in the high-plane position of the current node in the Z-axis direction. Therefore, the current node can be considered to belong to a Z-plane and be a high-plane in the Z-axis direction.

[0087] Furthermore, taking Figure 5A(a) as an example, the performance of octree coding and planar coding is compared. Figure 6 provides a schematic diagram of node coding order, i.e., the nodes are encoded in the order of 0, 1, 2, 3, 4, 5, 6, 7 as shown in Figure 6. Here, if octree coding is used for Figure 5A(a), the placeholder information of the current node is represented as: 10101010. However, if planar coding is used, firstly, an identifier needs to be encoded to indicate that the current node is a plane in the Z-axis direction; secondly, if the current node is a plane in the Z-axis direction, the planar position of the current node also needs to be represented; secondly, only the placeholder information of the lower plane nodes in the Z-axis direction (i.e., the placeholder information of the four child nodes 0, 2, 4, and 6) needs to be encoded. Therefore, encoding the current node based on planar coding only requires encoding 6 bits, which can reduce the representation by 2 bits compared to octree coding of related technologies. Based on this analysis, planar coding has a more significant coding performance than octree coding. Therefore, for an occupied node, if planar encoding is used in a certain dimension, the planar identifier (planarMode) and planar position (PlanePos) information of the current node in that dimension must first be represented. Then, the occupancy information of the current node is encoded based on the planar information. For example, Figure 7A shows a schematic diagram of planar identifier information. As shown in Figure 7A, the Z-axis direction is a low plane; correspondingly, the planar identifier information is true or 1, i.e., planarMode_Z = true; the planar position information is a low plane, i.e., PlanePosition_Z = low. Figure 7B shows another schematic diagram of planar identifier information. As shown in Figure 7B, the Z-axis direction is not a plane; correspondingly, the planar identifier information is false or 0, i.e., planarMode_Z = false.

[0088] It's important to note that for `PlaneMode_i`: 0 indicates the current node is not in a plane along the i-axis, and 1 indicates the current node is in a plane along the i-axis. If the current node is in a plane along the i-axis, then for `PlanePosition_i`: 0 indicates the current node is in a plane along the i-axis, and the plane is a low plane; 1 indicates the current node is in a high plane along the i-axis. Here, `i` represents the coordinate dimension, which can be the X-axis, Y-axis, or Z-axis, hence `i = 0, 1, 2`.

[0089] Octree-based geometric information encoding only achieves high compression rates for points that are correlated in the spatial domain. For points isolated in the geometric space, the direct coding model (DCM) can significantly reduce complexity. For all nodes in the octree, the DCM is not represented by flag information, but rather inferred from the current node's parent node and neighbor information. There are three ways to determine whether the current node is eligible for DCM encoding, as follows:

[0090] (1) The current node has no sibling child nodes, that is, the parent node of the current node has only one child node, and the parent node of the current node has only two occupied child nodes, that is, the current node has at most one neighbor node.

[0091] (2) The parent node of the current node has only the current node as its child node, and the six neighboring nodes that share a face with the current node are all empty nodes.

[0092] (3) The number of sibling nodes of the current node is greater than 1.

[0093] For example, Figure 8 provides a schematic diagram of IDCM encoding. If the current node is not eligible for DCM encoding, it will be partitioned into an octree. If it is eligible, the number of points contained in the node will be further determined. If the number of points is less than a threshold (e.g., 2), the node will be DCM encoded; otherwise, the octree partitioning will continue. When applying the DCM encoding mode, it is first necessary to encode whether the current node is a true isolated point, i.e., IDCM_flag. If IDCM_flag is true, the current node will be DCM encoded; otherwise, octree encoding will still be used. When the current node satisfies DCM encoding, the DCM encoding mode of the current node needs to be encoded. Currently, there are two DCM modes: (a) only one point exists (or multiple points, but they are duplicate points); (b) it contains two points. Finally, the geometric information of each point needs to be encoded, assuming the side length of the node is 2. d Encoding each component of the geometric coordinates of a node requires d bits, and this bit information is directly encoded into the bitstream. It's important to note that when encoding LiDAR point clouds, predictive encoding of the three-dimensional coordinate information is performed using LiDAR acquisition parameters, which further improves the encoding performance of geometric information.

[0094] It's important to note that when partitioning nodes down to leaf nodes, in the case of geometrically lossless encoding, the number of duplicate points in the leaf nodes needs to be encoded. Finally, the placeholder information of all nodes is encoded to generate a binary code stream. Additionally, G-PCC currently introduces a planar encoding mode. During the geometric partitioning process, it determines whether the child nodes of the current node are on the same plane. If the child nodes of the current node satisfy the condition of being on the same plane, that plane will be used to represent the child nodes of the current node.

[0095] For LiDAR point cloud data, geometric inter-frame predictive coding based on octrees can also be performed. For this type of point cloud data, the geometric inter-frame coding schemes provided by relevant technologies directly obtain the global motion matrix from the configuration file corresponding to the LiDAR point cloud data. For example, when performing inter-frame geometric coding on the current frame, the global motion matrix of the frame to be encoded is obtained from the configuration file corresponding to the LiDAR point cloud data. Then, this global motion matrix can be used to perform motion compensation on the reference frame to obtain the motion-compensated reference frame. The specific calculation method is as follows:

[0096] in, For the global motion matrix, This provides the reconstructed geometric information of the current reference frame. Finally, the geometric information of the motion-compensated reference frame can be used to perform inter-frame predictive coding on the geometric information of the current frame to be coded.

[0097] For octree-based geometric decoding, the decoding end follows a breadth-first traversal. Before decoding the placeholder information of each node, it first uses the reconstructed geometric information to determine whether the current node is suitable for planar decoding or IDCM decoding. If the current node meets the conditions for planar decoding, its planar identifier and planar position information are decoded first, followed by decoding the placeholder information based on the planar information. If the current node meets the conditions for IDCM decoding, it first decodes whether it is a true IDCM node. If it is, it continues to parse the DCM decoding mode of the current node, then obtains the number of points in the current DCM node, and finally decodes the geometric information of each point. For nodes that do not meet either the conditions for planar decoding or DCM decoding, the placeholder information of the current node is decoded. By continuously parsing the placeholder code of each node in this way, and sequentially dividing the nodes until a 1×1×1 unit cube is obtained, the division stops. The number of points contained in each leaf node is then obtained, and finally, the geometrically reconstructed point cloud information is recovered.

[0098] For LiDAR point cloud data, octree-based inter-frame geometric prediction decoding can also be performed. For this type of point cloud data, related techniques determine whether global motion compensation is needed for the current frame by parsing the syntax elements of the geometry parameter set (GPS). If motion compensation is needed, the global motion matrix of the current frame is then parsed. The global motion matrix obtained through analysis is then used to perform motion compensation on the reference frame. Finally, the motion-compensated reference frame is used to decode and reconstruct the geometric information of the current frame.

[0099] For geometric information encoding based on triangle soup (trisoup), the same geometric partitioning is required within the trisoup-based geometric information encoding framework. However, unlike geometric information encoding based on binary / quadtree / octree, this method does not need to partition the point cloud step by step into unit cubes with a side length of 1×1×1. Instead, partitioning stops when the side length of a sub-block is W. Based on the surface formed by the distribution of the point cloud in each block, at most twelve vertices are obtained between this surface and the twelve edges of the block. The vertex coordinates of each block are then encoded sequentially to generate a binary bitstream.

[0100] For point cloud geometric information reconstruction based on trisoups, the process at the decoding end first involves decoding vertex coordinates to complete triangular facet reconstruction, as shown in Figures 9A, 9B, and 9C. In Figure 9A, the block contains three intersection points (v1, v2, v3). The set of triangular faces formed by these three intersection points in a specific order is called a triangle soup, or trisoup, as shown in Figure 9B. Then, sampling is performed on this triangular facet set, and the resulting sampled points are used as the reconstructed point cloud within the block, as shown in Figure 9C.

[0101] For predictive geometry coding (PredGeomTree), the process involves: First, sorting the input point cloud. Current sorting methods include unordered, Morton order, azimuth order, and radial distance order. At the encoding end, the prediction tree structure is built using two different methods: KD-Tree (high-latency, slow mode) and low-latency, fast mode (using LiDAR calibration information). When using LiDAR calibration information, each point is assigned to a different laser, and the prediction tree structure is built according to the different lasers. Next, based on the prediction tree structure, each node in the prediction tree is traversed. By selecting different prediction modes, the geometric position information of the node is predicted to obtain the prediction residual, and the geometric prediction residual is quantized using quantization parameters. Finally, through continuous iteration, the prediction residual of the prediction tree node position information, the prediction tree structure, and the quantization parameters are encoded to generate a binary code stream.

[0102] For prediction tree-based geometric decoding, the decoder continuously parses the bitstream to reconstruct the prediction tree structure. Then, it obtains the geometric position prediction residual information and quantization parameters of each prediction node through parsing. Finally, it performs inverse quantization on the prediction residual to recover the reconstructed geometric position information of each node, thus completing the geometric reconstruction at the decoder.

[0103] After geometric encoding is completed, the geometric information needs to be reconstructed. Currently, attribute encoding mainly targets color information. First, the color information is converted from the RGB color space to the YUV color space. Then, the reconstructed geometric information is used to recolor the point cloud, so that the unencoded attribute information corresponds to the reconstructed geometric information. In color information encoding, there are two main transformation methods: one is distance-based lifting transformation that relies on LOD partitioning, and the other is direct RAHT transformation. Both methods convert the color information from the spatial domain to the frequency domain, obtaining high-frequency and low-frequency coefficients through the transformation. Finally, the coefficients are quantized and encoded to generate a binary code stream, as shown in Figures 4A and 4B.

[0104] Furthermore, when using geometric information to predict attribute information, Morton codes can be used for nearest neighbor search. The Morton code corresponding to each point in the point cloud can be obtained from the geometric coordinates of that point. The specific method for calculating the Morton code is described below. For each component represented by a d-bit binary number in three-dimensional coordinates, its three components can be represented as:

[0105] in, These are the highest bits of x, y, and z, respectively. to the lowest position The corresponding binary value. The Morton code M arranges x, y, and z in a crosswise manner starting from the highest bit and going down to the lowest bit successively. The calculation formula of M is as follows: To the lowest bit, the calculation formula of M is as follows:

[0106] Where Are the highest bit of M To the lowest bit Values. After obtaining the Morton code M of each point in the point cloud, the points in the point cloud are arranged in ascending order of the Morton code, and the weight value w of each point is set to 1.

[0107] As mentioned above, there are mainly three transformation methods for the encoding of attribute information, namely the lifting transformation, the prediction transformation, and the RAHT transformation. The lifting transformation and the prediction transformation perform point cloud prediction and transformation based on the generation order of LOD, while the RAHT transformation adaptively transforms the attribute information from bottom to top according to the construction levels of the octree. Next, the encoding methods of these three transformations will be introduced. First, the encoding method of the prediction transformation will be introduced in detail below.

[0108] In the current attribute prediction module of G-PCC, a nearest neighbor attribute prediction coding scheme based on the LOD structure is adopted. The construction methods of LOD include the LOD construction scheme based on distance, the LOD construction scheme based on a fixed sampling rate, and the LOD construction scheme based on the octree, etc. In the LOD construction scheme based on distance, before constructing LOD, the point cloud is first sorted by Morton to ensure strong attribute correlation between adjacent points. Figure 10 shows a schematic diagram of the LOD construction based on distance. As shown in Figure 10, the point cloud is divided into L different point cloud detail levels (Rl)l = 0, 1, … L - 1 according to the pre-set L Manhattan distances (dl)l = 0, 1, … L - 1, where (dl)l = 0, 1, … L - 1 satisfies dl < dl-1.

[0109] The construction process of LOD is as follows: (1) First, mark all points in the point cloud as unvisited, and establish a set V to store the set of points that have been visited; (2) In each iteration, traverse the points in the point cloud. If the current point has been visited, ignore this point, otherwise calculate the minimum distance D from the current point to the point set V. If D < dl, ignore this point; if D ≥ dl, mark the current point as visited and add the current point to the refinement layer Rl and the point set V; (3) The points in the detail level LODl are composed of the points in the refinement layers R0, R1, R2…Rl; (4) Repeat the above steps continuously until all points are marked as visited.

[0110] Based on the LOD structure, the attribute information of each point is predicted using a linearly weighted average of the reconstructed attribute information from points in the same or higher LOD layers. The maximum number of reference prediction neighbors is determined by the higher-level syntax elements of the encoder. For each point's attribute, a rate-distortion optimization algorithm is used at the encoder to select the attribute information of the N nearest neighbors for weighted prediction; alternatively, the attribute information of a single nearest neighbor is selected for prediction. Finally, the selected prediction mode and the prediction residual are encoded. In the LOD structure, the formula for predicting the attribute information of the current point can be as follows:

[0111] Where N represents the number of predicted points in the nearest neighbor set of current point i, Pi represents the sum of the N nearest neighbors of current point i, Dm represents the spatial geometric distance from the nearest neighbor m to current point i, Attrm represents the attribute information of the reconstructed nearest neighbor m, and Attr i ′ represents the attribute prediction information for the current point i, and the number of points N is a pre-set value.

[0112] To balance attribute encoding performance with parallel processing across different LOD layers, a switch can be introduced in the higher-level syntax elements of the encoder. This switch controls whether intra-LOD layer prediction is used. For example, if the switch is turned on, intra-LOD layer prediction is enabled, allowing predictions to be made using points within the same LOD layer. It's important to note that when the number of LOD layers is one, intra-LOD layer prediction will always be used.

[0113] Figure 11 illustrates the generation process of distance-based LOD point clouds. As shown in Figure 11, the first image on the left is the original point cloud, the second image on the left represents the outer contour of the point cloud, and in the following images, the point cloud details become clearer as the number of detail layers increases. The prediction process of attribute information for LOD structured point clouds is described in detail below.

[0114] Figure 12 illustrates the encoding process of attribute information in a Level of Detail (LOD) point cloud. After the LOD is constructed, according to the generation order of the LOD, the three nearest neighbors of the point to be encoded are first found from the encoded data points. The reconstructed attribute values ​​of these three nearest neighbors are used as candidate predicted values ​​for the point to be encoded. Then, the optimal predicted value is selected from these candidates using the rate-distortion optimization algorithm. For example, as shown in Table 1, when encoding the attribute value of point P2 in Figure 10, the predictor index of the attribute value of the nearest neighbor point P4 can be set to 1; the predictor indices of the attribute values ​​of the second nearest neighbor point P5 and the third nearest neighbor point P0 can be set to 2 and 3, respectively; the predictor index of the weighted average of points P0, P5, and P4 can be set to 0; finally, the best predictor is selected using the rate-distortion optimization algorithm.

[0115] The formula for the weighted average is as follows:

[0116] In the formula The spatial geometric weights from nearest neighbor j to current point i are:

[0117] This represents the predicted attribute value for the current point i, and j represents the index of the three neighboring points. x represents the attribute value after the nearest neighbor is reconstructed. i ,y i ,z i The x-coordinates are the geometric coordinates of the current point i. ij ,y ij ,z ij Let be the geometric coordinates of the nearest neighbor point j.

[0118] Table 1

[0119] The attribute prediction value of the current point i is obtained through the above prediction. (k is the total number of points in the point cloud). Let (a i )i ∈0…k―1 If the original attribute value is given by the current point, then the attribute residual (r) i )i ∈0…k―1 Recorded as:

[0120] Further quantification of the predicted residuals:

[0121] In equation (7), Q iThis represents the quantized attribute residual at the current point i, where Qs is the quantization step size, which can be calculated from the quantization parameter (QP). After quantization, the quantization coefficients are arithmetically encoded to ultimately generate the attribute bitrate.

[0122] During the encoding process, the encoder reconstructs the attribute values ​​of the current point i. The purpose of this reconstruction is to predict subsequent points. Before reconstructing the attribute values, the residuals are inversely quantized. The residual after dequantization:

[0123] Compared with the predicted value The reconstructed value of point i is obtained by adding them together.

[0124] As described above, based on LOD (Level of Detail) partitioning, predicting the attribute value of the current point requires nearest neighbor lookup. Currently, there are two main types of nearest neighbor lookup methods: intra-frame nearest neighbor lookup and inter-frame nearest neighbor lookup. These two methods will be discussed in detail below.

[0125] Intra-frame nearest neighbor search can be divided into two methods: inter-layer nearest neighbor search and intra-layer nearest neighbor search. We will first introduce inter-layer nearest neighbor search. Figure 13 shows a schematic diagram of the refinement layer structure based on LOD partitioning. As shown in Figure 13, after LOD partitioning, different refinement layers R will form a pyramid-like structure. The method of inter-layer nearest neighbor search can be illustrated in Figure 14. First, based on the method shown in Figure 10, the geometric information is partitioned into different LOD layers, resulting in LOD0, LOD1, and LOD2. During the inter-layer nearest neighbor search process, points in LOD0 are used to predict the attributes of points in the next LOD layer. The process of inter-layer nearest neighbor search will be described in detail below.

[0126] During the entire LOD partitioning process, three sets exist: O(k), L(k), and I(k). Here, k is the index of the LOD layer during partitioning, and I(k) is the input point set for the current LOD layer. After LOD partitioning, we obtain the sets O(k) and L(k). The O(k) set stores the sampled point set, and L(k) is the point set in the current LOD layer. The entire LOD partitioning process is shown below:

[0127] (1) Initialization;

[0128] If k = 0, L(k) ← {}; otherwise, L(k) ← L(k-1).

[0129] O(k)←{}

[0130] (2) Using the LOD partitioning algorithm, the sampling points are stored in O(k), and the remaining points are partitioned into L(k);

[0131] (3) When performing the next iteration, I←O(k).

[0132] It's important to note that since the entire LOD partitioning process is based on Morton codes, O(k), L(k), and I(k) store the Morton code indices corresponding to the points. During inter-layer nearest neighbor lookups, points in the L(k) set are searched for their nearest neighbors in the O(k) set. The specific search method will be detailed below.

[0133] First, nearest neighbor lookup is performed based on spatial relationships. As shown in Figure 15A, when predicting the current point P, a neighbor search is performed using the parent block (Block B) corresponding to point P. Figure 15B shows a schematic diagram of neighbor blocks that are coplanar, collinear, and intervertical with the current parent block. As shown in Figure 15B, points within coplanar and collinear neighbor blocks with the current parent block are searched for to predict attributes. That is, the corresponding spatial block is obtained using the coordinates of the current point, and then a nearest neighbor lookup is performed in the previously encoded LOD layer to find spatial blocks that are coplanar, collinear, and intervertical with the current block to obtain the N nearest neighbors of the current point.

[0134] If the N nearest neighbors of the current point are still not found after searching for coplanar, collinear, and concurrent nearest neighbors, then a fast search algorithm will be used to find the N nearest neighbors of the current point. See Figure 16 for details. Figure 16 illustrates the method of finding the nearest neighbor for the current point. As shown in Figure 16, when performing inter-layer attribute prediction, the Morton code corresponding to the current point can be obtained using the geometric coordinates of the current point. Then, based on the Morton code of the current point, the first reference point (j) with a Morton code greater than the current point is found in the reference frame, and a nearest neighbor search is performed within the range [j-searchRange, j+searchRange]. The specific method of updating the nearest neighbor is the same as that of inter-frame nearest neighbor search, which will be described when introducing inter-frame nearest neighbor search, and will not be repeated here. The following section provides a detailed introduction to intra-layer nearest neighbor search.

[0135] Figure 17 illustrates the nearest neighbor lookup method within the attribute information layer. As shown in Figure 17, when intra-layer prediction is enabled, a nearest neighbor lookup is performed on the set of encoded points within the same LOD layer to obtain the N nearest neighbors of the current point (inter-layer nearest neighbor lookup is also performed). The nearest neighbor lookup method can be based on fast lookup. For example, as shown in Figure 18, assuming the Morton code index of the current point is i, a nearest neighbor lookup will be performed in [i+1, i+searchRange]. The specific nearest neighbor lookup method is consistent with the inter-frame block-based fast lookup method, and will not be elaborated here. The inter-frame nearest neighbor lookup method will be described in detail below.

[0136] Referring to Figure 16, when performing inter-frame prediction of attributes, the Morton code corresponding to the current point is obtained using the geometric coordinates of the current point to be predicted. Based on the Morton code of the current point, the first reference point (j) with a Morton code greater than that of the current point is found in the reference frame. Then, the nearest neighbor search is performed in the range of [j-searchRange,j+searchRange].

[0137] Currently, nearest neighbor searches within and between frames are performed on a block-based basis. Figure 19 shows a schematic diagram of the prediction structure for neighborhood search based on Morton codes. For example, as shown in Figure 19, when performing a neighborhood search on the current point (Morton code index i), the points in the reference frame are divided into N (N=3) layers according to the Morton code. The division method can be as follows:

[0138] First layer: Assuming the points in the reference frame are numPoints, first divide the points in the reference frame into a block for every M (M=25=32) points;

[0139] Second layer: Based on the first layer, the blocks of the first layer are divided into one block every M (M=25=32) blocks according to the order of the Morton code;

[0140] The third layer: Based on the second layer, the blocks of the first layer are divided into one block every M (M=25=32) blocks, following the order of the Morton code.

[0141] The final predicted structure is shown in Figure 19.

[0142] When performing attribute prediction based on the prediction structure shown in Figure 19, assuming the Morton code index of the current point to be encoded is i, the first point in the reference frame with an index j that is greater than or equal to the current point's Morton code is obtained. The block index of the reference point is calculated based on j, as shown below:

[0143] First layer: BucketSize_0 = 25 = 32;

[0144] Second layer: BucketSize_1 = 25 = 32 × BucketSize_0 = 1024;

[0145] Third layer: BucketSize_2 = 25 = 32 × BucketSize_1 = 32768.

[0146] Assuming the reference range in the prediction frame for the current point is [j-searchRange, j+searchRange], the starting index of the third layer is calculated using j-searchRange, and the ending index of the third layer is calculated using j+searchRange. First, within the blocks of the third layer, it is determined whether some blocks of the second layer require nearest neighbor lookups. Second, for each block in the first layer, it is determined whether a lookup is needed. If some blocks in the first layer require nearest neighbor lookups, then the nearest neighbors in some blocks of the first layer are updated point-by-point. The method of calculating blocks based on indexes is described below.

[0147] Assuming the Morton code index corresponding to the current point is index, then the index of the corresponding third-level block is: idx_2 = index / BucketSize_2 (10)

[0148] After obtaining the block index idx_2 of the third level, we can use idx_2 to obtain the start index and end index of the block corresponding to the current block in the second level: startIdx1=idx_2×BucketSize_1 (11) endIdx=idx_2×BucketSize_1+BucketSize_1-1 (12)

[0149] The same algorithm is used to obtain the index of the first-level block from the index of the second-level block.

[0150] When performing nearest neighbor lookup based on blocks, it is determined whether the current block needs to undergo nearest neighbor lookup, which is equivalent to filtering the nearest neighbor lookup of blocks. Each spatial block can be obtained through two variables: minPos and maxPos. minPos represents the minimum value of the block, and maxPos represents the maximum value of the block. Assuming the distance to the farthest point among the N nearest neighbors of the current point is Dist, the coordinates of the point to be encoded are (x, y, z), and the current block is represented as (minPos, maxPos), where minPos is the minimum value in the three dimensions of the bounding box, and maxPos is the maximum value in the three dimensions of the bounding box, the distance D between the current point and the bounding box is calculated as follows: int dx=int(std::max(std::max(minPos[0]-point[0],0), point[0]-maxPos[0])) (13) int dy=int(std::max(std::max(minPos[1]-point[1],0), point[1]-maxPos[1])) (14) int dz=int(std::max(std::max(minPos[2]-point[2],0), point[2]-maxPos[2])) (15) D=dx+dy+dz (16)

[0151] The current block will only be traversed if D is less than or equal to Dist.

[0152] The above section introduced the predictive transform coding method for point cloud attribute information. The following section will introduce the lifting transform coding method for point cloud attribute information in detail.

[0153] Figure 20 illustrates the encoding process of the lifting transform. As shown in Figure 20, the lifting transform also predicts and encodes point cloud attributes based on LOD (Level of Detail). The difference from the prediction transform described above is that the lifting transform divides the LOD into high and low layers. Then, prediction is performed in reverse order of the LOD generation layers, and an update operator is introduced during the prediction process to update the quantization weights of points in the lower-level LOD to improve prediction accuracy. Attribute values ​​of points in the lower-level LOD are frequently used for predicting attribute values ​​of points in the higher-level LOD; therefore, points in the lower-level LOD should have a greater influence. Referring again to Figure 20, the encoding method of the lifting transform can be divided into three steps: segmentation, prediction, and update. These three steps will be described in detail below.

[0154] Step 1: Segmentation Process

[0155] The segmentation process divides the complete LOD layer into a low LOD layer L(N) and a high LOD layer H(N). If a point cloud has three LOD layers, i.e., (LOD... l ) l＝0,1,2 After partitioning, LOD2 is a higher LOD level, denoted as H(N), (LOD l ) l＝0,1 The lower LOD level is denoted as L(N).

[0156] Step 2: Prediction Process

[0157] In the higher-level LOD, the attribute information of the nearest neighbor point in the lower level is selected as the attribute prediction value P(N) of the current point to be encoded. The prediction residual D(N) is denoted as: D(N)=H(N)―P(N) (17)

[0158] Step 3: Update Process

[0159] The attribute prediction residual D(N) in the high-level LOD is updated to obtain U(N), and the attribute values ​​of the points in the low-level LOD are boosted using U(N), as shown in the following formula: L′(N)=L(N)+U(N) (18)

[0160] The above process will iterate continuously from high to low LOD until the lowest LOD level is reached.

[0161] The LOD-based prediction scheme gives greater influence to points in the lower LOD layers. The lifting wavelet transform method introduces quantization weights and updates the prediction residuals based on the prediction residual D(N) and the distances between predicted points and their neighbors. Finally, the quantization weights from the transform process are used for adaptive quantization of the prediction residuals. It's important to note that the quantization weight value for each point can be determined through geometric reconstruction at the decoding end; therefore, the quantization weights should not be encoded.

[0162] The RAHT transform employs the Haar wavelet transform, which can transform the attribute information of point clouds from the spatial domain to the frequency domain, thereby further reducing the correlation between point cloud attribute information. Figure 21 shows an example of the RAHT transform process. As shown in Figure 21, RAHT performs wavelet transform based on the hierarchical structure of an octree, thus associating attribute information with octree nodes. For the attribute information of occupied nodes within the same parent node, the transformation is performed recursively from bottom to top. For each node in each layer, the transformation is performed in the three dimensions of x, y, and z (see Figure 22), until the root node of the octree is reached. During the hierarchical transformation process, the direct current (DC) coefficients (or low-pass coefficients) obtained after the transformation of nodes in the same layer are passed to the nodes in the next layer for further transformation, while all alternating current (AC) coefficients (or high-pass coefficients) are quantized and encoded.

[0163] Figure 23 is a schematic diagram of the RAHT transform and the inverse RAHT transform. Assume g′ L,2x,y,z Let g′L,2x+1,y,z be the DC coefficients of two neighboring points in layer L. After the RAHT transformation, the information of layer L-1 is the AC coefficients f′. L―1,x,y,z and DC coefficient g′ L―1,x,y,z f′ L―1,x,y,z No further transformation will be performed; quantization encoding will proceed directly. L―1,x,y,z The process will continue searching for neighboring points for transformation. If no neighboring points are found, then g′ can be transformed. L―1,x,y,z The transformation is directly passed to layer L-2. In other words, the RAHT transformation is only effective for nodes with neighbors; nodes without neighbors are directly passed to the next higher layer. During the above transformation process, g′ L,2x,y,z The weights corresponding to g′L,2x+2,y,z (the weights can be determined based on the number of non-empty child nodes within the node) are w′ respectively. L,2x,y,z and w′L,2x+1,y,z (abbreviated as w′0 and w′1), g′ L―1,x,y,z The weight is w′ L―1,x,y,z Then the general transformation formula for the RAHT transform is:

[0164] In equation (19), T w0,w1 The transformation matrix can be determined based on equation (20):

[0165] The transformation matrix is ​​adaptively updated as the weights corresponding to each point change. The above transformation process is continuously iterated and updated according to the partitioning structure of the octree until the root node of the octree is reached.

[0166] Based on the RAHT transform, attribute information can be predicted intra-frame using RAHT, meaning that attribute information can be predicted intra-frame using RAHT combined with transform coding. This coding mode will be described in detail below.

[0167] As shown in Figure 24, the RAHT transform is based on the hierarchical order of an octree, continuously transforming from the voxel level until the root node is reached, thus completing the hierarchical transform coding of the entire attribute information. In RAHT intra-frame prediction combined with transform coding, attribute information can also be predicted and transformed based on the hierarchical order of the octree. The difference is that the RAHT intra-frame prediction combined with transform coding process can continuously transform from the root node down to the voxel level. During each RAHT transform, attribute information can be predicted and transformed based on 2×2×2 blocks.

[0168] The structure of the attribute information encoding block can be seen in Figure 24. In Figure 24, the dark gray block is the current block to be encoded, and the light gray block is the neighboring block that is coplanar and collinear with the current block. The attribute information of the current block can be normalized based on equations (21) to (23): A node =∑ p∈node attribute(p) (21) w node =∑ p∈node 1=#{p∈node} (22) a node =A node / w node (twenty three)

[0169] Specifically, the attribute information of the current block can first be obtained based on the attribute information of the nodes in the current block, i.e., A. node For example, a simple summation operation can be performed on the attribute information of the nodes in the current block to determine A. node Then, we can use the attribute information of the current block and the number of nodes in the current block (i.e., w) node The mean value 'a' of the attribute information of the current block is obtained by normalization. node Next, the average value of the attribute information of the current block can be used for transformation encoding.

[0170] Figure 25 illustrates the overall process of RAHT intra-frame prediction combined with transform coding for attribute information. In Figure 25(d), the attribute information of the current block is shown, and in Figure 25(e), the attribute information of the predicted block is shown by linear weighted fitting using the neighborhood attribute information of the current block. Then, attribute transformations are performed on the attribute information of the current block and the attribute information of the predicted block to obtain DC coefficients and AC coefficients, respectively. Then, predictive coding is performed on the AC coefficients. The attribute information of the predicted block is obtained by linear fitting based on the method shown in Figure 26.

[0171] Figure 26 is an example of a linear fitting method for the neighborhood attribute information of the current block. As shown in Figure 26, firstly, 19 neighborhood blocks of the current block can be determined. Secondly, the attribute information of each sub-block can be linearly weighted and predicted using the spatial geometric distance between the neighborhood blocks and each sub-block in the current block to obtain the attribute information of the predicted block. Then, the attribute information of the predicted block can be transformed. For example, equations (24) to (26) in Figure 27 can be used to predict and transform the attribute information (equation (24) represents the transformation method of the attribute information of the current block, equation (25) represents the transformation method of the attribute information of the predicted block, and equation (26) outputs the predicted residual information):

[0172] When performing inter-frame predictive coding of attribute information, if inter-frame predictive coding is initiated, the RAHT attribute transform coding structure is first constructed based on the geometric information of the current node. This involves continuously merging nodes at the voxel level until the root node of the entire RAHT transform tree is obtained, thus yielding the hierarchical transform coding structure corresponding to the attribute information. Next, based on the RAHT transform structure, the root node is divided to obtain N child nodes (N less than or equal to 8) for each node. Unlike the RAHT intra-frame prediction combined with transform coding mode, the RAHT inter-frame prediction combined with transform coding mode utilizes the node information of the reference frame. For example, the attribute information of the N child nodes of the current node can be subjected to RAHT transform to obtain DC and AC coefficients. Then, inter-frame prediction can be performed on the AC coefficients of the N child nodes in the following manner.

[0173] For example, if the inter-frame predicted node of the current node is valid (i.e., the co-node of the current node exists in the reference frame), then the attribute information of the predicted node is directly used as the attribute predicted value of the current node.

[0174] For example, if the current node can find a node in the reference frame's buffer that is exactly at the same position as the current node (i.e., a sibling node of the current node exists in the reference frame), then the attribute prediction values ​​of the AC coefficients of the current node's N child nodes can be determined based on the AC coefficients of the M child nodes contained in that sibling node. For instance, if the AC coefficient of the inter-frame prediction node corresponding to a certain child node is not zero, then the AC coefficient of that inter-frame prediction node is directly used as the prediction value of that child node; if the AC coefficient of the inter-frame prediction node corresponding to a certain child node is zero, then the AC coefficient of the intra-frame prediction node corresponding to that child node can be used as the prediction value.

[0175] For example, if the inter-frame predicted node of the current node is invalid (i.e., the corresponding node of the current node in the reference frame does not exist), the attribute predicted value of the adjacent node within the frame can be used as the attribute predicted value of the current node.

[0176] Furthermore, after initiating RAHT inter-frame prediction, the optimal RAHT prediction mode can be selected for each layer. This RAHT prediction mode can be either RAHT intra-frame prediction mode or RAHT inter-frame prediction mode. If the cost of RAHT intra-frame prediction mode is less than the cost of RAHT inter-frame prediction mode, then RAHT intra-frame prediction can be performed on the current layer; otherwise, RAHT inter-frame prediction is performed.

[0177] It's also understandable that, for the G-PCC codec framework, the general testing conditions are as follows:

[0178] (1) There are 4 types of test conditions:

[0179] Condition 1: Geometric position is finitely lossy, and properties are lossy;

[0180] Condition 2: Geometric position is lossless, but attributes are lossy;

[0181] Condition 3: Geometric position is lossless, but attribute loss is limited;

[0182] Condition 4: Geometric position is lossless and attribute is lossless.

[0183] (2) The general test sequence includes four types: Cat1A, Cat1B, Cat3-fused, and Cat3-frame. Among them, Cat3-frame point cloud only contains reflectivity attribute information, Cat1A and Cat1B point clouds only contain color attribute information, and Cat3-fused point cloud contains both color and reflectivity attribute information.

[0184] (3) Technical approach: There are two approaches, distinguished by the algorithm used in geometric compression.

[0185] Technical route 1: Octree coding branch.

[0186] At the encoding end, the bounding box is divided into sub-cubes in sequence. The non-empty sub-cubes (containing points in the point cloud) are further divided until the leaf nodes are 1×1×1 unit cubes. In the case of geometric lossless encoding, the number of points contained in the leaf nodes needs to be encoded. Finally, the encoding of the geometric octree is completed, and a binary code stream is generated.

[0187] On the decoding end, the decoding end obtains the placeholder code of each node by continuously parsing in the order of breadth-first traversal, and then divides the nodes in turn until a 1×1×1 unit cube is obtained and the division stops. In the case of geometric lossless decoding, it is necessary to parse the number of points contained in each leaf node, and finally recover the geometric reconstruction point cloud information.

[0188] Technical route 2: Predictive tree coding branch.

[0189] On the encoding side, the prediction tree structure is built using two different methods: one based on KD-Tree (high latency, slow mode) and the other using LiDAR calibration information (low latency, fast mode). Using LiDAR calibration information, each point can be assigned to a different laser, and the prediction tree structure is built according to the different lasers. Next, based on the prediction tree structure, each node in the prediction tree is traversed. By selecting different prediction modes, the geometric position information of the node is predicted to obtain the prediction residual, and quantization parameters are used to quantize the geometric prediction residual. Finally, through continuous iteration, the prediction residual of the prediction tree node position information, the prediction tree structure, and the quantization parameters are encoded to generate a binary code stream.

[0190] At the decoding end, the decoding end continuously parses the bitstream and reconstructs the prediction tree structure. Then, it obtains the geometric position prediction residual information and quantization parameters of each prediction node through parsing. Furthermore, it performs inverse quantization on the prediction residual to recover the reconstructed geometric position information of each node, and finally completes the geometric reconstruction at the decoding end.

[0191] The preceding section detailed inter-frame prediction of point cloud attribute information based on the G-PCC encoding / decoding framework. In related technologies, the decision to enable inter-frame prediction coding for attribute information can be determined using high-level syntax elements (e.g., `attrInterPredictionEnabled`). For example, when predictively coding the attribute information in each slice, geometric information can be used at the encoder to adaptively determine whether inter-frame prediction coding is enabled for the current slice's attribute information, and the syntax element `slice_attr_inter_prediction` is passed to the decoder. Correspondingly, at the decoder, the `slice_attr_inter_prediction` syntax element is parsed to determine whether inter-frame prediction decoding is enabled for the current slice.

[0192] Figure 28 illustrates inter-frame prediction within the G-PCC encoding / decoding framework. As shown in Figure 28, when inter-frame prediction of attribute information is enabled for a slice, the corresponding reference frame information (including geometric and attribute information) is obtained in the prediction buffer. Then, as shown in Figure 29, the corresponding prediction unit is obtained in the reference frame using the spatial position of the current slice. Continuing with Figure 28, when encoding the geometric information of the current slice, the global motion matrix is ​​used to perform motion compensation on the geometric information of the reference frame (e.g., rotation and / or translation based on the geometric information of the reference frame using the global motion matrix). Then, the motion-compensated geometric information of the reference frame is used to perform inter-frame prediction coding on the current slice. When performing inter-frame prediction coding on the attribute information of the current slice, after obtaining the prediction unit, the attribute information of the reference frame is directly used to perform inter-frame prediction coding on the current slice.

[0193] As can be seen from the above, the inter-frame predictive coding scheme for attribute information provided by the relevant technology directly uses the attribute information of the reference frame to perform inter-frame predictive coding of the attribute information of the unit to be coded (such as the strip to be coded), without performing motion compensation processing on the reference frame. This will result in a certain error between the attribute information of the unit to be coded and the attribute information in the reference frame, thereby reducing the coding efficiency of attribute information.

[0194] To address the aforementioned issues, this application provides a point cloud encoding method, comprising: performing motion compensation on a first reference point cloud based on first motion information to determine a second reference point cloud; determining predicted values ​​of attribute information or geometric information of a current point cloud based on the second reference point cloud; and determining residual values ​​of attribute information or geometric information of the current point cloud based on the predicted values ​​of attribute information or geometric information of the current point cloud.

[0195] This application embodiment also provides a point cloud decoding method, including: decoding a bitstream to determine first motion information; performing motion compensation on a first reference point cloud based on the first motion information to determine a second reference point cloud; determining predicted values ​​of attribute information or geometric information of the current point cloud based on the second reference point cloud; and determining reconstructed values ​​of attribute information or geometric information of the current point cloud based on the predicted values ​​of attribute information or geometric information of the current point cloud.

[0196] This application's embodiments introduce a motion-compensated inter-frame prediction scheme for attribute information, which helps improve the encoding and decoding efficiency of point cloud attribute information.

[0197] The point cloud decoding method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0198] Figure 30 is a flowchart illustrating the point cloud decoding method provided in an embodiment of this application. The decoding method in Figure 30 can be applied to a decoder. In some implementations, this decoding method can be applied to G-PCC. Alternatively, in other implementations, this decoding method can be applied to a geometry-based solid content test model (GES-TM).

[0199] Referring to Figure 30, in step S3010, the bitstream is decoded to determine the first motion information. The first motion information may be, for example, a motion vector or a motion matrix. The first motion information may be global motion (rigid motion) information or local motion (flexible motion) information.

[0200] In step S3020, motion compensation is performed on the first reference point cloud based on the first motion information to determine the second reference point cloud. Performing motion compensation on the first reference point cloud based on the first motion information can be understood as performing spatial transformations (such as translation and / or rotation transformations) on the geometric information of points in the first reference point cloud based on the first motion information.

[0201] The reference point cloud obtained after motion compensation of the first reference point cloud can be called the motion-compensated first reference point cloud. The motion-compensated first reference point cloud refers to the reference point cloud formed by points whose geometric information has been subjected to motion compensation (such as translation and / or rotation transformation). For example, assuming the first motion information is a global motion matrix, this global motion matrix can be used to perform rotation transformation on the geometric information of the points in the first reference point cloud to obtain the motion-compensated first reference point cloud. The specific calculation method is as follows:

[0202] in, For the global motion matrix, The geometric information (reconstructed geometric information) of the points in the first reference point cloud.

[0203] The geometric information mentioned here may include, for example, the geometric starting position information (slice_offset) of the current point cloud. As an example, the geometric information in the first reference point cloud can be offset relative to the geometric starting position information of the current point cloud to obtain the motion-compensated first reference point cloud.

[0204] Step S3020 mentions performing motion compensation on the first reference point cloud based on the first motion information. The points in the first reference point cloud are determined based on the geometric information of the current point cloud. However, if the geometric information of the points in the first reference point cloud shifts after motion compensation, it may no longer match the geometric information of the current point cloud, thus affecting the decoding efficiency.

[0205] In some implementations, at least one point in the motion-compensated first reference point cloud can be processed based on the geometric information of the current point cloud.

[0206] The processing mentioned here may include, for example, adjusting the range of geometric information values ​​of points in the motion-compensated first reference point cloud to a certain spatial range, or, in other words, performing a clipping operation (i.e., deleting these points) on points in the motion-compensated first reference point cloud that exceed a certain spatial range, thereby obtaining a second reference point cloud. This spatial range can be determined by the geometric information of the current point cloud. For example, this spatial range can be defined by a bounding box (i.e., a first bounding box) determined based on the spatial position information of the current point cloud. At least one point in the first reference point cloud mentioned above can refer to a point outside the aforementioned spatial range (determined based on the geometric information of the current point cloud). For example, processing points in the motion-compensated first reference point cloud that exceed the first bounding box. Restricting points in the second reference point cloud to the same bounding box as points in the current point cloud helps improve the decoding efficiency of the attribute information of the current point cloud.

[0207] In some scenarios, the geometric position information of processed points in the second reference point cloud (such as points with adjusted geometric information) may overlap with that of unprocessed points in the second reference point cloud. In this case, the average of the attribute information of these two points can be used as the predicted attribute information of the points in the current point cloud (a similar operation can be applied to geometric information).

[0208] In some implementations, a clip operation can be performed on the points in the motion-compensated first reference point cloud to make the geometric information in the second reference point cloud non-negative.

[0209] In step S3030, the predicted values ​​of the attribute information or geometric information of the current point cloud are determined based on the second reference point cloud.

[0210] For example, the attribute information of the second reference point cloud can be used as the predicted value of the attribute information of the current point cloud. Similarly, the geometric information of the second reference point cloud can be used as the predicted value of the geometric information of the current point cloud.

[0211] In step S3040, the reconstructed values ​​of the attribute information or geometric information of the current point cloud are determined based on the predicted values ​​of the attribute information or geometric information of the current point cloud.

[0212] After obtaining the predicted values ​​of the attribute information of the current point cloud, the reconstructed values ​​of the attribute information of the current point cloud can be determined based on the predicted values ​​and the residual values ​​of the attribute information of the current point cloud. For example, the reconstructed values ​​of the attribute information of the current point cloud can be obtained by summing the predicted values ​​and the residual values ​​of the attribute information of the current point cloud.

[0213] After obtaining the predicted values ​​of the geometric information of the current point cloud, the reconstructed values ​​of the geometric information of the current point cloud can be determined based on the predicted values ​​and the residual values ​​of the geometric information of the current point cloud. For example, the reconstructed values ​​of the geometric information of the current point cloud can be obtained by summing the predicted values ​​and the residual values ​​of the geometric information of the current point cloud.

[0214] It should be understood that steps S3010 to S3040 describe the decoding process of the attribute information (or geometric information) of the current point cloud. This application embodiment does not limit the specific decoding method for the attribute information. For example, the decoding methods mentioned above based on lifting transform, predictive transform, or RAHT transform can be used. Before decoding the attribute information of the current point cloud, the geometric information of the current point cloud can be decoded first.

[0215] The geometric and attribute information of the current point cloud can share the same motion information to perform motion compensation on the reference point cloud, as described above, or they can each have their own motion information to perform motion compensation on the reference point cloud.

[0216] For example, motion compensation can be performed on the first reference point cloud based on the second motion information (which can be global motion information or local motion information, such as the global motion matrix) to determine the motion-compensated reference point cloud; then, the predicted value of the geometric information of the current point cloud can be determined based on the motion-compensated first reference point cloud.

[0217] During inter-frame decoding of the current point cloud's attribute information, the motion information used in the inter-frame decoding of geometric information can be reused. This eliminates the need to transmit two types of motion information separately through the bitstream, thus reducing bit overhead. Alternatively, in some implementations, the geometric and attribute information of the current point cloud can use their respective motion information to perform motion compensation on the first reference point cloud during inter-frame decoding. In this case, the first and second motion information can be transmitted separately through the bitstream. Using different motion information for inter-frame decoding of geometric and attribute information improves decoding flexibility.

[0218] If the inter-frame decoding of geometric information uses second motion information to perform motion compensation on the first reference point cloud, a similar operation can be used in the inter-frame prediction process of the geometric information of the current point cloud to restrict the motion-compensated first reference point cloud to a certain spatial range. For example, a third reference point cloud can be determined from the motion-compensated first reference point cloud based on the geometric information of the current point cloud; then, the predicted value of the geometric information of the current point cloud can be determined based on the third reference point cloud. The third reference point cloud mentioned here can, for example, be located in the same bounding box as the points in the current point cloud, and the bounding box is determined based on the geometric information of the current point cloud (e.g., it can be the bounding box in which the current point cloud is located). Restricting the points in the reference point cloud and the points in the current point cloud to the same bounding box helps to improve the decoding efficiency of the geometric information of the current point cloud.

[0219] In some implementations, the method in Figure 30 further includes parsing the bitstream to determine first identification information. The first identification information indicates the inter-frame prediction mode of the attribute information of the first slice of the current point cloud. The first identification information can be carried at any position in the bitstream. For example, it can be carried in the attribute brick header (ABH). The first identification information can indicate that the inter-frame prediction mode of the attribute information of the first slice is either a first inter-frame prediction mode or a second inter-frame prediction mode, where the first inter-frame prediction mode is a motion-compensated inter-frame prediction mode and the second inter-frame prediction mode is a non-motion-compensated inter-frame prediction mode. If the first identification information indicates that the inter-frame prediction mode of the attribute information of the current point cloud is the first inter-frame prediction mode, then inter-frame prediction of the first slice can be performed as shown in Figure 30. If the first identification information indicates that the inter-frame prediction mode of the attribute information of the current point cloud is the second inter-frame prediction mode, then inter-frame prediction of the first slice can be performed as provided by related technologies (as shown in Figure 28). In some implementations, the first identification information can be represented using the syntax element `motionCompensatedAttrInterPred`. When `motionCompensatedAttrInterPred` takes the first value (e.g., 1 or true), the inter-frame prediction mode for the attribute information of the first stripe is the first inter-frame prediction mode; when `motionCompensatedAttrInterPred` takes the second value (e.g., 0 or false), the inter-frame prediction mode for the attribute information of the first stripe is the second inter-frame prediction mode. Alternatively, the first identifier information can be represented by the syntax element `enableMotionCompensated`. When `enableMotionCompensated` takes the first value (e.g., 1 or true), the inter-frame prediction mode for the attribute information of the first stripe is the first inter-frame prediction mode; when `enableMotionCompensated` takes the second value (e.g., 0 or false), the inter-frame prediction mode for the attribute information of the first stripe is the second inter-frame prediction mode.

[0220] In some implementations, when the attribute information uses RAHT-based decoding, a second identifier can be introduced. This second identifier can be obtained from the decoded bitstream. This second identifier can be used to indicate the inter-frame prediction mode of the attribute information of the first RAHT layer (which can be any RAHT layer; different RAHT layers can have their own corresponding second identifiers). The inter-frame prediction mode of the attribute information of the first RAHT layer is either a third or a fourth inter-frame prediction mode. The third inter-frame prediction mode is a motion-compensated inter-frame prediction mode. The fourth inter-frame prediction mode is a non-motion-compensated inter-frame prediction mode. This second identifier can be represented by `enableMotionCompensatedLayer[lvl]`. When `enableMotionCompensatedLayer[lvl]` takes the first value (e.g., 1 or true), the inter-frame prediction mode of the attribute information of the first RAHT layer is the third inter-frame prediction mode; when `enableMotionCompensatedLayer[lvl]` takes the second value (e.g., 0 or false), the inter-frame prediction mode of the attribute information of the first RAHT layer is the fourth inter-frame prediction mode.

[0221] In some implementations, when the attribute information uses a Level of Detail (LOD)-based decoding method, a third identification information can be introduced. This third identification information can be obtained through decoding the bitstream. This third identification information can be used to indicate the inter-frame prediction mode of the attribute information of the first LOD layer (which can be any LOD layer; different LOD layers can have their own corresponding third identification information). The inter-frame prediction mode of the attribute information of the first LOD layer is either the fifth inter-frame prediction mode or the sixth inter-frame prediction mode. The fifth inter-frame prediction mode is a motion-compensated prediction mode, and the sixth inter-frame prediction mode is an inter-frame prediction mode without motion compensation.

[0222] As mentioned above, the embodiments of this application are applicable to decoding methods based on lifting transform, predictive transform, or RAHT transform. The following description uses RAHT transform as an example to illustrate the decoding method for point cloud attribute information.

[0223] After determining the second reference point cloud based on the preceding content, a RAHT transform can be performed on the first stripe to be decoded in the current point cloud to determine at least one transform block; a RAHT transform can also be performed on the second stripe in the second reference point cloud to determine at least one reference transform block. Here, the transform block corresponds to the transform coefficients of the attribute information, such as the high-pass or low-pass coefficients of the attribute information. The second stripe is determined based on the geometric information of the first stripe to be decoded. For example, the first stripe and the first stripe to be decoded may be located in the same bounding box.

[0224] In the decoding method of RAHT transform, the method of determining the predicted value of the attribute information of the current point cloud may include: predicting the current transform block according to the reference transform block (or the first reference transform block), for example, using the attribute transform coefficients of the first reference transform block as the attribute transform coefficients of the current transform block.

[0225] In the above text, performing a RAHT transformation on the second strip may yield multiple reference transformation blocks. The first reference transformation block corresponding to the current transformation block can be determined based on the geometric information of the current transformation block. For example, the first reference transformation block can be determined among multiple reference transformation blocks based on the coordinate information of the current transformation block in the RAHT structure and / or the hierarchical information of the RAHT layer.

[0226] In the RAHT transform decoding method, determining the reconstructed values ​​of the attribute information of the current point cloud can include: determining the reconstructed values ​​of the attribute transform coefficients of the current transform block based on the predicted values ​​of the attribute transform coefficients of the current transform block. For example, the predicted values ​​of the attribute transform coefficients of the current transform block and the residual values ​​of the attribute transform coefficients of the current transform block can be summed to obtain the reconstructed values ​​of the attribute transform coefficients of the current transform block. Then, an inverse transform is performed on the reconstructed values ​​of the attribute transform coefficients of the current transform block to determine the reconstructed values ​​of the attribute information of the current point cloud (or the current decoded strip).

[0227] In some implementations, the first parameter can be determined by parsing the bitstream, and then the motion information used for motion compensation can be determined based on the first parameter.

[0228] Table 2 shows the test results of the solutions provided in the embodiments of this application.

[0229] Table 2

[0230] In Table 2, `ford_01_q1mm` represents the sequence of attribute information prediction, `TMC13-v24` represents the pixel depth (bits per pixel, BPP) of attribute information encoding under related technologies, `Current` represents the BPP of attribute information encoding in this embodiment, and `Coding type` represents the type of encoded frame. As can be seen from Table 2, the BPP of attribute information encoding in this embodiment is reduced compared to the BPP of attribute information encoding under related technologies, with a reduction of approximately 1%. Therefore, the solution provided in this application can reduce the BPP of attribute information encoding, thereby helping to improve encoding efficiency.

[0231] The point cloud decoding method provided by the embodiments of this application has been described in detail above with reference to Figure 30. The point cloud encoding method provided by the embodiments of this application will be described in detail below with reference to Figure 31.

[0232] Figure 31 is a flowchart illustrating the point cloud encoding method provided in an embodiment of this application. The encoding method in Figure 31 can be applied to an encoder. In some implementations, this encoding method can be applied to G-PCC. Alternatively, in other implementations, this encoding method can be applied to a geometry-based solid content test model (GES-TM).

[0233] Referring to Figure 31, in step S3110, motion compensation is performed on the first reference point cloud based on the first motion information to determine the second reference point cloud.

[0234] The first motion information can be, for example, a motion vector or a motion matrix. The first motion information can be global motion (rigid motion) information or local motion (flexible motion) information.

[0235] Motion compensation of the first reference point cloud based on the first motion information can be understood as performing spatial transformation (such as translation and / or rotation transformation) on the geometric information of points in the first reference point cloud based on the first motion information.

[0236] The reference point cloud obtained after motion compensation of the first reference point cloud can be called the motion-compensated first reference point cloud. The motion-compensated first reference point cloud refers to the reference point cloud formed by points whose geometric information has been subjected to motion compensation (such as translation and / or rotation transformation). For example, assuming the first motion information is a global motion matrix, this global motion matrix can be used to perform rotation transformation on the geometric information of the points in the first reference point cloud to obtain the motion-compensated first reference point cloud. The specific calculation method is as follows:

[0237] in, For the global motion matrix, The geometric information (reconstructed geometric information) of the points in the first reference point cloud.

[0238] The geometric information mentioned here may include, for example, the geometric starting position information (slice_offset) of the current point cloud. As an example, the geometric information in the first reference point cloud can be offset relative to the geometric starting position information of the current point cloud to obtain the motion-compensated first reference point cloud.

[0239] Step S3110 mentions performing motion compensation on the first reference point cloud based on the first motion information. The points in the first reference point cloud are determined based on the geometric information of the current point cloud. However, if the geometric information of the points in the first reference point cloud shifts after motion compensation, it may no longer match the geometric information of the current point cloud, thus affecting the coding efficiency.

[0240] In some implementations, at least one point in the motion-compensated first reference point cloud can be processed based on the geometric information of the current point cloud.

[0241] The processing mentioned here may include, for example, adjusting the range of geometric information values ​​of points in the motion-compensated first reference point cloud to a certain spatial range, or, in other words, performing a clipping operation (i.e., deleting these points) on points in the motion-compensated first reference point cloud that exceed a certain spatial range, thereby obtaining a second reference point cloud. This spatial range can be determined by the geometric information of the current point cloud. For example, this spatial range can be defined by a bounding box (i.e., a first bounding box) determined based on the spatial position information of the current point cloud. At least one point in the first reference point cloud mentioned above can refer to a point that exceeds the aforementioned spatial range (determined based on the geometric information of the current point cloud). For example, processing points in the motion-compensated first reference point cloud that exceed the first bounding box. Restricting the points in the second reference point cloud to the same bounding box as the points in the current point cloud helps improve the encoding efficiency of the attribute information of the current point cloud.

[0242] In some scenarios, the geometric position information of processed points in the second reference point cloud (such as points with adjusted geometric information) may overlap with that of unprocessed points in the second reference point cloud. In this case, the average of the attribute information of these two points can be used as the predicted attribute information of the points in the current point cloud (a similar operation can be applied to geometric information).

[0243] In some implementations, a clip operation can be performed on the points in the motion-compensated first reference point cloud to make the geometric information in the second reference point cloud non-negative.

[0244] In step S3120, the predicted values ​​of the attribute information or geometric information of the current point cloud are determined based on the second reference point cloud.

[0245] For example, the attribute information of the second reference point cloud can be used as the predicted value of the attribute information of the current point cloud. Similarly, the geometric information of the second reference point cloud can be used as the predicted value of the geometric information of the current point cloud.

[0246] In step S3130, the residual values ​​of the attribute information or geometric information of the current point cloud are determined based on the predicted values ​​of the attribute information or geometric information of the current point cloud.

[0247] After obtaining the predicted values ​​of the attribute information of the current point cloud, the residual values ​​of the attribute information can be determined based on the predicted values ​​and the original values ​​of the attribute information. For example, the residual values ​​of the attribute information can be obtained by subtracting the predicted values ​​from the original values. Then, the residual values ​​of the attribute information can be quantized and entropy encoded.

[0248] After obtaining the predicted values ​​of the geometric information of the current point cloud, the residual values ​​of the geometric information of the current point cloud can be determined based on the predicted values ​​and the original values ​​of the geometric information. For example, the residual values ​​of the geometric information of the current point cloud can be obtained by subtracting the predicted values ​​from the original values. Then, the residual values ​​of the geometric information of the current point cloud can be quantized and entropy encoded.

[0249] It should be understood that steps S3110 to S3130 describe the encoding process of the attribute information (or geometric information) of the current point cloud. This application embodiment does not limit the specific encoding method for the attribute information. For example, the encoding methods mentioned above based on lifting transform, prediction transform, or RAHT transform can be used. Before encoding the attribute information of the current point cloud, the geometric information of the current point cloud can be encoded first.

[0250] The geometric and attribute information of the current point cloud can share the same motion information to perform motion compensation on the reference point cloud, as described above, or they can each have their own motion information to perform motion compensation on the reference point cloud.

[0251] For example, motion compensation can be performed on the first reference point cloud based on the second motion information (which can be global motion information or local motion information, such as the global motion matrix) to determine the motion-compensated reference point cloud; then, the predicted value of the geometric information of the current point cloud can be determined based on the motion-compensated first reference point cloud.

[0252] In the inter-frame coding of the current point cloud's attribute information, the motion information used in the geometric information inter-frame coding process can be reused. This eliminates the need to transmit two types of motion information separately through the bitstream, thus reducing bit overhead. Alternatively, in other implementations, the geometric and attribute information of the current point cloud can use their respective motion information to perform motion compensation on the first reference point cloud during inter-frame coding. In this case, the first and second motion information can be transmitted separately through the bitstream. Using different motion information for geometric and attribute information inter-frame coding improves coding flexibility.

[0253] If inter-frame coding of geometric information uses second motion information to perform motion compensation on the first reference point cloud, a similar operation can be used in the inter-frame prediction process of the geometric information of the current point cloud to restrict the motion-compensated first reference point cloud to a certain spatial range. For example, a third reference point cloud can be determined from the motion-compensated first reference point cloud based on the geometric information of the current point cloud; then, the predicted value of the geometric information of the current point cloud can be determined based on the third reference point cloud. The third reference point cloud mentioned here can, for example, be located in the same bounding box as the points in the current point cloud, and the bounding box is determined based on the geometric information of the current point cloud (e.g., it can be the bounding box in which the current point cloud is located). Restricting the points in the reference point cloud and the points in the current point cloud to the same bounding box helps to improve the coding efficiency of the geometric information of the current point cloud.

[0254] As mentioned earlier, related technologies can employ motion compensation schemes when performing inter-frame coding of geometric information. However, this motion compensation is only applicable to inter-frame coding of geometric information from LiDAR point clouds. For non-LiDAR point clouds, these technologies use a method of copying the geometric information of a reference point cloud for inter-frame prediction. This approach is necessary because LiDAR point clouds can provide acquisition parameter information, which can then be used for motion compensation (i.e., using this parameter information to determine motion information and then using that motion information for motion compensation). Non-LiDAR point clouds, however, lack acquisition parameter information and therefore cannot undergo motion compensation.

[0255] The first motion information and / or the second motion information mentioned in the embodiments of this application can be obtained based on the parameter information provided by the lidar point cloud. Alternatively, the first motion information and / or the second motion information provided in the embodiments of this application can be determined based on the difference between the current point cloud and the reference point cloud.

[0256] Taking the first motion information as an example, it can be determined based on the difference between the geometric information of the current point cloud and the geometric information of the first reference point cloud. This difference can be, for example, the mean square error (MSE), or other types of difference metrics. As an example, the first motion information can be the motion matrix that minimizes the difference between the geometric information of the current point cloud and the geometric information of the first reference point cloud.

[0257] Taking the first motion information as an example, it can be determined based on the difference between the attribute information of the current point cloud and the attribute information of the first reference point cloud. This difference can be, for example, MSE (Mean Separation of Elements), or other types of difference metrics. As an example, the first motion information can be the motion matrix that minimizes the difference between the attribute information of the current point cloud and the attribute information of the first reference point cloud.

[0258] Taking the first motion information as an example, it can be determined based on a first difference between the geometric information of the current point cloud and the geometric information of the first reference point cloud, and a second difference between the attribute information of the current point cloud and the attribute information of the first reference point cloud. The first and / or second differences mentioned here can be, for example, MSE, or other types of difference metrics. As an example, the first motion information can be a motion matrix that minimizes the weighted sum of the first and second differences.

[0259] In some implementations, the method of Figure 31 further includes writing first identification information into the bitstream. The first identification information is used to indicate the inter-frame prediction mode of the attribute information of the first strip of the current point cloud. The first identification information can be carried at any position in the bitstream. For example, the first identification information can be carried in the attribute brick header (ABH). The first identification information can indicate that the inter-frame prediction mode of the attribute information of the first strip is a first inter-frame prediction mode or a second inter-frame prediction mode, where the first inter-frame prediction mode is a motion-compensated inter-frame prediction mode and the second inter-frame prediction mode is a non-motion-compensated inter-frame prediction mode. If the first identification information indicates that the inter-frame prediction mode of the attribute information of the current point cloud is the first inter-frame prediction mode, then inter-frame prediction of the first strip can be performed as shown in Figure 30. If the first identification information indicates that the inter-frame prediction mode of the attribute information of the current point cloud is the second inter-frame prediction mode, then inter-frame prediction of the first strip can be performed as provided by related technologies (as shown in Figure 28). In some implementations, the first identification information can be represented using the syntax element `motionCompensatedAttrInterPred`. When `motionCompensatedAttrInterPred` takes the first value (e.g., 1 or true), the inter-frame prediction mode for the attribute information of the first stripe is the first inter-frame prediction mode; when `motionCompensatedAttrInterPred` takes the second value (e.g., 0 or false), the inter-frame prediction mode for the attribute information of the first stripe is the second inter-frame prediction mode. Alternatively, the first identifier information can be represented by the syntax element `enableMotionCompensated`. When `enableMotionCompensated` takes the first value (e.g., 1 or true), the inter-frame prediction mode for the attribute information of the first stripe is the first inter-frame prediction mode; when `enableMotionCompensated` takes the second value (e.g., 0 or false), the inter-frame prediction mode for the attribute information of the first stripe is the second inter-frame prediction mode.

[0260] At the encoding end, the value of the first identifier information can be determined based on a rate-distortion optimization algorithm. For example, inter-frame prediction of the attribute information of the first stripe can be performed according to the first inter-frame prediction mode to determine the first rate-distortion cost, and inter-frame prediction of the attribute information of the first stripe can be performed according to the second inter-frame prediction mode to determine the second rate-distortion cost. Then, based on the first and second rate-distortion costs, the inter-frame prediction mode of the attribute information of the first stripe can be determined from the first and second inter-frame prediction modes.

[0261] In some implementations, when the attribute information uses RAHT-based encoding, the second identifier information can be written into the bitstream. This second identifier information can be used to indicate the inter-frame prediction mode of the attribute information of the first RAHT layer (which can be any RAHT layer, and different RAHT layers can have their own corresponding second identifier information). The inter-frame prediction mode of the attribute information of the first RAHT layer is either the third inter-frame prediction mode or the fourth inter-frame prediction mode. The third inter-frame prediction mode is an inter-frame prediction mode based on motion compensation. The fourth inter-frame prediction mode is an inter-frame prediction mode without motion compensation. This second identifier information can be represented by enableMotionCompensatedLayer[lvl]. When enableMotionCompensatedLayer[lvl] takes the first value (such as 1 or true), the inter-frame prediction mode of the attribute information of the first RAHT layer is the third inter-frame prediction mode; when enableMotionCompensatedLayer[lvl] takes the second value (such as 0 or false), the inter-frame prediction mode of the attribute information of the first RAHT layer is the fourth inter-frame prediction mode.

[0262] At the encoding end, the value of the second identifier information can be determined based on a rate-distortion optimization algorithm. For example, inter-frame prediction of the attribute information of the first RAHT layer can be performed based on a third inter-frame prediction mode to determine the third rate-distortion cost, and inter-frame prediction of the attribute information of the first RAHT layer can be performed based on a fourth inter-frame prediction mode to determine the fourth rate-distortion cost. Then, based on the third and fourth rate-distortion costs, the inter-frame prediction mode of the attribute information of the first RAHT layer can be determined from the third and fourth inter-frame prediction modes.

[0263] In some implementations, when the attribute information uses a Level-of-Depth (LOD) based encoding method, third identification information can be written into the bitstream. This third identification information can be used to indicate the inter-frame prediction mode of the attribute information of the first LOD layer (which can be any LOD layer, and different LOD layers can have their own corresponding third identification information). The inter-frame prediction mode of the attribute information of the first LOD layer is either the fifth inter-frame prediction mode or the sixth inter-frame prediction mode. The fifth inter-frame prediction mode is a motion-compensated prediction mode, and the sixth inter-frame prediction mode is an inter-frame prediction mode without motion compensation.

[0264] At the encoding end, the value of the third identification information can be determined based on a rate-distortion optimization algorithm. For example, inter-frame prediction of the attribute information of the first LOD layer can be performed based on the fifth inter-frame prediction mode to determine the fifth rate-distortion cost; inter-frame prediction of the attribute information of the first LOD layer can be performed based on the sixth inter-frame prediction mode to determine the sixth rate-distortion cost; based on the fifth and sixth rate-distortion costs, the inter-frame prediction mode corresponding to the first LOD layer can be determined from the fifth and sixth inter-frame prediction modes.

[0265] As mentioned above, the inter-frame prediction mode for attribute information can be determined based on the rate-distortion cost. The rate-distortion cost can be determined based on the distortion parameters, bit rate parameters, and Lagrange factors of their respective inter-frame prediction modes. For example, for the first inter-frame prediction mode, the distortion parameters can be determined based on the reconstructed attribute information and the original attribute information of the first inter-frame prediction mode; then, the number of coded bits required for the first inter-frame prediction mode can be determined, thereby determining the bit rate parameters; then, the first rate-distortion cost of the first inter-frame prediction mode can be determined based on the following formula (27): J=D+λ×R (27)

[0266] Where J represents the rate-distortion cost, D represents the distortion parameter, R represents the bit rate parameter, and λ represents the Lagrange factor.

[0267] The Lagrange factor can be determined based on the quantization parameters of the current point cloud's attribute information. For example, the value of λ can be determined based on the following formula (28): λ=2^((QP-4) / 6)×N (28)

[0268] Here, QP represents the quantification parameter of attribute information. N can be a predefined value. The value of N can be associated with the type of attribute information. For example, the value of N can be different for reflectance and color information.

[0269] As mentioned earlier, the embodiments of this application are applicable to encoding methods based on lifting transform, predictive transform, or RAHT transform. The following description uses RAHT transform as an example to illustrate the encoding method for point cloud attribute information.

[0270] After determining the second reference point cloud based on the preceding content, a RAHT transform can be performed on the first stripe to be encoded in the current point cloud to determine at least one transform block; a RAHT transform can also be performed on the second stripe in the second reference point cloud to determine at least one reference transform block. Here, the transform blocks correspond to the transform coefficients of the attribute information, such as the high-pass or low-pass coefficients of the attribute information. The second stripe is determined based on the geometric information of the first stripe to be encoded. For example, the first stripe and the first stripe to be encoded may be located in the same bounding box.

[0271] In the RAHT transform encoding method, the method of determining the predicted value of the attribute information of the current point cloud can include: predicting the current transform block based on the reference transform block (or the first reference transform block), for example, using the attribute transform coefficients of the first reference transform block as the attribute transform coefficients of the current transform block.

[0272] In the above text, performing a RAHT transformation on the second strip may yield multiple reference transformation blocks. The first reference transformation block corresponding to the current transformation block can be determined based on the geometric information of the current transformation block. For example, the first reference transformation block can be determined among multiple reference transformation blocks based on the coordinate information of the current transformation block in the RAHT structure and / or the hierarchical information of the RAHT layer.

[0273] In the RAHT transform encoding method, determining the residual value of the attribute information of the current point cloud can include: determining the residual value of the attribute transform coefficients of the current transform block based on the predicted value and the original value of the attribute transform coefficients of the current transform block. For example, the residual value of the attribute transform coefficients of the current transform block can be obtained by subtracting the predicted value and the original value of the attribute transform coefficients of the current transform block. Then, the residual value of the attribute transform coefficients of the current transform block is quantized and entropy encoded.

[0274] In some implementations, the first parameter can be determined based on the motion information used for motion compensation; then the first parameter is written into the bitstream.

[0275] The embodiments of this application are described in more detail below with specific examples. It should be noted that the examples below are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific numerical values ​​or scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or variations based on the examples given below, and such modifications or variations also fall within the scope of the embodiments of this application.

[0276] In related technologies, when inter-frame prediction of attribute information is enabled for a slice, the corresponding reference point cloud information (including geometric and attribute information) is obtained in the prediction buffer. Then, as shown in Figure 29, the corresponding prediction unit is obtained in the reference point cloud using the spatial position of the current slice. Referring again to Figure 28, when encoding the geometric information of the current slice, a global motion matrix (e.g., rotation and translation) is used to perform motion compensation on the geometric information of the reference point cloud. Then, the motion-compensated geometric information of the reference point cloud is used to perform inter-frame prediction coding on the current slice. When performing inter-frame prediction coding on the attribute information of the current slice, after obtaining the prediction unit, the attribute information of the reference point cloud is directly used to perform inter-frame prediction coding on the current slice. That is to say, when performing inter-frame prediction on the point cloud attribute information, the first reference point cloud (without motion compensation) in the prediction buffer is used to predict the point cloud attribute information. However, related technologies do not consider whether it is appropriate to use the first reference point cloud for inter-frame prediction, which to some extent limits the coding performance of the point cloud. The following sections will provide a detailed description of the scheme provided in this example, focusing on both the encoding and decoding ends.

[0277] The encoding scheme is as follows:

[0278] Step 1: Use the geometric information of the reference point cloud to perform inter-frame predictive coding of the current frame. When performing inter-frame predictive coding of the geometric information of the current slice, the global motion matrix is ​​used to perform motion compensation on the reference point cloud, and the motion-compensated reference point cloud is used to perform inter-frame predictive coding of the geometric information of the current frame;

[0279] Step 2: After completing the geometric information encoding, when performing inter-frame prediction encoding on the attribute information of the current slice, the corresponding prediction unit is obtained in the reference point cloud using the spatial location of the current slice, as shown in Figure 29.

[0280] Step 3: After obtaining the prediction unit, the geometric information of the reference point cloud is motion compensated using the global motion matrix. That is, the geometric information of the reference point cloud is spatially rotated using the global motion matrix. The specific calculation method is as follows:

[0281] Where (x,y,z) represents the reconstructed geometric information of the reference point cloud. This is the global motion matrix.

[0282] It's important to note that when using the global motion matrix to perform motion compensation on the geometric information of the reference point cloud, the geometric information of the reference point cloud is first offset relative to the geometric starting position (slice_offset) of the current slice to obtain the prediction unit of the current slice to be encoded. Secondly, after performing motion compensation using the global motion matrix, it is necessary to ensure that the final reference geometric information (x′, y′, z′) are all non-negative.

[0283] Step 4: After performing global motion compensation on the geometric information of the reference point cloud, the reference point cloud is used to perform inter-frame predictive coding on the attribute information of the current frame.

[0284] The decoding solution is as follows:

[0285] Step 1: Use the geometric information of the reference point cloud to perform inter-frame prediction decoding of the current frame. When performing inter-frame prediction decoding of the geometric information of the current slice, the global motion matrix is ​​used to perform motion compensation on the reference point cloud, and the motion-compensated reference point cloud is used to perform inter-frame prediction decoding of the geometric information of the current frame.

[0286] Step 2: After completing the geometric information decoding, when performing inter-frame prediction decoding on the attribute information of the current slice, the corresponding prediction unit is obtained in the reference point cloud using the spatial position of the current slice, as shown in Figure 29.

[0287] Step 3: After obtaining the prediction unit, the geometric information of the reference point cloud is motion compensated using the global motion matrix. That is, the geometric information of the reference point cloud is spatially rotated using the global motion matrix. The specific calculation method is as follows:

[0288] Where (x,y,z) represents the reconstructed geometric information of the reference point cloud. This is the global motion matrix.

[0289] It's important to note that when using the global motion matrix to perform motion compensation on the geometric information of the reference point cloud, the geometric information of the reference point cloud is first offset relative to the geometric starting position (slice_offset) of the current slice to obtain the prediction unit of the current slice to be encoded. Secondly, after performing motion compensation using the global motion matrix, it is necessary to ensure that the final reference geometric information (x′, y′, z′) are all non-negative.

[0290] Step 4: After performing global motion compensation on the geometric information of the reference point cloud, the reference point cloud is used to perform inter-frame prediction decoding on the attribute information of the current frame.

[0291] The syntax elements in the attribute data unit header are as follows:

[0292] The syntax elements from the fourth to the third to last row of the above syntax elements are new syntax elements introduced in this example based on the syntax elements provided by related technologies (motionCompensatedAttrInterPred). According to this syntax element, when inter-frame predictive coding is enabled in the current slice, this syntax element determines whether motion compensation is performed on the reference point cloud. When this syntax element is true, motion compensation is enabled; otherwise, it is not enabled. The default is false, meaning motion compensation is not enabled by default.

[0293] In the example scheme described above, the global motion matrix (GMM) in the geometric coding scheme is used to perform motion compensation on the reference point cloud. This makes the reference point cloud more similar to the geometric and attribute information of the current frame, thereby improving the attribute coding performance of the current frame. This example can also use a GMM that does not rely on geometric information. That is, when geometric information is encoded at the encoder, the geometric information of the current frame, the attribute information to be encoded, and the geometric and attribute information of the reference point cloud are used. The MSE of the attribute information is used as a metric to calculate the global motion matrix between the reference point cloud and the current frame to be encoded, i.e., the global motion matrix of the attribute information. Finally, this global motion matrix is ​​used to perform motion compensation on the geometric information of the reference point cloud, and this motion matrix is ​​passed to the decoder. The decoder uses the parsed motion matrix to reconstruct and restore the attribute information of the current slice.

[0294] In some implementations, a rate-distortion optimization algorithm is introduced at the encoding end. The algorithm adaptively determines whether motion compensation is enabled for the current slice. That is, there are two predictive coding modes to choose from: after motion compensation or with the original reference point cloud geometric information, the optimal coding mode is selected using the Lagrange comprehensive cost factor. The cost is calculated as follows: J=D+λxR (31)

[0295] Where J represents the rate-distortion cost, D represents the distortion parameter, R represents the bit rate parameter, and λ represents the Lagrange factor.

[0296] The Lagrange factor can be determined based on the quantization parameters of the attribute information of the current layer node. For example, the value of λ can be determined based on the following formula (30):

[0297] Here, QP represents the quantification parameter of attribute information. N can be a predefined value. The value of N can be associated with the type of attribute information. For example, the value of N can be different for reflectance and color information.

[0298] This example can encode the global motion matrix of attribute information separately, or it can use the global motion matrix of geometric information.

[0299] The syntax elements in the corresponding attribute data cell header are as follows:

[0300] At the decoding end, the above syntax elements are parsed. The syntax elements from the tenth to the third-to-last line are new syntax elements introduced in this example based on the syntax elements provided by related technologies. According to these syntax elements, when `enableMotionCompensated` is `True`, it indicates that motion compensation is performed on the reference point cloud; otherwise, it indicates that no motion compensation is performed. This scheme can encode the global motion matrix vector of attributes separately, and can also utilize the global motion matrix vector of geometry.

[0301] In some implementations, when performing RAHT inter-frame predictive coding on attribute information, the encoder introduces a rate-distortion optimization algorithm to adaptively determine whether motion compensation is enabled for different RAHT layers of the current slice. That is, there are two predictive coding modes to choose from: after motion compensation or with the original reference point cloud geometric information, the optimal coding mode is selected using the Lagrange comprehensive cost factor. The cost is calculated as follows: J=D+λxR (33)

[0302] Where J represents the rate-distortion cost, D represents the distortion parameter, R represents the bit rate parameter, and λ represents the Lagrange factor.

[0303] The Lagrange multiplier can be determined based on the quantization parameters of the attribute information of the current layer node. For example, the value of λ can be determined based on the following formula:

[0304] Here, QP represents the quantification parameter of attribute information. N can be a predefined value. The value of N can be associated with the type of attribute information. For example, the value of N can be different for reflectance and color information.

[0305] Finally, the encoding mode of the different RAHT layers of the current slice, enable Motion Compensated Layer[lvl], needs to be passed to the decoder. When enable Motion Compensated Layer[lvl] is True, it means that motion compensation of the reference point cloud information is required when decoding the attributes of the lvl layer; otherwise, it means that no motion compensation is performed. The decoder adaptively selects either the motion-compensated reference point cloud information or the original reference point cloud information for inter-frame prediction decoding of attributes based on the mode of each layer.

[0306] Similarly, a syntax element `enableMotionCompensated` needs to be added to the high-level syntax of each slice's properties to specify whether motion compensation is performed on the reference point cloud. When `enableMotionCompensated` is True, it means that motion compensation is performed on the reference point cloud; otherwise, it means that motion compensation is not performed.

[0307] On the encoding side, enableMotionCompensated is False if reference point cloud compensation is not enabled in each layer of RAHT, otherwise it is True.

[0308] The syntax elements in the corresponding attribute data cell header are as follows:

[0309] In some implementations, the above syntax elements are parsed at the decoding end. The syntax elements from the eleventh to the third-to-last line of the above syntax elements are new syntax elements introduced in this example based on the syntax elements provided by related technologies. According to this syntax element, when the point cloud attribute information uses inter-frame prediction (based on disableAttrInterPred), this example will determine the target prediction mode from two prediction modes, or in other words, determine whether to perform inter-frame prediction of attribute information based on the motion-compensated reference point cloud (based on enableMotionCompensated). If enableMotionCompensated is true, it means that inter-frame prediction of attribute information is performed based on the motion-compensated reference point cloud. Then, the global motion matrix corresponding to the current slice and the decoding mode of each RAHT layer (based on enableMotionCompensatedLayer[lvl]) are parsed. This syntax element specifies whether motion compensation of the reference point cloud is required when performing inter-frame prediction decoding of the attribute information of the lvl layer. If enableMotionCompensatedLayer[lvl] is true, motion compensation is required for the current RAHT layer; otherwise, motion compensation is not required. Finally, the solution from this example is used to reconstruct the property information of the current slice.

[0310] This example proposes a method to obtain a global motion matrix vector based on the geometric and attribute information of a reference point cloud and the current frame to be encoded, and then use this global motion vector to perform motion compensation on the reference point cloud. This example is independent of whether the data belongs to a LiDAR point cloud or whether point cloud data acquisition parameters are provided. This example uses the MSE (Mean Sequence Size Equation) of the geometric and attribute information of the reference point cloud and the frame to be encoded as a metric to ultimately obtain the global motion matrix vector of the reference point cloud.

[0311] This example is further extended to attribute information encoding. When performing inter-frame predictive coding on attribute information, the attribute information of the reference point cloud and the attribute information of the code frame are used to perform motion compensation on the attribute information of the reference point cloud, thereby improving the encoding efficiency of attribute information. Finally, the global motion matrix vector of the attribute information needs to be passed to the decoding end.

[0312] This example proposes a clip operation. After motion compensation is performed on the geometric information of the reference point cloud and the reference point cloud for the attribute information encoding part, a clip operation is required to restrict the reference point cloud and the current frame to the same slice bounding box, thereby improving the encoding efficiency of geometric and attribute information.

[0313] This example further optimizes the encoding unit of the global motion matrix vector: Currently, the geometric global motion matrix vector is a frame-level parameter, and each slice directly reuses the frame-level motion matrix vector. This scheme further optimizes the encoding unit of the global motion matrix vector to the slice encoding unit or the RAHT encoding layer unit of the attribute encoding part, thereby further improving the encoding efficiency of the point cloud.

[0314] The method embodiments of this application have been described in detail above with reference to Figures 1 to 31. The apparatus embodiments of this application will be described in detail below with reference to Figures 32 to 35. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be referred to the preceding method embodiments.

[0315] Figure 32 is a schematic diagram of the structure of a decoder provided in an embodiment of this application. As shown in Figure 32, the decoder 3200 may include a first determining unit 3210, a second determining unit 3220, a third determining unit 3230, and a fourth determining unit 3240.

[0316] The first determining unit 3210 is configured to decode the bitstream and determine the first motion information.

[0317] The second determining unit 3220 is configured to perform motion compensation on the first reference point cloud based on the first motion information to determine the second reference point cloud.

[0318] The third determining unit 3230 is configured to determine the predicted value of the attribute information or geometric information of the current point cloud based on the second reference point cloud.

[0319] The fourth determining unit 3240 is configured to determine the reconstructed value of the attribute information or geometric information of the current point cloud based on the predicted value of the attribute information or geometric information of the current point cloud.

[0320] In some implementations, the second determining unit 3220 is further configured to perform motion compensation on the first reference point cloud based on the first motion information to determine the motion-compensated first reference point cloud; and to process at least one point in the motion-compensated first reference point cloud based on the geometric information of the current point cloud to determine the second reference point cloud, wherein the at least one point is determined based on the geometric information of the current point cloud.

[0321] In some implementations, the second determining unit 3220 is further configured to delete the at least one point; or to adjust the geometric information of the at least one point into a first bounding box, the first bounding box being determined based on the geometric information of the current point cloud.

[0322] In some implementations, the decoder 3200 is further configured to parse the bitstream, determine the second motion information, perform motion compensation on the first reference point cloud based on the second motion information, and determine the motion-compensated first reference point cloud. The motion-compensated first reference point cloud is used to perform inter-frame prediction on the geometric information of the current point cloud. The first motion information is different from the second motion information.

[0323] In some implementations, the decoder 3200 is further configured to decode the bitstream and determine first identification information. The first identification information is used to indicate the inter-frame prediction mode of the attribute information of the first strip of the current point cloud. The inter-frame prediction mode of the attribute information of the first strip is either a first inter-frame prediction mode or a second inter-frame prediction mode. The first inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the second inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

[0324] In some implementations, the decoder 3200 is further configured to decode the bitstream and determine second identification information. The second identification information is used to indicate the inter-frame prediction mode of the attribute information of the first region adaptive hierarchical transformation (RAHT) layer of the current point cloud. The inter-frame prediction mode of the attribute information of the first RAHT layer is either a third inter-frame prediction mode or a fourth inter-frame prediction mode. The third inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the fourth inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

[0325] In some implementations, the decoder 3200 is further configured to decode the bitstream and determine third identification information. The third identification information is used to indicate the inter-frame prediction mode of the attribute information of the first level of detail (LOD) layer of the current point cloud. The inter-frame prediction mode of the attribute information of the first LOD layer is either a fifth inter-frame prediction mode or a sixth inter-frame prediction mode. The fifth inter-frame prediction mode is a prediction mode based on motion compensation, and the sixth inter-frame prediction mode is a prediction mode not based on motion compensation.

[0326] In some implementations, the third determining unit 3230 is further configured to perform RAHT transformation on the second stripe in the second reference point cloud to determine at least one reference transform block; perform RAHT transformation on the first stripe to be decoded in the current point cloud to determine at least one transform block; and perform inter-frame prediction on the current transform block in the at least one transform block according to the at least one reference transform block to determine the predicted value of the attribute transform coefficient of the current transform block; wherein the second stripe is determined based on the geometric information of the first stripe to be decoded.

[0327] In some implementations, the third determining unit 3230 is further configured to determine a first reference transform block from the at least one reference transform block based on the geometric information of the current transform block; and to determine the predicted value of the attribute transform coefficient of the current transform block based on the attribute transform coefficient of the first reference transform block.

[0328] In some implementations, the geometric information of the current transform block includes: the coordinate information of the current transform block and / or the hierarchical information of the RAHT layer.

[0329] In some implementations, the fourth determining unit 3240 is further configured to determine the reconstructed value of the attribute transformation coefficient of the current transform block based on the predicted value of the attribute transformation coefficient of the current transform block;

[0330] The reconstructed values ​​of the attribute transformation coefficients of the current transform block are inversely transformed to determine the reconstructed values ​​of the attribute information of the first strip to be decoded.

[0331] In some implementations, the decoder 3200 is further configured to parse the bitstream, determine the first parameter, and determine the first motion information based on the first parameter.

[0332] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0333] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0334] Therefore, this application provides a computer-readable storage medium for use in a decoder 3200. The computer-readable storage medium stores a computer program that, when executed by a processor, implements the decoding method described in any of the foregoing embodiments.

[0335] Based on the composition of the decoder 3200 described above and the computer-readable storage medium, refer to Figure 33, which shows a schematic diagram of the specific hardware structure of the decoder 3300 provided in this embodiment of the application. As shown in Figure 33, the decoder 3300 may include: a communication interface 3310, a memory 3320, and a processor 3330; the various components are coupled together through a bus system 3340. It is understood that the bus system 3340 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 3340 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 3340 in Figure 33.

[0336] The communication interface 3310 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0337] Memory 3320 is used to store computer programs;

[0338] Processor 3330, when running the computer program, performs the following:

[0339] Decode the bitstream to determine the first motion information;

[0340] Motion compensation is performed on the first reference point cloud based on the first motion information to determine the second reference point cloud;

[0341] Based on the second reference point cloud, determine the predicted values ​​of the attribute information or geometric information of the current point cloud;

[0342] Based on the predicted values ​​of the attribute information or geometric information of the current point cloud, the reconstructed values ​​of the attribute information or geometric information of the current point cloud are determined.

[0343] It is understood that the memory 3320 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM). The memory 3320 of the systems and methods described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0344] The processor 3330 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 3330 or by software instructions. The processor 3330 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 3320. Processor 3330 reads the information in memory 3320 and completes the steps of the above method in conjunction with its hardware.

[0345] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), DSP devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or externally.

[0346] Alternatively, as another embodiment, the processor 3330 is also configured to execute the decoding method described in any of the foregoing embodiments when running the computer program.

[0347] Figure 34 is a schematic diagram of the structure of an encoder provided in an embodiment of this application. As shown in Figure 34, the encoder 3400 includes a first determining unit 3410, a second determining unit 3420, and a third determining unit 3430.

[0348] The first determining unit 3410 is configured to perform motion compensation on the first reference point cloud based on the first motion information, and determine the second reference point cloud.

[0349] The second determining unit 3420 is configured to determine the predicted value of the attribute information or geometric information of the current point cloud based on the second reference point cloud.

[0350] The third determining unit 3430 is configured to determine the residual value of the attribute information or geometric information of the current point cloud based on the predicted value of the attribute information or geometric information of the current point cloud.

[0351] In some implementations, the first determining unit 3410 is further configured to perform motion compensation on the first reference point cloud based on the first motion information to determine the motion-compensated first reference point cloud; and to process at least one point in the motion-compensated first reference point cloud based on the geometric information of the current point cloud to determine the second reference point cloud, wherein the at least one point is determined based on the geometric information of the current point cloud.

[0352] In some implementations, the first determining unit 3410 is further configured to delete the at least one point; or to adjust the geometric information of the at least one point into a first bounding box, the first bounding box being determined based on the geometric information of the current point cloud.

[0353] In some implementations, the encoder 3400 is further configured to perform motion compensation on the first reference point cloud based on the second motion information to determine the motion-compensated first reference point cloud, wherein the motion-compensated first reference point cloud is used to perform inter-frame prediction of the geometric information of the current point cloud, and the first motion information is different from the second motion information.

[0354] In some implementations, the first motion information is determined based on one or more of the following: the geometric information of the current point cloud and the geometric information of the first reference point cloud; the attribute information of the current point cloud and the attribute information of the first reference point cloud.

[0355] In some implementations, the first motion information is determined based on one or more of the following: the difference between the geometric information of the current point cloud and the geometric information of the first reference point cloud; the difference between the attribute information of the current point cloud and the attribute information of the first reference point cloud.

[0356] In some implementations, the difference is the mean squared error.

[0357] In some implementations, the encoder 3400 is further configured to write first identification information into the bitstream. The first identification information is used to indicate the inter-frame prediction mode of the attribute information of the first strip of the current point cloud. The inter-frame prediction mode of the attribute information of the first strip is either a first inter-frame prediction mode or a second inter-frame prediction mode. The first inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the second inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

[0358] In some implementations, the encoder 3400 is further configured to perform inter-frame prediction on the attribute information of the first stripe according to the first inter-frame prediction mode to determine a first rate-distortion cost; perform inter-frame prediction on the attribute information of the first stripe according to the second inter-frame prediction mode to determine a second rate-distortion cost; and determine the inter-frame prediction mode of the attribute information of the first stripe from the first inter-frame prediction mode and the second inter-frame prediction mode according to the first rate-distortion cost and the second rate-distortion cost.

[0359] In some implementations, the encoder 3400 is further configured to write second identification information into the bitstream. The second identification information is used to indicate the inter-frame prediction mode of the attribute information of the first region adaptive hierarchical transformation (RAHT) layer of the current point cloud. The inter-frame prediction mode of the attribute information of the first RAHT layer is either a third inter-frame prediction mode or a fourth inter-frame prediction mode. The third inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the fourth inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

[0360] In some implementations, the encoder 3400 is further configured to perform inter-frame prediction on the attribute information of the first RAHT layer according to the third inter-frame prediction mode to determine a third rate distortion cost; perform inter-frame prediction on the attribute information of the first RAHT layer according to the fourth inter-frame prediction mode to determine a fourth rate distortion cost; and determine the inter-frame prediction mode of the attribute information of the first RAHT layer from the third inter-frame prediction mode and the fourth inter-frame prediction mode according to the third rate distortion cost and the fourth rate distortion cost.

[0361] In some implementations, the encoder 3400 is further configured to write third identification information into the bitstream. The third identification information is used to indicate the inter-frame prediction mode of the attribute information of the first level of detail (LOD) layer of the current point cloud. The inter-frame prediction mode of the attribute information of the first LOD layer is either a fifth inter-frame prediction mode or a sixth inter-frame prediction mode. The fifth inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the sixth inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

[0362] In some implementations, the encoder 3400 is further configured to perform inter-frame prediction on the attribute information of the first LOD layer according to the fifth inter-frame prediction mode to determine the fifth rate distortion cost; perform inter-frame prediction on the attribute information of the first LOD layer according to the sixth inter-frame prediction mode to determine the sixth rate distortion cost; and determine the inter-frame prediction mode of the attribute information of the first LOD layer from the fifth inter-frame prediction mode and the sixth inter-frame prediction mode according to the fifth rate distortion cost and the sixth rate distortion cost.

[0363] In some implementations, the second determining unit 3420 is further configured to perform RAHT transformation on the second stripe in the second reference point cloud to determine at least one reference transform block; perform RAHT transformation on the first stripe to be encoded in the current point cloud to determine at least one transform block; and perform inter-frame prediction on the current transform block in the at least one transform block according to the at least one reference transform block to determine the predicted value of the attribute transform coefficient of the current transform block; wherein the second stripe is determined based on the geometric information of the first stripe to be encoded.

[0364] In some implementations, the second determining unit 3420 is further configured to determine a first reference transformation block from the at least one reference transformation block based on the geometric information of the current transformation block; and to determine the predicted value of the attribute transformation coefficient of the current transformation block based on the attribute transformation coefficient of the first reference transformation block.

[0365] In some implementations, the coordinate information of the current transform block and / or the hierarchical information of the RAHT layer are included.

[0366] In some implementations, the third determining unit 3430 is further configured to determine the residual value of the attribute transformation coefficient of the current transform block based on the predicted value of the attribute transformation coefficient of the current transform block and the original value of the attribute transformation coefficient of the current transform block.

[0367] In some implementations, the encoder 3400 is further configured to quantize the residual values ​​of the attribute transformation coefficients of the current transform block.

[0368] In some implementations, the encoder 3400 is further configured to determine a first parameter based on the first motion information and write the first parameter into the bitstream.

[0369] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0370] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, ROM, RAM, magnetic disks, or optical disks.

[0371] Therefore, this application provides a computer-readable storage medium for use in an encoder 3400. The computer-readable storage medium stores a computer program that, when executed by a processor, implements the decoding method described in any of the foregoing embodiments.

[0372] Based on the composition of the encoder 3400 described above and the computer-readable storage medium, refer to Figure 35, which shows a schematic diagram of the specific hardware structure of the encoder 3500 provided in this embodiment of the application. As shown in Figure 35, the encoder 3500 may include: a communication interface 3510, a memory 3520, and a processor 3530; the various components are coupled together through a bus system 3540. It is understood that the bus system 3540 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 3540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 3540 in Figure 35.

[0373] The communication interface 3510 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0374] Memory 3520 is used to store computer programs;

[0375] Processor 3530, when running the computer program, performs the following:

[0376] Motion compensation is performed on the first reference point cloud based on the first motion information to determine the second reference point cloud;

[0377] Based on the second reference point cloud, determine the predicted values ​​of the attribute information or geometric information of the current point cloud;

[0378] Based on the predicted values ​​of the attribute information or geometric information of the current point cloud, determine the residual values ​​of the attribute information or geometric information of the current point cloud.

[0379] It is understood that the memory 3520 in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be ROM, PROM, EPROM, EEPROM, or flash memory. Volatile memory may be RAM, which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as SRAM, DRAM, SDRAM, DDRSDRAM, ESDRAM, SLDRAM, and DRRAM. The memory 3520 of the systems and methods described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0380] The processor 3530 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of the processor 3530 or through software instructions. The processor 3530 may be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 3520, and the processor 3530 reads the information in memory 3520 and, in conjunction with its hardware, completes the steps of the above method.

[0381] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, DSPDs, PLDs, FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0382] Alternatively, as another embodiment, the processor 3530 is also configured to execute the encoding method described in any of the foregoing embodiments when running the computer program.

[0383] This application also provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium for storing bit streams. The bit streams can be generated by using an encoding method of an encoder, or the bit streams can be decoded by using a decoding method of a decoder. The decoding method can be the decoding method described in any of the preceding embodiments, and the encoding method can be the encoding method described in any of the preceding embodiments.

[0384] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0385] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0386] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0387] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0388] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A point cloud decoding method, applied to a decoder, comprising: Decode the bitstream to determine the first motion information; Motion compensation is performed on the first reference point cloud based on the first motion information to determine the second reference point cloud; Based on the second reference point cloud, determine the predicted values ​​of the attribute information or geometric information of the current point cloud; Based on the predicted values ​​of the attribute information or geometric information of the current point cloud, the reconstructed values ​​of the attribute information or geometric information of the current point cloud are determined.

2. The method according to claim 1, wherein, The step of performing motion compensation on the first reference point cloud based on the first motion information to determine the second reference point cloud includes: Motion compensation is performed on the first reference point cloud based on the first motion information to determine the motion-compensated first reference point cloud. Based on the geometric information of the current point cloud, at least one point in the motion-compensated first reference point cloud is processed to determine the second reference point cloud, wherein the at least one point is determined based on the geometric information of the current point cloud.

3. The method according to claim 2, wherein, The processing of at least one point in the motion-compensated first reference point cloud includes: Delete at least one of the points; or, The geometric information of the at least one point is adjusted into a first bounding box, which is determined based on the geometric information of the current point cloud.

4. The method according to claim 1, wherein, The method further includes: Analyze the bitstream to determine the second motion information; Motion compensation is performed on the first reference point cloud based on the second motion information to determine the motion-compensated first reference point cloud. The motion-compensated first reference point cloud is used to perform inter-frame prediction of the geometric information of the current point cloud. The first motion information is different from the second motion information.

5. The method according to claim 1, wherein, The method further includes: Decode the bitstream and determine the first identification information. The first identification information is used to indicate the inter-frame prediction mode of the attribute information of the first strip of the current point cloud. The inter-frame prediction mode of the attribute information of the first strip is either a first inter-frame prediction mode or a second inter-frame prediction mode. The first inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the second inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

6. The method according to claim 1, wherein, The method further includes: Decode the bitstream and determine the second identification information. The second identification information is used to indicate the inter-frame prediction mode of the attribute information of the first region of the current point cloud in the adaptive hierarchical transformation RAHT layer. The inter-frame prediction mode of the attribute information of the first RAHT layer is either the third inter-frame prediction mode or the fourth inter-frame prediction mode. The third inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the fourth inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

7. The method according to claim 1, wherein, The method further includes: Decode the bitstream and determine the third identification information. The third identification information is used to indicate the inter-frame prediction mode of the attribute information of the first level of detail (LOD) layer of the current point cloud. The inter-frame prediction mode of the attribute information of the first LOD layer is either the fifth inter-frame prediction mode or the sixth inter-frame prediction mode. The fifth inter-frame prediction mode is a prediction mode based on motion compensation, and the sixth inter-frame prediction mode is a prediction mode not based on motion compensation.

8. The method according to claim 1, wherein, The step of determining the predicted values ​​of the attribute information or geometric information of the current point cloud based on the second reference point cloud includes: Perform RAHT transformation on the second strip in the second reference point cloud to determine at least one reference transformation block; Perform RAHT transformation on the first stripe to be decoded in the current point cloud to determine at least one transform block; Based on the at least one reference transform block, perform inter-frame prediction on the current transform block in the at least one transform block to determine the predicted values ​​of the attribute transform coefficients of the current transform block; The second stripe is determined based on the geometric information of the first stripe to be decoded.

9. The method according to claim 8, wherein, The step of performing inter-frame prediction on the current transform block in the at least one transform block based on the at least one reference transform block, and determining the predicted values ​​of the attribute transform coefficients of the current transform block, includes: Based on the geometric information of the current transform block, a first reference transform block is determined from the at least one reference transform block; Based on the attribute transformation coefficients of the first reference transform block, the predicted values ​​of the attribute transformation coefficients of the current transform block are determined.

10. The method according to claim 9, wherein, The geometric information of the current transform block includes: The coordinate information of the current transform block and / or the hierarchical information of the RAHT layer.

11. The method according to any one of claims 8 to 10, wherein, Determining the reconstructed values ​​of the attribute information or geometric information of the current point cloud based on the predicted values ​​of the attribute information or geometric information of the current point cloud includes: Based on the predicted values ​​of the attribute transformation coefficients of the current transform block, the reconstructed values ​​of the attribute transformation coefficients of the current transform block are determined; The reconstructed values ​​of the attribute transformation coefficients of the current transform block are inversely transformed to determine the reconstructed values ​​of the attribute information of the first strip to be decoded.

12. The method according to claim 1, wherein, The decoded bitstream determines the first motion information, including: Analyze the bitstream to determine the first parameter; The first motion information is determined based on the first parameter.

13. A point cloud encoding method, applied to an encoder, comprising: Motion compensation is performed on the first reference point cloud based on the first motion information to determine the second reference point cloud; Based on the second reference point cloud, determine the predicted values ​​of the attribute information or geometric information of the current point cloud; Based on the predicted values ​​of the attribute information or geometric information of the current point cloud, determine the residual values ​​of the attribute information or geometric information of the current point cloud.

14. The method according to claim 13, wherein, The step of performing motion compensation on the first reference point cloud based on the first motion information to determine the second reference point cloud includes: Motion compensation is performed on the first reference point cloud based on the first motion information to determine the motion-compensated first reference point cloud. Based on the geometric information of the current point cloud, at least one point in the motion-compensated first reference point cloud is processed to determine the second reference point cloud, wherein the at least one point is determined based on the geometric information of the current point cloud.

15. The method according to claim 14, wherein, The processing of at least one point in the motion-compensated first reference point cloud includes: Delete at least one of the points; or, The geometric information of the at least one point is adjusted into a first bounding box, which is determined based on the geometric information of the current point cloud.

16. The method according to claim 13, wherein, The method further includes: Motion compensation is performed on the first reference point cloud based on the second motion information to determine the motion-compensated first reference point cloud. The motion-compensated first reference point cloud is used to perform inter-frame prediction of the geometric information of the current point cloud. The first motion information is different from the second motion information.

17. The method according to claim 13, wherein, The first motion information is determined based on one or more of the following: The geometric information of the current point cloud and the geometric information of the first reference point cloud; The attribute information of the current point cloud and the attribute information of the first reference point cloud.

18. The method according to claim 17, wherein, The first motion information is determined based on one or more of the following: The difference between the geometric information of the current point cloud and the geometric information of the first reference point cloud; The difference between the attribute information of the current point cloud and the attribute information of the first reference point cloud.

19. The method according to claim 18, wherein, The difference is the mean square error.

20. The method according to claim 13, wherein, The method further includes: Write the first identification information into the bitstream. The first identification information is used to indicate the inter-frame prediction mode of the attribute information of the first strip of the current point cloud. The inter-frame prediction mode of the attribute information of the first strip is either a first inter-frame prediction mode or a second inter-frame prediction mode. The first inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the second inter-frame prediction mode is an inter-frame prediction mode not based on motion compensation.

21. The method according to claim 20, wherein, The method includes: Based on the first inter-frame prediction mode, the attribute information of the first stripe is predicted inter-frame to determine the first rate distortion cost. Based on the second inter-frame prediction mode, the attribute information of the first stripe is predicted inter-frame to determine the second rate distortion cost; Based on the first rate-distortion cost and the second rate-distortion cost, the inter-frame prediction mode of the attribute information of the first stripe is determined from the first inter-frame prediction mode and the second inter-frame prediction mode.

22. The method according to claim 13, wherein, The method further includes: The second identification information is written into the bitstream. The second identification information is used to indicate the inter-frame prediction mode of the attribute information of the first region of the current point cloud adaptive hierarchical transformation RAHT layer. The inter-frame prediction mode of the attribute information of the first RAHT layer is either the third inter-frame prediction mode or the fourth inter-frame prediction mode. The third inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the fourth inter-frame prediction mode is an inter-frame prediction mode without motion compensation.

23. The method according to claim 22, wherein, The method further includes: Based on the third inter-frame prediction mode, inter-frame prediction is performed on the attribute information of the first RAHT layer to determine the third rate distortion cost. Based on the fourth inter-frame prediction mode, inter-frame prediction is performed on the attribute information of the first RAHT layer to determine the fourth rate distortion cost. Based on the third rate-distortion cost and the fourth rate-distortion cost, from the third inter-frame prediction mode and the fourth inter-frame prediction mode The formula determines the inter-frame prediction mode of the attribute information of the first RAHT layer.

24. The method according to claim 13, wherein, The method further includes: The third identification information is written into the bitstream. The third identification information is used to indicate the inter-frame prediction mode of the attribute information of the first level of detail (LOD) layer of the current point cloud. The inter-frame prediction mode of the attribute information of the first LOD layer is either the fifth inter-frame prediction mode or the sixth inter-frame prediction mode. The fifth inter-frame prediction mode is an inter-frame prediction mode based on motion compensation, and the sixth inter-frame prediction mode is an inter-frame prediction mode without motion compensation.

25. The method according to claim 24, wherein, The method further includes: Based on the fifth inter-frame prediction mode, perform inter-frame prediction on the attribute information of the first LOD layer to determine the fifth rate distortion cost; Based on the sixth inter-frame prediction mode, perform inter-frame prediction on the attribute information of the first LOD layer to determine the sixth rate distortion cost; Based on the fifth rate distortion cost and the sixth rate distortion cost, the inter-frame prediction mode of the attribute information of the first LOD layer is determined from the fifth inter-frame prediction mode and the sixth inter-frame prediction mode.

26. The method according to claim 13, wherein, The step of determining the predicted values ​​of the attribute information or geometric information of the current point cloud based on the second reference point cloud includes: Perform RAHT transformation on the second strip in the second reference point cloud to determine at least one reference transformation block; Perform RAHT transformation on the first stripe to be encoded in the current point cloud to determine at least one transform block; Based on the at least one reference transform block, perform inter-frame prediction on the current transform block in the at least one transform block to determine the predicted values ​​of the attribute transform coefficients of the current transform block; The second stripe is determined based on the geometric information of the first stripe to be encoded.

27. The method according to claim 26, wherein, The step of performing inter-frame prediction on the current transform block in the at least one transform block based on the at least one reference transform block, and determining the predicted values ​​of the attribute transform coefficients of the current transform block, includes: Based on the geometric information of the current transform block, a first reference transform block is determined from the at least one reference transform block; Based on the attribute transformation coefficients of the first reference transform block, the predicted values ​​of the attribute transformation coefficients of the current transform block are determined.

28. The method according to claim 27, wherein, The geometric information of the current transform block includes: The coordinate information of the current transform block and / or the hierarchical information of the RAHT layer.

29. The method according to any one of claims 26 to 28, wherein, The step of determining the residual value of the attribute information or geometric information of the current point cloud based on the predicted value of the attribute information or geometric information of the current point cloud includes: The residual value of the attribute transformation coefficient of the current transform block is determined based on the predicted value of the attribute transformation coefficient of the current transform block and the original value of the attribute transformation coefficient of the current transform block.

30. The method according to claim 29, wherein, The method further includes: Quantize the residual values ​​of the attribute transformation coefficients of the current transform block.

31. The method according to claim 13, wherein, The method further includes: Based on the first motion information, determine the first parameter; Write the first parameter into the bitstream.

32. A decoder, comprising: The first determining unit is configured to decode the bitstream and determine the first motion information; The second determining unit is configured to perform motion compensation on the first reference point cloud based on the first motion information, and determine the second reference point cloud. The third determining unit is configured to determine the predicted value of the attribute information or geometric information of the current point cloud based on the second reference point cloud. The fourth determining unit is configured to determine the reconstructed values ​​of the attribute information or geometric information of the current point cloud based on the predicted values ​​of the attribute information or geometric information of the current point cloud.

33. A decoder, comprising: Memory, used to store computer programs; A processor, configured to perform the method as described in any one of claims 1 to 12 when running the computer program.

34. An encoder, comprising: The first determining unit is configured to perform motion compensation on the first reference point cloud based on the first motion information, and determine the second reference point cloud. The second determining unit is configured to determine the predicted value of the attribute information or geometric information of the current point cloud based on the second reference point cloud. The third determining unit is configured to determine the residual value of the attribute information or geometric information of the current point cloud based on the predicted value of the attribute information or geometric information of the current point cloud.

35. An encoder, comprising: Memory, used to store computer programs; A processor, configured to perform the method as described in any one of claims 13 to 31 when running the computer program.

36. A non-volatile computer-readable storage medium for storing a bitstream, said bitstream being generated by an encoding method using an encoder, or said bitstream being decoded by a decoding method using a decoder, wherein, The decoding method is the method as described in any one of claims 1 to 12, and the encoding method is the method as described in any one of claims 13 to 31.

37. A computer-readable storage medium storing a bitstream generated by the method of any one of claims 13 to 31.

38. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 12, or 13 to 31.

Images