Point cloud encoding and decoding methods, encoder, decoder, and storage medium

Abstract

The present application provides point cloud encoding and decoding methods, an encoder, a decoder, and a storage medium. The point cloud decoding method comprises: performing region adaptive hierarchical transform (RAHT) on a portion of point clouds in a reference frame, and determining at least one prediction block (S1110); performing RAHT transform on a first slice to be decoded in a current frame, and determining one or more transform blocks (S1120); on the basis of the at least one prediction block, performing inter-frame prediction on a current transform block among the one or more transform blocks, and determining a predicted value of an attribute transform coefficient of the current transform block (S1130); and, on the basis of the predicted value of the attribute transform coefficient of the current transform block, determining a reconstructed value of the attribute transform coefficient of the current transform block (S1140).

Description

Point cloud coding method, coder and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud coding, and particularly relates to a point cloud coding method, a coder and a storage medium. BACKGROUND

[0002] In the process of encoding the attribute information of the point cloud, the attribute information of a current to-be-encoded node can be predicted and encoded based on region adaptive hierarchal transform (RAHT) inter-frame prediction. How to improve the accuracy of RAHT inter-frame prediction to improve the coding performance of the attribute information of the point cloud is a problem to be solved.

[0003] SUMMARY

[0004] Embodiments of the present application provide a point cloud coding method, a coder and a storage medium. Each aspect of the present application is introduced as follows.

[0005] In a first aspect, a point cloud decoding method is provided, applied to a decoder, including: performing region adaptive hierarchal transform (RAHT) on part of the point cloud in a reference frame to determine at least one prediction block; performing RAHT transform on a first to-be-decoded strip in a current frame to determine one or more transform blocks; performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of an attribute transform coefficient of the current transform block; and determining a reconstructed value of the attribute transform coefficient of the current transform block according to the predicted value of the attribute transform coefficient of the current transform block.

[0006] In a second aspect, a point cloud encoding method is provided, applied to an encoder, including: performing region adaptive hierarchal transform (RAHT) on part of the point cloud in a reference frame to determine at least one prediction block; performing RAHT transform on a first to-be-encoded strip in a current frame to determine one or more transform blocks; performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of an attribute transform coefficient of the current transform block; and determining a residual value of the attribute transform coefficient of the current transform block according to the predicted value of the attribute transform coefficient of the current transform block.

[0007] In a third aspect, a decoder is provided, comprising: a first transform unit configured to perform a region-adaptive hierarchical transform (RAHT) on a part of point cloud in a reference frame to determine at least one prediction block; a second transform unit configured to perform the RAHT on a first to-be-decoded strip in a current frame to determine one or more transform blocks; a prediction unit configured to perform inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a prediction value of an attribute transform coefficient of the current transform block; and a determination unit configured to determine a reconstructed value of the attribute transform coefficient of the current transform block according to the prediction value of the attribute transform coefficient of the current transform block.

[0008] In a fourth aspect, a decoder is provided, comprising: a memory configured to store a computer program; and a processor configured to execute the method of the first aspect when running the computer program.

[0009] In a fifth aspect, an encoder is provided, comprising: a first transform unit configured to perform a region-adaptive hierarchical transform (RAHT) on a part of point cloud in a reference frame to determine at least one prediction block; a second transform unit configured to perform the RAHT on a first to-be-encoded strip in a current frame to determine one or more transform blocks; a prediction unit configured to perform inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a prediction value of an attribute transform coefficient of the current transform block; and a determination unit configured to determine a residual value of the attribute transform coefficient of the current transform block according to the prediction value of the attribute transform coefficient of the current transform block.

[0010] In a sixth aspect, an encoder is provided, comprising: a memory configured to store a computer program; and a processor configured to execute the method of the second aspect when running the computer program.

[0011] In a seventh aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is executed to implement the method of the first aspect or the second aspect.

[0012] In an eighth aspect, a non-volatile computer-readable storage medium storing a bitstream is provided, the bitstream is generated by using an encoding method of an encoder, or the bitstream is decoded by using a decoding method of a decoder, wherein the decoding method is the method of the first aspect, and the encoding method is the method of the second aspect.

[0013] In a ninth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a bitstream generated according to the method of the second aspect.

[0014] The RAHT transform is based on the wavelet transform of the octree hierarchy, and attribute information is associated with the octree nodes. The attribute information of the occupied nodes in the same parent node is recursively transformed in a bottom-up manner until the root node of the octree is reached. In the process of hierarchical transformation, all neighbor points in the same layer are transformed to obtain the transform coefficients (or transform blocks).

[0015] For the current frame to be encoded, it is encoded slice by slice, that is, a slice is a unit of RAHT transformation, that is, neighbor points in the same layer belonging to a slice are transformed to obtain a transform block. In the related art, when performing RAHT inter-frame prediction encoding, the entire frame of the reference frame is used for RAHT transformation, that is, a frame is a unit of RAHT transformation, that is, neighbor points in the same layer belonging to a frame are transformed to obtain a prediction block. The number and spatial positions of the neighbor points of the two are obviously not equal, and there may be a large redundancy between the transform coefficients corresponding to the transform block and the transform coefficients corresponding to the prediction block. Therefore, the related art of performing RAHT transformation on the entire frame of the reference frame to obtain an inter-frame prediction block may not be optimal, and there may be a problem of low prediction accuracy.

[0016] In the present embodiment, when performing RAHT inter-frame prediction encoding, the partial point cloud of the reference frame is used for RAHT transformation. Compared with the related art of performing RAHT transformation on the entire frame of the reference frame, the prediction block obtained is more matched with the transform block, and the accuracy of inter-frame prediction is higher, thereby helping to improve the coding and decoding performance of the point cloud attribute information. BRIEF DESCRIPTION OF DRAWINGS

[0017] FIG. 1A is a schematic diagram of a three-dimensional point cloud image.

[0018] FIG. 1B is a partial enlarged view of a three-dimensional point cloud image.

[0019] FIG. 2A is a schematic diagram of six viewing angles of a point cloud image.

[0020] FIG. 2B is a schematic diagram of a data storage format corresponding to a point cloud image.

[0021] FIG. 3 is a schematic diagram of a network architecture of point cloud coding.

[0022] FIG. 4A is a schematic diagram of the composition framework of a G-PCC encoder.

[0023] FIG. 4B is a schematic diagram of the composition framework of a G-PCC decoder.

[0024] FIG. 5 is an example diagram of the RAHT transformation process.

[0025] FIG. 6 is another example diagram of the RAHT transformation process.

[0026] FIG. 7 is a schematic diagram of RAHT transform and RAHT inverse transform.

[0027] FIG. 8 is a schematic diagram of RAHT inter prediction coding structure.

[0028] FIG. 9 is an example diagram of hierarchy of RAHT transform.

[0029] FIG. 10 is a schematic diagram of a flow of RAHT inter prediction.

[0030] FIG. 11 is a schematic diagram of a flow of a decoding method provided by an embodiment of the present application.

[0031] FIG. 12 is a schematic diagram of a flow of an encoding method provided by an embodiment of the present application.

[0032] FIG. 13 is a schematic diagram of a flow of RAHT inter prediction provided by an embodiment of the present application.

[0033] FIG. 14 is a schematic diagram of a structure of a decoder provided by an embodiment of the present application.

[0034] FIG. 15 is a schematic diagram of a structure of a decoder provided by another embodiment of the present application.

[0035] FIG. 16 is a schematic diagram of a structure of an encoder provided by an embodiment of the present application.

[0036] FIG. 17 is a schematic diagram of a structure of an encoder provided by another embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to enable a person skilled in the art to more fully understand the features and technical content of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are used only for reference and are not intended to limit the embodiments of the present application.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description herein is for describing the embodiments of the present application only and is not intended to limit the present application.

[0039] In the following description, "some implementations" are related to a subset of all possible embodiments, but it can be understood that "some implementations" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0040] It should also be noted that the terms "first", "second", and "third" in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order of the objects. Understandably, "first", "second", and "third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0041] A point cloud is a three-dimensional representation of an object surface. A point cloud (data) of an object surface can be collected by photoelectric radar, laser radar, laser scanner, multi-view camera, and the like.

[0042] A point cloud is a set of discrete points that are randomly distributed in space and express the spatial structure and surface properties of a three-dimensional object or scene. FIG. 1A shows a three-dimensional point cloud image, and FIG. 1B shows a partial enlarged view of the three-dimensional point cloud image. As can be seen, the point cloud surface is composed of densely distributed points.

[0043] A two-dimensional image has information expressed at each pixel point, and the distribution is regular, so there is no need to record the position information. However, the distribution of points in a point cloud in three-dimensional space has randomness and irregularity, so the position of each point in space needs to be recorded in order to completely express a point cloud. Similar to a two-dimensional image, each position has corresponding attribute information during the collection process, which is usually an RGB color value, and the color value reflects the color of the object. For a point cloud, the attribute information corresponding to each point, in addition to color information, also includes reflectance value, which is commonly used to reflect the surface material of the object. Therefore, point cloud data usually includes position information and attribute information of points. The position information of points can also be referred to as geometric information of points. For example, the geometric information of points can be three-dimensional coordinate information (x, y, z) of points. The attribute information of points can include color information and / or reflectance, and the like. For example, the reflectance can be one-dimensional reflectance information (r); the color information can be information on any color space, or the color information can be three-dimensional color information such as RGB information. Here, R represents red (red, R), G represents green (green, G), and B represents blue (blue, B). For another example, the color information can be luma chroma (YCbCr, YUV) information. Y represents luma, Cb (U) represents blue chroma, and Cr (V) represents red chroma.

[0044] According to the laser measurement principle, the point cloud obtained can include the three-dimensional coordinate information of the point and the reflectivity value of the point. For example, according to the photogrammetry principle, the point cloud obtained can include the three-dimensional coordinate information of the point and the three-dimensional color information of the point. For another example, the point cloud obtained in combination with the laser measurement and the photogrammetry principle can include the three-dimensional coordinate information of the point, the reflectivity value of the point, and the three-dimensional color information of the point.

[0045] As shown in FIGS. 2A and 2B, a point cloud image and its corresponding data storage format are shown. FIG. 2A provides six viewing angles of the point cloud image, and FIG. 2B is composed of a file header information part and a data part. The header information includes the data format, the data representation type, the total number of points of the point cloud, and the content represented by the point cloud. For example, the point cloud is in the “.ply” format, represented by ASCII code, with a total number of points of 207242, and each point has three-dimensional coordinate information (x, y, z) and three-dimensional color information (r, g, b).

[0046] The point cloud can be classified according to the acquisition method as follows:

[0047] Static point cloud: the object is static, and the device for acquiring the point cloud is also static;

[0048] Dynamic point cloud: the object is moving, but the device for acquiring the point cloud is static;

[0049] Dynamic acquisition of point cloud: the device for acquiring the point cloud is moving.

[0050] For example, the point cloud can be classified into two categories according to the use as follows:

[0051] Category one: machine perception point cloud, which can be used in autonomous navigation system, real-time inspection system, geographic information system, visual sorting robot, rescue robot, and the like;

[0052] Category two: human eye perception point cloud, which can be used in digital cultural heritage, free viewpoint broadcast, three-dimensional immersive communication, three-dimensional immersive interaction, and the like.

[0053] The point cloud can flexibly and conveniently express the spatial structure and surface properties of a three-dimensional object or scene, and can provide strong reality under the premise of ensuring accuracy because the point cloud is obtained by directly sampling a real object. Therefore, the point cloud is widely used in virtual reality games, computer-aided design, geographic information systems, autonomous navigation systems, digital cultural heritage, free viewpoint broadcast, three-dimensional immersive remote presentation, three-dimensional reconstruction of biological tissues and organs, and the like.

[0054] The acquisition of point cloud mainly has the following ways: computer generation, 3D laser scanning, 3D photogrammetry, etc. The computer can generate the point cloud of virtual three-dimensional objects and scenes; 3D laser scanning can obtain the point cloud of static real-world three-dimensional objects or scenes, and can obtain million-level point cloud per second; 3D photogrammetry can obtain the point cloud of dynamic real-world three-dimensional objects or scenes, and can obtain ten million-level point cloud per second. These technologies reduce the cost and time period of point cloud data acquisition, and improve the accuracy of data. The change of point cloud data acquisition method makes it possible to obtain a large amount of point cloud data, and with the increase of application demand, the processing of massive 3D point cloud data encounters the bottleneck of storage space and transmission bandwidth limitation.

[0055] Exemplarily, taking a point cloud video with a frame rate of 30 frames per second (fps) as an example, the number of points of each frame of point cloud is 700,000, each point has coordinate information xyz (float) and color information RGB (uchar), and the data amount of 10s point cloud video is about 0.7 million × (4 Byte × 3 + 1 Byte × 3) × 30 fps × 10s = 3.15 GB, wherein 1 Byte is 10 bit; while the YUV sampling format is 4:2:0, the frame rate of 1280 × 720 two-dimensional video is 24 fps, and the data amount of 10s is about 1280 × 720 × 12 bit × 24 fps × 10s = 0.33 GB, and the data amount of 10s two-view three-dimensional video is about 0.33 × 2 = 0.66 GB. As can be seen, the data amount of point cloud video far exceeds that of two-dimensional video and three-dimensional video of the same length. Therefore, in order to better realize data management, save server storage space, reduce transmission flow and transmission time between server and client, point cloud compression has become a key problem to promote the development of point cloud industry.

[0056] That is, since the point cloud is a collection of massive points, storing the point cloud will not only consume a large amount of memory, but also be not conducive to transmission, and there is no such large bandwidth to support the transmission of point cloud without compression in the network layer, therefore, the point cloud needs to be compressed.

[0057] At present, the point cloud coding framework that can compress point cloud can be a geometry-based point cloud compression (G-PCC) coding framework or a video-based point cloud compression (V-PCC) coding framework provided by a moving picture experts group (MPEG), or an AVS-PCC coding framework provided by an AVS. The G-PCC coding framework can be used for compressing a first type of static point cloud and a third type of dynamically acquired point cloud, which can be based on a test model compression 13 (TMC13), and the V-PCC coding framework can be used for compressing a second type of dynamic point cloud, which can be based on a test model compression 2 (TMC2). Therefore, the G-PCC coding framework is also referred to as a point cloud codec TMC13, and the V-PCC coding framework is also referred to as a point cloud codec TMC2.

[0058] An embodiment of the present application provides a network architecture of a point cloud coding system including a decoding method and an encoding method. FIG. 3 is a schematic diagram of a network architecture of a point cloud coding provided by an embodiment of the present application. As shown in FIG. 3, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. The electronic devices in the implementation process can be various types of devices with a point cloud coding function, for example, the electronic devices can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital telephone, a video telephone, a television, a sensing device, a server, and the like, and the present application is not limited thereto. The decoder or the encoder in the embodiment of the present application can be the above-mentioned electronic devices.

[0059] In the embodiment of the present application, the electronic device with the point cloud coding function generally includes a point cloud encoder (i.e., an encoder) and a point cloud decoder (i.e., a decoder).

[0060] The following describes related technologies by taking the G-PCC coding framework and the AVS coding framework as examples.

[0061] It can be understood that in the point cloud G-PCC coding framework, for the point cloud data to be encoded, the point cloud data is first divided into multiple slices through slice division. In each slice, the geometry information of the point cloud and the attribute information corresponding to each point are encoded separately.

[0062] FIG. 4A shows a schematic diagram of a G-PCC encoder. As shown in FIG. 4A, in the geometry encoding process, the geometry information is first converted in coordinates, so that all the point clouds are contained in a bounding box, and then quantized. After the point cloud quantization, a part of the point clouds have the same geometry information, and then it is determined based on parameters whether to remove the repeated points. The quantization and removal of repeated points are also called the voxelization process. Then the bounding box is divided by octree or prediction tree construction. In this process, the points in the divided leaf nodes are arithmetically encoded to generate a binary geometry bitstream; or the vertices generated by the division are arithmetically encoded (surface fitting based on the vertices) to generate a binary geometry bitstream. In the attribute encoding process, after the geometry encoding is completed and the geometry information is reconstructed, the color information (i.e., attribute information) is converted from the RGB color space to the YUV color space. Then, the point cloud is recolored using the reconstructed geometry information, so that the unencoded attribute information corresponds to the reconstructed geometry information. Attribute encoding is mainly for color information, and in the color information encoding process, there are mainly three transformation methods, which are: predicting transform (PT), lifting transform (LT), and region adaptive hierarchical transform (RAHT). PT and LT are based on the generation order of LOD (Level of Detail) to predictively encode the point cloud, and RAHT is based on the construction level of the octree to adaptively transform the attribute information from bottom to top. LT and RAHT convert the color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through transformation, and finally quantize the coefficients, and then arithmetically encode the quantized coefficients to generate a binary attribute bitstream.

[0063] FIG. 4B shows a schematic diagram of a G-PCC decoder. As shown in FIG. 4B, for the obtained binary bitstream, the geometry bitstream and the attribute bitstream in the binary bitstream are first independently decoded. In the decoding of the geometry bitstream, the geometry information of the point cloud is obtained through arithmetic decoding-reconstructing octree / reconstructing prediction tree-reconstructing geometry-coordinate inverse conversion; in the decoding of the attribute bitstream, the attribute information of the point cloud is obtained through arithmetic decoding-dequantization-LOD division / RAHT-color inverse conversion, and the point cloud data to be encoded (i.e., the output point cloud) is restored based on the geometry information and the attribute information.

[0064] It should be noted that, as shown in FIG. 4A or FIG. 4B, the current geometry coding of G-PCC can be divided into octree-based geometry coding (indicated by a dashed box) and prediction tree-based geometry coding (indicated by a dotted box).

[0065] For octree-based geometry encoding (OctGeomEnc), the octree-based geometry encoding includes: first, coordinate conversion is performed on the geometry information, so that all point clouds are contained in a bounding box. Then, quantization is performed, which mainly plays a scaling role. Due to quantization rounding, the geometry information of a part of points is the same. Whether to remove the duplicate points is determined according to a parameter. The quantization and removal of duplicate points are also called the voxelization process. Next, the bounding box is continuously divided into a tree (such as an octree, a quadtree, a binary tree, etc.) in a breadth-first traversal order, and the occupancy code of each node is encoded. In the related art, a company proposes an implicit geometry division method. First, the bounding box of the point cloud is calculated Assuming d x > d y > d z , the bounding box corresponds to a cuboid. In geometry division, first, a binary tree division is performed based on the x-axis to obtain two child nodes; until the condition d x = d y > d z is met, a quadtree division is performed based on the x and y axes to obtain four child nodes; and when the condition d x = d y = d z is met, an octree division is continuously performed until the leaf node obtained by the division is a 1x1x1 unit cube, and the points in the leaf node are encoded to generate a binary code stream. In the process of binary tree / quadtree / octree division, two parameters K and M are introduced. Parameter K indicates the maximum number of binary tree / quadtree divisions before octree division; and parameter M is used to indicate the minimum block edge length corresponding to the binary tree / quadtree division, which is 2 M . At the same time, K and M must satisfy the condition: assuming d max = max(d x , d y , d z ), d min = min(d x , d y , d z ), parameter K satisfies: K ≥ d max -d min ; and parameter M satisfies: M ≥ d minThe parameters K and M satisfy the above conditions because the priority of the partitioning mode in the current G-PCC is binary tree, quad tree and octree, and when the node block size does not satisfy the condition of binary tree / quad tree, the octree partitioning is performed on the node until the leaf node of the minimum unit 1*1*1 is reached. The geometry coding mode based on octree can effectively encode the geometry information of the point cloud by utilizing the correlation between the adjacent points in the space, but for some relatively flat nodes or nodes with plane characteristics, the coding efficiency of the geometry information of the point cloud can be further improved by utilizing the plane coding.

[0066] It is mentioned above that there are three transformation modes for the encoding of attribute information, one is prediction transformation, the second is promotion transformation, and the third is RAHT transformation. The RAHT transformation is described in detail below.

[0067] The RAHT transformation adopts the Haar wavelet transformation, which can transform the attribute information of the point cloud from the spatial domain to the frequency domain, thereby further reducing the correlation between the attribute information of the point cloud. Fig. 5 is an example diagram of the RAHT transformation process. As shown in Fig. 5, the RAHT is based on the hierarchical structure of the octree for wavelet transformation, thereby associating the attribute information with the octree nodes. The attribute information of the occupied nodes in the same parent node is recursively transformed in a bottom-up manner, and the nodes in each layer are respectively transformed from the x, y and z dimensions (see Fig. 6), until the root node of the octree is reached. In the process of hierarchical transformation, the direct current (DC) coefficients (or low-pass coefficients) obtained after the transformation of the nodes in the same layer are transmitted to the nodes in the upper layer for further transformation, and all the alternating current (AC) coefficients (or high-pass coefficients) are quantized and encoded.

[0068] Fig. 7 is a schematic diagram of the RAHT transformation and the RAHT inverse transformation. It is assumed that g′ L,2x,y,z and g′L,2x+1,y,z are two DC coefficients of the points adjacent to each other in the L layer. After the RAHT transformation, the information of the L-1 layer is the AC coefficient f′ L-1,x,y,z and the DC coefficient g′ L-1,x,y,z . f′ L-1,x,y,z will no longer be transformed and will be directly quantized and encoded, and g′ L-1,x,y,z will continue to search for adjacent points for transformation. If no adjacent points are found, g′ L-1,x,y,z can be directly transmitted to the L-2 layer. That is, the RAHT transformation is only valid for the nodes with adjacent points, and the nodes without adjacent points will be directly transmitted to the upper layer. In the above transformation process, g′ L,2x,y,zand the weight corresponding to g' L,2x+2,y,z (the weight can be determined based on the number of non-empty child nodes in the node) is w' L,2x,y,z and w' L,2x+1,y,z (abbreviated as w' 0 and w' 1), g' L-1,x,y,z The weight of g' L,2x+1,y,z is w' L-1,x,y,z The general transform formula of the RAHT transform is:

[0069] In formula (1), T w0,w1 is a transform matrix, which can be determined based on formula (2):

[0070] The transform matrix is updated adaptively with the change of the weight corresponding to each point. The above transform process is iteratively updated according to the division structure of the octree until the root node of the octree.

[0071] The RAHT transform includes two prediction modes: RAHT inter-frame prediction transform and RAHT intra-frame prediction transform. Next, the RAHT inter-frame prediction transform process is described in detail.

[0072] First, the RAHT attribute transform coding structure is constructed based on the geometric information, that is, the transformation is continuously performed at the voxel level until the root node is obtained, so as to complete the hierarchical transform coding of the entire attribute information. In this way, the inter-frame attribute coding structure is constructed, as shown in FIG. 8.

[0073] Then, the geometric information of the current node to be coded is used to obtain the same-position prediction node of the current node to be coded in the reference frame, and the geometric information and attribute information of the reference node are used to obtain the predicted attribute of the current node to be coded.

[0074] Next, the attribute prediction value of the current node to be coded is obtained according to the following two different ways:

[0075] The first way: the inter-frame prediction node of the current node is valid: that is, the same-position node exists, and the attribute of the prediction node is directly taken as the attribute prediction value of the current node to be coded;

[0076] The second way: the inter-frame prediction node of the current node is invalid: that is, the same-position node does not exist, and the attribute prediction value of the intra-frame adjacent node is used as the attribute prediction value of the node to be coded.

[0077] Finally, the attribute of the current node to be coded is predicted using the obtained attribute prediction value.

[0078] It is mentioned above that when the RAHT inter-frame prediction transform coding is performed, the RAHT attribute transform coding structure is first constructed based on the geometric information. Next, the multiple hierarchical structures of the RAHT attribute transform coding are described in detail in combination with FIG. 9 to facilitate understanding of the contents of the present scheme.

[0079] As shown in FIG. 9, a frame of point cloud can be divided into multiple slices. For a point cloud to be decoded, the point cloud is decoded independently according to the slices. One slice is a unit of RAHT transformation / inverse transformation, and one slice performs RAHT transformation / inverse transformation once. In one unit of RAHT transformation / inverse transformation, there are multiple RAHT transformation layers. One RAHT transformation layer can be further divided into multiple RAHT transformation blocks. The process of RAHT inter-frame prediction transformation coding can also be understood as the process of RAHT inter-frame prediction of a RAHT transformation block.

[0080] As mentioned above, the RAHT transformation is based on the wavelet transformation of the hierarchical structure of the octree, and the attribute information is associated with the nodes of the octree. The attribute information of the occupied nodes in the same parent node is recursively transformed in a bottom-up manner until the root node of the octree is reached. In the process of hierarchical transformation, all neighbor points in the same layer are transformed to obtain the transformation coefficients (or transformation blocks).

[0081] FIG. 10 shows a RAHT inter-frame prediction decoding framework of point cloud attribute information in the related art. As shown in FIG. 10, for a current frame to be encoded, the frame is encoded slice by slice, that is, one slice is a unit of RAHT transformation, that is, neighbor points in the same layer belonging to one slice are transformed to obtain a transformation block. In the related art, when performing RAHT inter-frame prediction coding, the geometry information and attribute information of the entire frame of the reference frame point cloud are used for RAHT transformation, that is, one frame is a unit of RAHT transformation, that is, neighbor points in the same layer belonging to one frame are transformed to obtain a prediction block. It can be seen that the number and spatial positions of the neighbor points of the two are obviously not equal, and there can be a large redundancy between the transformation coefficients corresponding to the transformation blocks and the transformation coefficients corresponding to the prediction blocks. Therefore, the RAHT transformation based on the entire frame of the reference frame to obtain the prediction block in the related art can not be optimal, and there can be a problem of low prediction accuracy.

[0082] To solve the above problem, an embodiment of the present application provides an encoding method, comprising: performing region adaptive hierarchical transformation (RAHT) on part of point clouds in a reference frame to determine at least one prediction block; performing RAHT transformation on a first to-be-encoded slice in a current frame to determine one or more transformation blocks; performing inter-frame prediction on a current transformation block in the one or more transformation blocks according to the at least one prediction block to determine a predicted value of an attribute transformation coefficient of the current transformation block; and determining a residual value of the attribute transformation coefficient of the current transformation block according to the predicted value of the attribute transformation coefficient of the current transformation block.

[0083] The embodiment of the present application further provides a decoding method, comprising: performing RAHT on part of point clouds in a reference frame to determine at least one prediction block; performing RAHT on a first to-be-decoded strip in a current frame to determine one or more transform blocks; performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of an attribute transform coefficient of the current transform block; and determining a reconstructed value of the attribute transform coefficient of the current transform block according to the predicted value of the attribute transform coefficient of the current transform block.

[0084] When performing RAHT inter-frame prediction coding, the embodiment of the present application performs RAHT on part of point clouds in a reference frame, and the obtained prediction block and transform block are more matched and the accuracy of inter-frame prediction is higher compared with the related art which performs RAHT on the whole frame of the reference frame, thereby helping to improve the coding and decoding performance of point cloud attribute information.

[0085] The decoding method provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0086] FIG. 11 is a flowchart of a decoding method provided by the embodiment of the present application. The decoding method of FIG. 11 can be applied to a decoder. The decoding method of FIG. 11 can be used to decode attribute information of a point cloud. In some implementations, the decoding method can be applied to G-PCC. Alternatively, in other implementations, the decoding method can be applied to a geometry explore solid test model (GES-TM). The GES-TM is a coding framework proposed for dense point clouds, such as point clouds collected in an augmented reality (AR) or virtual reality (VR) scene.

[0087] Referring to FIG. 11, in step S1110, RAHT is performed on part of point clouds in a reference frame to determine at least one prediction block. The reference frame here can be a motion-compensated reference frame or a non-motion-compensated reference frame.

[0088] In step S1120, RAHT is performed on a first to-be-decoded strip in a current frame to determine one or more transform blocks.

[0089] The transform block corresponds to an attribute transform coefficient, such as a low-pass coefficient or a high-pass coefficient, of attribute information after RAHT. The prediction block can be understood as a transform block in a reference frame used for inter-frame prediction.

[0090] The partial point cloud mentioned above can be obtained in various ways. In some embodiments, the partial point cloud can be determined based on the geometric information of the slice in the current frame. For example, the partial point cloud can include all points in a bounding box (or a first bounding box), which is determined according to the geometric information of the slice to be decoded.

[0091] For example, first, the size of the bounding box corresponding to the current slice is determined; then, the reference frame point cloud is divided according to the position information of the current slice and the size of the bounding box; finally, the points of the reference frame in the bounding box are subjected to RAHT transformation as the partial point cloud.

[0092] In the embodiments of the present application, the partial point cloud in the reference frame determined based on the geometric information (such as the bounding box) of the slice to be decoded is more similar to the spatial position information and attribute information of the slice to be decoded, and thus the prediction block and the transform block are more matched, which helps to improve the accuracy of inter-frame prediction.

[0093] In some scenarios, there can be some special slices to be decoded in the current frame, and the bounding box of the slice to be decoded cannot find the partial point cloud for determining the prediction block in the reference frame, for example, the bounding box can not contain points in the reference frame.

[0094] Suppose that the current frame includes a first slice to be decoded and a first slice. The first slice and the first slice to be decoded are two different slices, the first slice to be decoded corresponds to a first bounding box (determined based on geometric position information), and the first slice corresponds to a second bounding box. In the embodiments of the present application, in the case that the first slice to be decoded cannot find the partial point cloud in the reference frame based on the first bounding box, the first slice to be decoded can also find the partial point cloud in the reference frame based on the second bounding box for RAHT transformation. Or, the first slice to be decoded can use the prediction block corresponding to the first slice for inter-frame prediction.

[0095] The above-mentioned first slice can be any slice in the current frame except the first slice to be decoded. The first slice can be a decoded slice or an undecoded slice. For example, the first slice can be the slice before the first slice to be decoded in the decoding order. Since the geometric position information of the first slice is more likely to be similar to the geometric position information of the slice before it in the decoding order, the partial point cloud in the reference frame determined based on the bounding box of the slice before it is closer to the first slice, thereby helping to improve the accuracy of prediction.

[0096] Alternatively, in the case that the first slice to be decoded cannot find the partial point cloud in the reference frame based on the first bounding box, the prediction block of the first slice to be decoded can also be obtained based on the RAHT transformation of the entire reference frame, and the inter-frame prediction of the transform block of the first slice to be decoded is performed.

[0097] In some implementations, the part of the point cloud can also be a slice of the reference frame. As mentioned above, for the current frame, decoding is performed slice by slice, that is, a slice is a unit of RAHT transformation, that is, neighbor points belonging to the same slice are transformed to obtain a transformed block. Therefore, if the reference frame is also a slice for RAHT transformation, the obtained prediction block and the transformed block can be more matched.

[0098] For example, a second slice in the reference frame can be subjected to RAHT transformation to obtain a prediction block, and a third slice in the current frame can be subjected to RAHT transformation to obtain a transformed block. Then, the transformed block is inter-frame predicted based on the prediction block. In order to make the second slice and the third slice more matched, identification information of the second slice and identification information of the third slice can be set to correspond to each other. For example, an identification value of the second slice and an identification value of the third slice are the same.

[0099] In step S1130, a current transformed block in the one or more transformed blocks is inter-frame predicted according to at least one prediction block to determine a prediction value of an attribute transform coefficient of the current transformed block.

[0100] For the current frame to be decoded, decoding is performed slice by slice, that is, a slice is a unit of RAHT transformation, that is, neighbor points belonging to the same slice are transformed to obtain a transformed block. In the related art, when performing RAHT inter-frame prediction decoding, the geometry information and the attribute information of the whole frame of the point cloud of the reference frame are used for RAHT transformation, that is, a frame is a unit of RAHT transformation, that is, neighbor points belonging to the same frame are transformed to obtain a prediction block. It can be seen that the number and spatial positions of the neighbor points of the two are obviously not equal, and there can be a large redundancy between the transform coefficients corresponding to the transformed block and the transform coefficients corresponding to the prediction block. Therefore, the prediction block obtained by the related art based on the whole frame of the reference frame for RAHT transformation can not be optimal, and there can be a problem of low prediction accuracy.

[0101] The embodiments of the present application use part of the point cloud of the reference frame for RAHT transformation when performing RAHT inter-frame prediction decoding. Compared with the related art based on the whole frame of the reference frame for RAHT transformation, the prediction block obtained is more matched with the transformed block, and the accuracy of inter-frame prediction is higher, thereby helping to improve the coding and decoding performance of the point cloud attribute information.

[0102] In some implementations, the manner of performing step S1130 can include: determining a prediction block of the current transformed block from the at least one prediction block; and then determining a prediction value of an attribute transform coefficient of the current block according to an attribute transform coefficient of the prediction block of the current transformed block.

[0103] In some implementations, the prediction block of the current transform block can be determined based on geometric location information. For example, first geometric location information can be determined based on the current block; then, the prediction block of the current transform block can be determined from at least one prediction block based on the first geometric location information. This first geometric location information may, for example, include coordinate information and / or the hierarchical information of the RAHT layer.

[0104] The first geometric position information mentioned above can be the geometric position information of the parent block of the current transform block; or, the first geometric position information can also be the geometric position information of the current transform block.

[0105] For example, a transform block in the reference frame with the same geometric position as the current transform block can be used as a prediction block. Alternatively, a parent block in the reference frame can be found whose geometric position is the same as the parent block of the current transform block; then, a child block from that parent block can be used as a prediction block.

[0106] In step S1140, the predicted value of the attribute transformation coefficient of the current block is determined based on the attribute transformation coefficient of the predicted block of the current transform block.

[0107] In some implementations, the method of performing step S1140 may include: using the sum of the predicted value of the attribute transformation coefficient of the current transform block and the residual value of the attribute transformation coefficient of the current transform block as the reconstructed value of the attribute transformation coefficient of the current transform block.

[0108] The test results obtained by testing the encoding and decoding method provided in the embodiments of this application are described below.

[0109] In Table 1, End-to-End BD-AttrRate( The delta attribute rate (BD-AttrRate) represents the bitrate savings of the end-to-end attribute bitstream under a fixed PSNR. A decrease in BD-AttrRate means a reduction in bitrate and improved performance while maintaining the same PSNR; conversely, an increase in BD-AttrRate means an increase in bitrate and decreased performance while maintaining the same PSNR. In other words, the greater the decrease in BD-AttrRate, the better the compression effect.

[0110] According to the test results in Table 1, the encoding and decoding method provided in this application embodiment achieves a 3.6% gain in reflectivity under C2 conditions, thus improving the encoding and decoding performance.

[0111] Table 1: Performance test results under multi-slice conditions based on TMC13V25

[0112] The decoding method provided by the embodiments of the present application is described in detail above in combination with FIG. 11. The encoding method provided by the embodiments of the present application is described in detail below in combination with FIG. 12.

[0113] FIG. 12 is a flowchart of an encoding method provided by the embodiments of the present application. The encoding method of FIG. 12 can be applied to an encoder. The encoding method of FIG. 12 can be used to encode attribute information of a point cloud. In some implementations, the encoding method can be applied to G-PCC. Alternatively, in some other implementations, the encoding method can be applied to GES-TM. GES-TM is a coding framework proposed for dense point clouds (such as point clouds collected by AR or VR scenes).

[0114] Referring to FIG. 12, in step S1210, a part of the point cloud in the reference frame is subjected to RAHT transformation to determine at least one prediction block. The reference frame here can be a motion-compensated reference frame or a reference frame that is not motion-compensated.

[0115] In step S1220, the first to-be-encoded slice in the current frame is subjected to RAHT transformation to determine one or more transformation blocks.

[0116] The transformation block corresponds to attribute transformation coefficients of the attribute information after RAHT transformation, such as low-pass coefficients or high-pass coefficients. The prediction block can be understood as a transformation block in the reference frame used for inter-frame prediction.

[0117] The part of the point cloud mentioned above can be obtained in various ways. In some implementations, the part of the point cloud can be determined based on the geometric information of the slice in the current frame. For example, the part of the point cloud can include all points in a bounding box (or a first bounding box), and the bounding box is determined according to the geometric information of the first to-be-encoded slice.

[0118] For example, first, the size of the bounding box corresponding to the current slice is determined; then, the reference frame point cloud is divided according to the position information of the current slice and the size of the bounding box; and finally, the points of the reference frame in the bounding box are subjected to RAHT transformation as the part of the point cloud.

[0119] In the embodiments of the present application, the part of the point cloud in the reference frame determined based on the geometric information (such as the bounding box) of the to-be-encoded slice is more similar to the spatial position information and the attribute information of the to-be-encoded slice, and thus the prediction block and the transformation block are more matched, which helps to improve the accuracy of inter-frame prediction.

[0120] In some scenarios, there can be some special to-be-encoded slices in the current frame, and the bounding box of the to-be-encoded slice cannot find a part of the point cloud in the reference frame to determine a prediction block, for example, the bounding box can not contain points in the reference frame.

[0121] It is assumed that the current frame includes a first to-be-encoded slice and a first slice. The first slice and the first to-be-encoded slice are two different slices, the first to-be-encoded slice corresponds to a first bounding box (determined based on geometric position information), and the first slice corresponds to a second bounding box. In the embodiment of the present application, in the case that the first to-be-encoded slice cannot find part of the point cloud in the reference frame based on the first bounding box, the first to-be-encoded slice can also find part of the point cloud in the reference frame based on the second bounding box to perform RAHT transformation. In other words, the first to-be-encoded slice can use the prediction block corresponding to the first slice to perform inter-frame prediction.

[0122] The above-mentioned first slice can be any slice in the current frame except the first to-be-encoded slice. The first slice can be an encoded slice or an unencoded slice. For example, the first slice can be the slice before the first to-be-encoded slice in the encoding order. Since the geometric position information of the first slice is more likely to be similar to the geometric position information of the slice before it in the encoding order, the part of the point cloud in the reference frame determined using the bounding box of the slice before it is closer to the first slice, thereby helping to improve the accuracy of prediction.

[0123] Alternatively, in the case that the first to-be-encoded slice cannot find part of the point cloud in the reference frame based on the first bounding box, the transformation block of the first to-be-encoded slice can also be inter-frame predicted based on the prediction block obtained by performing RAHT transformation on the entire reference frame.

[0124] In other implementations, the part of the point cloud can also be a slice of the reference frame. As mentioned above, the current frame is encoded slice by slice, that is, a slice is a unit of RAHT transformation, that is, the neighbor points belonging to the same slice in the same layer are transformed to obtain a transformation block. Therefore, if the reference frame is also a slice for RAHT transformation, the prediction block obtained by the transformation can be more matched with the transformation block.

[0125] For example, the second slice in the reference frame can be subjected to RAHT transformation to obtain a prediction block, and the third slice in the current frame can be subjected to RAHT transformation to obtain a transformation block. Then, the transformation block is inter-frame predicted based on the prediction block. In order to make the second slice and the third slice more easily matched, the identification information of the second slice and the identification information of the third slice can be set to correspond to each other. For example, the identification value of the second slice and the identification value of the third slice are set to be the same.

[0126] In step S1230, the current transformation block in the one or more transformation blocks is inter-frame predicted according to at least one prediction block to determine a predicted value of the attribute transformation coefficient of the current transformation block.

[0127] For the current frame to be encoded, it is encoded slice by slice, that is, a slice is a unit of RAHT transformation, that is, the neighbor points in the same layer belonging to a slice are transformed to obtain a transform block. In the related art, when performing RAHT inter-frame prediction encoding, the geometry information and attribute information of the whole frame of the reference frame point cloud are used for RAHT transformation, that is, a frame is a unit of RAHT transformation, that is, the neighbor points in the same layer belonging to a frame are transformed to obtain a prediction block. It can be seen that the number and spatial position of the neighbor points of the two are obviously not equal, and there may be a large redundancy between the transform coefficients corresponding to the transform block and the transform coefficients corresponding to the prediction block. Therefore, the related art of performing RAHT transformation on the whole frame of the reference frame to obtain a prediction block may not be optimal, and there may be a problem of low prediction accuracy.

[0128] The embodiment of the present application uses part of the point cloud of the reference frame to perform RAHT transformation when performing RAHT inter-frame prediction encoding. Compared with the related art of performing RAHT transformation on the whole frame of the reference frame, the prediction block obtained is more matched with the transform block, and the accuracy of inter-frame prediction is higher, thereby helping to improve the coding and decoding performance of the point cloud attribute information.

[0129] In some implementations, the manner of performing step S1230 can include determining a prediction block of the current transform block from the at least one prediction block, and then determining a prediction value of the attribute transform coefficient of the current block according to the attribute transform coefficient of the prediction block of the current transform block.

[0130] In some implementations, the prediction block of the current transform block can be determined according to the geometry position information. For example, first geometry position information can be determined according to the current block, and then the prediction block of the current transform block is determined from the at least one prediction block according to the first geometry position information. The first geometry position information here can include coordinate information and / or layer level information of the RAHT layer, for example.

[0131] The above-mentioned first geometry position information can be the geometry position information of the parent block of the current transform block, or the first geometry position information can also be the geometry position information of the current transform block.

[0132] For example, a transform block with the same geometry position as the geometry position of the current transform block can be found in the reference frame as a prediction block. For another example, a parent block with the same geometry position as the geometry position of the parent block of the current transform block can be found in the reference frame, and then a child block in the parent block is taken as a prediction block.

[0133] In step S1240, a residual value of the attribute transform coefficient of the current transform block is determined according to the prediction value of the attribute transform coefficient of the current transform block.

[0134] In some implementations, the manner of performing step S1240 can include taking the difference between the predicted value of the attribute transform coefficient of the current transform block and the original value of the attribute transform coefficient of the current transform block as the residual value of the attribute transform coefficient of the current transform block.

[0135] The embodiments of the present application will be described in more detail below with reference to specific examples. It should be noted that the examples below are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific values or specific scenarios exemplified. Those skilled in the art can obviously make various equivalent modifications or changes to the examples given below, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0136] The present example first calculates the bounding box size of the current slice of the current frame to be decoded, and divides the points of the reference frame point cloud according to the calculated position and bounding box size of the current slice. Then, the points belonging to the bounding box in the reference frame are retained to obtain a new sub-reference frame point cloud (i.e., the partial point cloud in the foregoing), and the sub-reference frame point cloud is subjected to RAHT transformation to obtain the prediction block of the current slice. If the bounding box corresponding to the current slice cannot find a sub-reference frame point cloud in the reference frame, the current slice can use a decoded slice sub-reference frame point cloud for inter-frame prediction.

[0137] The RAHT inter-frame prediction transformation decoding of the attribute information provided by the present example is described in detail below in conjunction with FIG. 13.

[0138] As shown in FIG. 13, the decoding end process of the present example includes:

[0139] (1) Calculate the bounding box size of the current slice x of the current frame to be decoded;

[0140] (2) Generate a sub-point cloud sequence: divide the points of the reference frame point cloud, and retain the points whose positions belong to the bounding box in the reference frame to obtain a new sub-reference frame point cloud pointcloud x;

[0141] (3) Perform RAHT transformation on the sub-reference frame point cloud pointcloud x to obtain the RAHT inter-frame reference information of the current slice x to be decoded;

[0142] For all current transform blocks of the current slice x to be decoded, when performing RAHT inter-frame prediction, it can include:

[0143] (4) RAHT inter-frame prediction: calculate the geometric position of the current transform block; find the prediction block of the current transform block:

[0144] (a) finding a transform block in the reference frame having the same geometric position as the current transform block as the prediction block; or

[0145] (b) finding a parent block in the reference frame having the same geometric position as the parent block of the current transform block; and then finding a child block in the parent block in the reference frame as the prediction block of the current transform block.

[0146] The RAHT inter-prediction transform coding mode of the attribute information provided by the present example is described in detail in the foregoing, and is not repeated here.

[0147] The method embodiments of the present application are described in detail above in combination with FIGS. 1 to 13, and the device embodiments of the present application are described in detail below in combination with FIGS. 14 to 17. It should be understood that the description of the method embodiments corresponds to the description of the device embodiments, and therefore, the parts not described in detail can be referred to the foregoing method embodiments.

[0148] FIG. 14 is a structural schematic diagram of a decoder provided by an embodiment of the present application. As shown in FIG. 14, the decoder 1400 can include a first transform unit 1410, a second transform unit 1420, a prediction unit 1430, and a determination unit 1440.

[0149] The first transform unit 1410 is configured to perform region adaptive hierarchical transform (RAHT) on part of a point cloud in a reference frame to determine at least one prediction block.

[0150] The second transform unit 1420 is configured to perform RAHT transform on a first to-be-decoded strip in a current frame to determine one or more transform blocks.

[0151] The prediction unit 1430 is configured to perform inter-prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of an attribute transform coefficient of the current transform block.

[0152] The determination unit 1440 is configured to determine a reconstructed value of the attribute transform coefficient of the current transform block according to the predicted value of the attribute transform coefficient of the current transform block.

[0153] In some implementations, the part of the point cloud is determined based on geometric information of a strip in the current frame.

[0154] In some implementations, the points of the part of the point cloud are in a first bounding box, and the first bounding box is determined based on geometric information of the first to-be-decoded strip.

[0155] In some embodiments, the points of the partial point cloud are in a second bounding box, and the second bounding box is determined based on geometry information of a first slice in the current frame, and the first slice is different from the first to-be-decoded slice.

[0156] In some embodiments, the first slice is a decoded slice.

[0157] In some embodiments, the first slice is a previous slice of the first to-be-decoded slice in a decoding order.

[0158] In some embodiments, the partial point cloud is a second slice.

[0159] In some embodiments, the first to-be-decoded slice is a third slice, and the identification information of the second slice corresponds to identification information of the third slice.

[0160] In some embodiments, the prediction unit 1430 is further configured to determine, from the at least one prediction block, a prediction block of the current transform block; and determine a prediction value of the attribute transform coefficient of the current block according to a transform coefficient of the prediction block of the current transform block.

[0161] In some embodiments, the prediction unit 1430 is further configured to determine, according to the current block, first geometry position information; and determine, according to the first geometry position information, the prediction block of the current transform block from the at least one prediction block.

[0162] In some embodiments, the first geometry position information is geometry position information of a parent block of the current transform block, or the first geometry position information is geometry position information of the current transform block.

[0163] In some embodiments, the prediction unit 1430 is further configured to determine, according to the first geometry position information, a parent block of the one or more prediction blocks; and determine, according to the parent block of the one or more prediction blocks, the prediction block of the current transform block.

[0164] In some embodiments, the first geometry position information includes coordinate information and / or level information of a RAHT layer.

[0165] It can be understood that, in the embodiments of the present application, the "unit" can be part of a circuit, part of a processor, part of a program or software, etc., and of course can also be a module, and can also be non-modular. Moreover, the components in the embodiments can be integrated in a processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0166] The integrated unit, if implemented in the form of a software function module and not sold or used as an independent product, can be stored in a computer readable storage medium based on such understanding. The technical solutions of the embodiments essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0167] Therefore, the embodiments of the present application provide a computer readable storage medium applied to the decoder 1400, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the decoding method in any one of the foregoing embodiments.

[0168] Based on the composition of the decoder 1400 and the computer readable storage medium, referring to FIG. 15, a specific hardware structure schematic diagram of the decoder 1400 provided by the embodiments of the present application is shown. As shown in FIG. 15, the decoder 1500 can include a communication interface 1510, a memory 1520, and a processor 1530; and the various components are coupled together through a bus system 1540. It can be understood that the bus system 1540 is used to realize the connection communication between the components. The bus system 1540 includes a data bus, a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 1540 in FIG. 15. Among them,

[0169] The communication interface 1510 is configured to receive and send signals in the process of transceiving information with other external network elements;

[0170] The memory 1520 is configured to store a computer program;

[0171] The processor 1530 is configured to execute the following when running the computer program:

[0172] performing region adaptive hierarchical transform (RAHT) on part of the point cloud in the reference frame to determine at least one prediction block;

[0173] performing RAHT transform on the first to-be-decoded strip in the current frame to determine one or more transform blocks;

[0174] According to the at least one prediction block, inter-predict a current transform block of the one or more transform blocks to determine predicted values of attribute transform coefficients of the current transform block;

[0175] According to the predicted values of the attribute transform coefficients of the current transform block, determine reconstructed values of the attribute transform coefficients of the current transform block.

[0176] It can be understood that the memory 1520 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (read-only memory, ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), or a flash memory. The volatile memory can be a random access memory (random access memory, RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct memory bus random access memory (direct rambus RAM, DRRAM). The memory 1520 of the system and method described in the present application is intended to include but not limited to these and any other suitable types of memory.

[0177] The processor 1515 can be an integrated circuit chip on which signal processing capabilities are implemented. In implementation, the steps of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 1515. The processor 1515 described above can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 1520, and the processor 1515 reads the information in the memory 1520, and combines the hardware to complete the steps of the above method.

[0178] It can be understood that the embodiments described in the present application can be realized by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be realized in one or more application specific integrated circuits (ASICs), digital signal processing (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for executing functions described in the present application or a combination thereof. For software implementation, the technology described in the present application can be realized by modules (such as processes, functions, etc.) for executing functions described in the present application. The software code can be stored in the memory and executed by the processor. The memory can be implemented in the processor or outside the processor.

[0179] Optionally, as another embodiment, the processor 1515 is further configured to, when executing the computer program, perform the decoding method of any one of the preceding embodiments.

[0180] FIG. 16 is a structural schematic diagram of an encoder provided by an embodiment of the present application. As shown in FIG. 16, the encoder 1600 includes a first transform unit 1610, a second transform unit 1620, a prediction unit 1630, and a determination unit 1640.

[0181] The first transform unit 1610 is configured to perform region adaptive hierarchical transform (RAHT) on a part of point cloud in a reference frame to determine at least one prediction block.

[0182] The second transform unit 1620 is configured to perform RAHT transform on a first to-be-decoded slice in a current frame to determine one or more transform blocks.

[0183] The prediction unit 1630 is configured to perform inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a prediction value of an attribute transform coefficient of the current transform block.

[0184] The determination unit 1640 is configured to determine a residual value of the attribute transform coefficient of the current transform block according to the prediction value of the attribute transform coefficient of the current transform block.

[0185] In some implementations, the part of point cloud is determined based on geometric information of a slice in the current frame.

[0186] In some implementations, points of the part of point cloud are in a first bounding box, and the first bounding box is determined based on geometric information of the first to-be-encoded slice.

[0187] In some implementations, points of the part of point cloud are in a second bounding box, and the second bounding box is determined based on geometric information of a first slice in the current frame, and the first slice and the first to-be-encoded slice are two different slices.

[0188] In some implementations, the first slice is an encoded slice.

[0189] In some implementations, the first slice is a previous slice of the first to-be-encoded slice in an encoding order.

[0190] In some implementations, the part of point cloud is a second slice.

[0191] In some implementations, the first to-be-encoded slice is a third slice, and identification information of the second slice corresponds to identification information of the third slice.

[0192] In some embodiments, the prediction unit 1630 is further configured to determine a prediction block of the current transform block from the at least one prediction block; and determine the prediction value of the attribute transform coefficient of the current block according to a transform coefficient of the prediction block of the current transform block.

[0193] In some embodiments, the prediction unit 1630 is further configured to determine first geometric position information according to the current block; and determine the prediction block of the current transform block from the at least one prediction block according to the first geometric position information.

[0194] In some embodiments, the first geometric position information is geometric position information of a parent block of the current transform block; or the first geometric position information is geometric position information of the current transform block.

[0195] In some embodiments, the prediction unit 1630 is further configured to determine a parent block of the at least one prediction block according to the first geometric position information; and determine the prediction block of the current transform block according to the parent block of the at least one prediction block.

[0196] In some embodiments, the first geometric position information includes coordinate information and / or level information of a RAHT layer.

[0197] It can be understood that, in the embodiments of the present application, the "unit" can be a part of circuit, a part of processor, a part of program or software, etc., and of course can also be a module, and can also be non-modular. Moreover, the components in the embodiments can be integrated in a processing unit, or can be physically present individually, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0198] When the integrated unit is realized in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the embodiments. The foregoing storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0199] Therefore, the embodiment of the present application provides a computer readable storage medium applied to the encoder 1600, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the decoding method in any one of the foregoing embodiments.

[0200] Based on the components of the encoder 1600 and the computer readable storage medium, referring to FIG. 17, a specific hardware structure schematic diagram of the encoder 1600 is shown. As shown in FIG. 17, the encoder 1700 can include a communication interface 1710, a memory 1720 and a processor 1730; each component is coupled together through a bus system 1740. It can be understood that the bus system 1740 is used to realize the connection communication between the components. The bus system 1740 includes a data bus, a power bus, a control bus and a status signal bus. However, in order to clearly illustrate, various buses are marked as the bus system 1740 in FIG. 17. Among them,

[0201] The communication interface 1710 is used for receiving and sending signals in the process of transceiving information with other external network elements;

[0202] The memory 1720 is used for storing a computer program;

[0203] The processor 1730 is used for executing the following when running the computer program:

[0204] Performing region adaptive hierarchical transform (RAHT) on part of the point cloud in the reference frame to determine at least one prediction block;

[0205] Performing RAHT transform on a first to-be-encoded strip in the current frame to determine one or more transform blocks;

[0206] Performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of an attribute transform coefficient of the current transform block;

[0207] Determining a residual value of the attribute transform coefficient of the current transform block according to the predicted value of the attribute transform coefficient of the current transform block.

[0208] It is to be understood that the memory 1720 in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a ROM, a PROM, an EPROM, an EEPROM, or a flash memory. The volatile memory can be a RAM used as an external cache. By way of example, and not limitation, a number of forms of RAM can be used, such as SRAM, DRAM, SDRAM, DDR SDRAM, ESDRAM, SLDRAM, and DRDRAM. The memory 1720 of the system and method described in the present application is intended to include, but not be limited to, these and any other suitable types of memory.

[0209] The processor 1730 can be an integrated circuit chip having a processing capability for signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 1730 or the instructions in the form of software. The processor 1730 described above can be a general processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory 1720, and the processor 1730 reads the information in the memory 1720 and completes the steps of the above method in combination with the hardware.

[0210] It can be understood that the embodiments described in the present application can be realized by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more ASICs, DSPs, DSPDs, PLDs, FPGAs, general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof. For software implementation, the technology described in the present application can be implemented by modules (such as processes, functions, etc.) for performing the functions described in the present application. The software code can be stored in the memory and executed by the processor. The memory can be implemented in the processor or outside the processor.

[0211] Alternatively, as another embodiment, the processor 1730 is further configured to execute the encoding method described in any one of the preceding embodiments when running the computer program.

[0212] The embodiment of the present application further provides a computer readable storage medium, which is a nonvolatile computer readable storage medium for storing a bitstream, the bitstream can be generated by using an encoding method of an encoder, or the bitstream is decoded by using a decoding method of a decoder, wherein the decoding method can be the decoding method in any one of the foregoing embodiments, and the encoding method can be the encoding method in any one of the foregoing embodiments.

[0213] It should be noted that, in the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0214] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0215] The methods disclosed in the several method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0216] The features disclosed in the several product embodiments provided by the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0217] The features disclosed in the several method or device embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method or device embodiments.

[0218] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A point cloud decoding method applied to a decoder, comprising: performing region adaptive hierarchical transform (RAHT) on a part of point clouds in a reference frame to determine at least one prediction block; performing RAHT on a first to-be-decoded slice in a current frame to determine one or more transform blocks; performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of attribute transform coefficients of the current transform block; determining a reconstructed value of the attribute transform coefficients of the current transform block according to the predicted value of the attribute transform coefficients of the current transform block.

2. The method of claim 1, wherein, The part of point clouds is determined based on geometry information of a slice in the current frame.

3. The method of claim 2, wherein, Points of the part of point clouds are in a first bounding box, which is determined based on geometry information of the first to-be-decoded slice.

4. The method of claim 2, wherein, Points of the part of point clouds are in a second bounding box, which is determined based on geometry information of a first slice in the current frame, and the first slice and the first to-be-decoded slice are two different slices.

5. The method of claim 4, wherein, The first slice is a decoded slice.

6. The method of claim 4 or 5, wherein, The first slice is a previous slice of the first to-be-decoded slice in a decoding order.

7. The method of claim 1, wherein, The part of point clouds is a second slice.

8. The method of claim 7, wherein, The first to-be-decoded slice is a third slice, and identification information of the second slice is the same as identification information of the third slice.

9. The method of claim 1, wherein, The performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of attribute transform coefficients of the current transform block comprises: determining a prediction block of the current transform block from the at least one prediction block; determining the predicted value of the attribute transform coefficients of the current block according to attribute transform coefficients of the prediction block of the current transform block.

10. The method of claim 9, wherein, The determining a prediction block of the current transform block from the at least one prediction block comprises: determining first geometry position information according to the current block; determining the prediction block of the current transform block from the at least one prediction block according to the first geometry position information.

11. The method of claim 10, wherein: the first geometry position information is geometry position information of a parent block of the current transform block; or the first geometry position information is geometry position information of the current transform block.

12. The method of claim 11, wherein, The determining the prediction block of the current transform block from the at least one prediction block according to the first geometry position information comprises: determining a parent block of the at least one prediction block according to the first geometry position information; determining the prediction block of the current transform block according to the parent block of the at least one prediction block.

13. The method of any one of claims 10 to 12, wherein, The first geometry position information comprises: coordinate information and / or level information of an RAHT layer.

14. A point cloud encoding method applied to an encoder, comprising: performing region adaptive hierarchical transform (RAHT) on a part of point clouds in a reference frame to determine at least one prediction block; performing RAHT on a first to-be-encoded slice in a current frame to determine one or more transform blocks; performing inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a predicted value of attribute transform coefficients of the current transform block; According to the prediction value of the attribute transform coefficient of the current transform block, a residual value of the attribute transform coefficient of the current transform block is determined.

15. The method of claim 14, wherein, The partial point cloud is determined based on geometry information of a slice in the current frame.

16. The method of claim 15, wherein, The points of the partial point cloud are in a first bounding box, and the first bounding box is determined based on geometry information of the first to-be-encoded slice.

17. The method of claim 15, wherein, The points of the partial point cloud are in a second bounding box, and the second bounding box is determined based on geometry information of a first slice in the current frame, and the first slice and the first to-be-encoded slice are two different slices.

18. The method of claim 17, wherein, The first slice is an encoded slice.

19. The method of claim 17 or 18, wherein, The first slice is a previous slice of the first to-be-encoded slice in an encoding order.

20. The method of claim 14, wherein, The partial point cloud is a second slice.

21. The method of claim 20, wherein, The first to-be-encoded slice is a third slice, and the identification information of the second slice is the same as the identification information of the third slice.

22. The method of claim 14, wherein, The determining, according to the at least one prediction block, the prediction value of the attribute transform coefficient of the current transform block in the one or more transform blocks, comprises: determining, from the at least one prediction block, a prediction block of the current transform block; determining, according to the attribute transform coefficient of the prediction block of the current transform block, the prediction value of the attribute transform coefficient of the current block.

23. The method of claim 22, wherein, The determining, from the at least one prediction block, the prediction block of the current transform block, comprises: determining first geometry position information according to the current block; determining, according to the first geometry position information, the prediction block of the current transform block from the at least one prediction block.

24. The method of claim 23, wherein: the first geometry position information is geometry position information of a parent block of the current transform block; or the first geometry position information is geometry position information of the current transform block.

25. The method of claim 24, wherein, The determining, according to the first geometry position information, the prediction block of the current transform block from the at least one prediction block, comprises: determining, according to the first geometry position information, a parent block of the at least one prediction block; determining, according to the parent block of the at least one prediction block, the prediction block of the current transform block.

26. The method of any one of claims 23-25, wherein, The first geometry position information comprises: coordinate information and / or level information of a RAHT layer.

27. A decoder, comprising: a first transform unit configured to perform region adaptive hierarchical transform (RAHT) on a partial point cloud in a reference frame to determine at least one prediction block; a second transform unit configured to perform RAHT on a first to-be-decoded slice in a current frame to determine one or more transform blocks; a prediction unit configured to perform inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a prediction value of an attribute transform coefficient of the current transform block; a determination unit configured to determine a reconstructed value of the attribute transform coefficient of the current transform block according to the prediction value of the attribute transform coefficient of the current transform block.

28. A decoder, comprising: a memory configured to store a computer program; a processor configured to execute the method of any one of claims 1 to 13 when running the computer program.

29. An encoder, comprising: a first transform unit, configured to perform a region-adaptive hierarchical transform (RAHT) on a part of point cloud in a reference frame to determine at least one prediction block; a second transform unit, configured to perform a RAHT on a first to-be-encoded slice in a current frame to determine one or more transform blocks; a prediction unit, configured to perform inter-frame prediction on a current transform block in the one or more transform blocks according to the at least one prediction block to determine a prediction value of attribute transform coefficients of the current transform block; a determination unit, configured to determine a residual value of the attribute transform coefficients of the current transform block according to the prediction value of the attribute transform coefficients of the current transform block.

30. An encoder, comprising: a memory configured to store a computer program; a processor configured to execute the computer program to perform the method of any one of claims 14-26.

31. A non-transitory computer readable storage medium storing a bitstream, the bitstream being generated by utilizing an encoding method of an encoder, or the bitstream being decoded by utilizing a decoding method of a decoder, wherein, The decoding method is the method of any one of claims 1-13, and the encoding method is the method of any one of claims 14-26.

32. A computer readable storage medium, storing a bitstream generated by the method of any one of claims 14-26.

33. A computer readable storage medium, wherein, The computer readable storage medium stores a computer program, and the computer program is executed to implement the method of any one of claims 1-13 or 14-26.

Images