Method, device and medium for point cloud coding and decoding

Abstract

The embodiment of the invention provides a solution for point cloud coding and decoding. The invention provides a method for point cloud coding and decoding. In the method, for a transition between a current frame of a point cloud sequence and a bitstream of the point cloud sequence, a prediction mode for a region of the current frame is determined. The prediction mode includes at least a first mode based on intra prediction and inter prediction. A prediction of the region is determined based on the prediction mode. The conversion is performed based on the prediction.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to video encoding and decoding techniques, and more specifically, to prediction mode determination. Background Technology

[0002] A point cloud is a collection of data points in a three-dimensional (3D) plane, where each point has defined coordinates on the X, Y, and Z axes. Therefore, point clouds can be used to represent the physical content of three-dimensional space. For a wide range of immersive applications, from augmented reality to autonomous vehicles, point clouds have proven to be a promising way to represent 3D visual data.

[0003] Point cloud encoding and decoding standards have largely evolved from the well-known MPEG organization. MPEG stands for Moving Picture Experts Group, one of the main standardization groups for multimedia processing. In 2017, the MPEG 3D Graphics Codec Group (3DG) released a Call for Proposals (CFP) document to begin developing point cloud encoding and decoding standards. The final standard will encompass two categories of solutions. Video-based point cloud compression (V-PCC or VPCC) is suitable for point sets with relatively uniform point distribution. Geometry-based point cloud compression (G-PCC or GPCC) is suitable for sparser distributions. However, the overall expectation is to further improve the encoding and decoding efficiency of conventional point cloud encoding and decoding techniques. Summary of the Invention

[0004] Embodiments of this disclosure provide a solution for point cloud encoding and decoding.

[0005] In a first aspect, a method for point cloud encoding and decoding is proposed. The method includes: a conversion between a current frame of a point cloud sequence and a bitstream of the point cloud sequence; determining a prediction mode for a region of the current frame, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; determining a prediction for the region based on the prediction mode; and performing a conversion based on the prediction.

[0006] In a second aspect, another method for point cloud encoding and decoding is proposed. This method includes: determining whether Region Adaptive Layered Transform (RAHT) attribute encoding and decoding is enabled for the current frame of a point cloud sequence and a bitstream of the point cloud sequence; and performing the transformation based on the RAHT attribute encoding and decoding, wherein if the RAHT attribute encoding and decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream, and wherein if the RAHT attribute encoding and decoding is disabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is not included in the bitstream.

[0007] In a third aspect, an apparatus for processing point cloud sequences is proposed. The apparatus for processing point cloud sequences includes a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to the first or second aspect of this disclosure.

[0008] In a fourth aspect, a non-transitory computer-readable storage medium is proposed. This non-transitory computer-readable storage medium stores instructions that cause a processor to execute a method according to the first or second aspect of this disclosure.

[0009] In a fifth aspect, a non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of a point cloud sequence generated by a method performed by a point cloud processing device. The method includes: determining a prediction mode for a region of a current frame of the point cloud sequence, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; determining a prediction of the region based on the prediction mode; and generating a bitstream based on the prediction.

[0010] In a sixth aspect, a method for storing a bitstream of a point cloud sequence is proposed. The method includes: determining a prediction mode for a region of the current frame of the point cloud sequence, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; determining a prediction of the region based on the prediction mode; generating a bitstream based on the prediction; and storing the bitstream in a non-transitory computer-readable recording medium.

[0011] In a seventh aspect, another non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of a point cloud sequence generated by a method performed by a point cloud processing apparatus. The method includes: determining whether Region Adaptive Layered Transform (RAHT) attribute encoding / decoding is enabled for the current frame of the point cloud sequence; and generating a bitstream based on the RAHT attribute encoding / decoding, wherein if the RAHT attribute encoding / decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream, and wherein if the RAHT attribute encoding / decoding is disabled for the point cloud sequence, the bitstream does not include at least one parameter of at least one quantization matrix for the RAHT coefficients.

[0012] In the eighth aspect, another method for storing a bitstream of a point cloud sequence is proposed. The method includes: determining whether Region Adaptive Layered Transform (RAHT) attribute encoding / decoding is enabled for the current frame of the point cloud sequence; generating a bitstream based on the RAHT attribute encoding / decoding; and storing the bitstream in a non-transitory computer-readable recording medium, wherein if RAHT attribute encoding / decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is indicated in the bitstream, and wherein if RAHT attribute encoding / decoding is disabled for the point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is not included in the bitstream.

[0013] This synopsis aims to present, in a simplified form, the selected concepts further described below in the detailed embodiments. This synopsis is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0014] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0015] Figure 1 A block diagram of an example point cloud encoding / decoding system according to some embodiments of the present disclosure is shown; Figure 2 A block diagram illustrating an example of a GPCC encoder according to some embodiments of the present disclosure is shown; Figure 3 A block diagram illustrating an example of a GPCC decoder according to some embodiments of the present disclosure is shown; Figure 4 The parent node for each child node of the transformation unit node is shown; Figure 5 An example flowchart illustrating improved attribute prediction when two reference frames are present, according to some embodiments of the present disclosure; Figure 6 A flowchart of a method for point cloud encoding and decoding according to some embodiments of the present disclosure is shown; Figure 7 A flowchart of another method for point cloud encoding and decoding according to some embodiments of the present disclosure is shown; and Figure 8 A block diagram of a computing device in which various embodiments of the present disclosure may be implemented is shown.

[0016] In all the accompanying drawings, the same or similar reference numerals usually indicate the same or similar elements. Detailed Implementation

[0017] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement this disclosure, and do not imply any limitation on the scope of this disclosure. In addition to the methods described below, the disclosure described herein can be implemented in various other ways.

[0018] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0019] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment is required to include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, it is claimed that, whether explicitly described or not, such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0020] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” “having,” “containing,” and / or “comprising” as used herein indicate the presence of the said features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0022] Example Environment Figure 1This is a block diagram illustrating an example point cloud encoding / decoding system 100 from which the techniques of this disclosure can be utilized. As shown, the point cloud encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a point cloud encoding device, and the destination device 120 may also be referred to as a point cloud decoding device. In operation, the source device 110 may be configured to generate encoded point cloud data, and the destination device 120 may be configured to decode the encoded point cloud data generated by the source device 110. The techniques of this disclosure are generally directed to encoding and / or decoding point cloud data, i.e., supporting point cloud compression. Encoding and decoding can be effective in compressing and / or decompressing point cloud data.

[0023] Source device 100 and destination device 120 may include any of a variety of devices, including desktop computers, laptops, tablets, set-top boxes, handsets (such as smartphones and mobile phones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, vehicles (e.g., land or sea vehicles, spacecraft, aircraft, etc.), robots, LiDAR devices, satellites, extended reality devices, etc. In some cases, source device 100 and destination device 120 may be equipped for wireless communication.

[0024] Source device 100 may include a data source 112, a memory 114, a GPCC encoder 116, and an input / output (I / O) interface 118. Destination device 120 may include an input / output (I / O) interface 128, a GPCC decoder 126, a memory 124, and a data consumer 122. According to this disclosure, the GPCC encoder 116 of source device 100 and the GPCC decoder 126 of destination device 120 may be configured to apply the point cloud encoding / decoding techniques of this disclosure. Therefore, source device 100 represents an example of an encoding device, and destination device 120 represents an example of a decoding device. In other examples, source device 100 and destination device 120 may include other components or arrangements. For example, source device 100 may receive data (e.g., point cloud data) from an internal or external source. Similarly, destination device 120 may interface with an external data consumer rather than including the data consumer in the same device.

[0025] Generally, data source 112 represents a source of point cloud data (i.e., raw, unencoded point cloud data) and can provide a continuous series of "frames" of point cloud data to GPCC encoder 116, which encodes the point cloud data for each frame. In some examples, data source 112 generates point cloud data. The data source 112 of source device 100 may include point cloud acquisition devices, such as any of various cameras or sensors, such as one or more cameras, an archive containing previously acquired point cloud data, a 3D scanner or light detection and ranging (LIDAR) device, and / or a data feed interface that receives point cloud data from a data content provider. Thus, in some examples, data source 112 may generate point cloud data based on signals from a LIDAR device. Alternatively or additionally, point cloud data may be generated by a computer from scanners, cameras, sensors, or other data. For example, data source 112 may generate point cloud data, or produce a combination of real-time point cloud data, archived point cloud data, and computer-generated point cloud data. In each case, the GPCC encoder 116 encodes the acquired, pre-acquired, or computer-generated point cloud data. The GPCC encoder 116 can rearrange the frames of the point cloud data from the receiving order (sometimes referred to as the "display order") to an encoding / decoding order for encoding and decoding. The GPCC encoder 116 can generate one or more bitstreams comprising the encoded point cloud data. The source device 100 can then output the encoded point cloud data via I / O interface 118 for reception and / or retrieval by, for example, the I / O interface 128 of the destination device 120. The encoded point cloud data can be directly transmitted to the destination device 120 via I / O interface 118 through network 130A. The encoded point cloud data can also be stored on storage medium / server 130B for access by the destination device 120.

[0026] The memory 114 of the source device 100 and the memory 124 of the destination device 120 may represent general-purpose memory. In some examples, memory 114 and memory 124 may store raw point cloud data, such as raw point cloud data from data source 112 and raw, decoded point cloud data from GPCC decoder 126. Additionally or alternatively, memory 114 and memory 124 may store software instructions executable by, for example, GPCC encoder 116 and GPCC decoder 126. Although memory 114 and memory 124 are shown separately from GPCC encoder 116 and GPCC decoder 126 in this example, it should be understood that GPCC encoder 116 and GPCC decoder 126 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 114 and memory 124 may store encoded point cloud data, such as encoded point cloud data output from GPCC encoder 116 and input to GPCC decoder 126. In some examples, portions of memory 114 and memory 124 may be allocated as one or more caches, for example, to store raw point cloud data, decoded and / or encoded point cloud data. For example, memory 114 and memory 124 may store point cloud data.

[0027] I / O interfaces 118 and 128 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where I / O interfaces 118 and 128 include wireless components, they may be configured to transmit data, such as encoded point cloud data, according to cellular communication standards such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc. In some examples where I / O interface 118 includes a wireless transmitter, they may be configured to transmit data, such as encoded point cloud data, according to other wireless standards such as the IEEE 802.11 specification. In some examples, source device 100 and / or destination device 120 may include corresponding system-on-chip (SoC) devices. For example, source device 100 may include a SoC device for performing functions belonging to GPCC encoder 116 and / or I / O interface 118, and destination device 120 may include a SoC device for performing functions belonging to GPCC decoder 126 and / or I / O interface 128.

[0028] The techniques disclosed herein can be applied to encoding and decoding to support any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices (e.g., local or remote servers), geographic mapping, or other applications.

[0029] The I / O interface 128 of the destination device 120 receives an encoded bitstream from the source device 110. The encoded bitstream may include signaling information defined by the GPCC encoder 116, which is also used by the GPCC decoder 126, such as syntax elements having values ​​representing the point cloud. The data consumer 122 uses the decoded data. For example, the data consumer 122 may use the decoded point cloud data to determine the location of physical objects. In some examples, the data consumer 122 may include a display for presenting images based on the point cloud data.

[0030] The GPCC encoder 116 and GPCC decoder 126 can each be implemented as any of a variety of suitable encoder circuitry and / or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the GPCC encoder 116 and GPCC decoder 126 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the GPCC encoder 116 and / or GPCC decoder 126 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0031] The GPCC encoder 116 and GPCC decoder 126 can operate according to encoding / decoding standards such as the Video Point Cloud Compression (VPCC) standard or the Geometric Point Cloud Compression (GPCC) standard. Generally, this disclosure may refer to the encoding and decoding of frames (e.g., encoding and decoding) to include the process of encoding or decoding data. Encoded bitstreams typically include a series of values ​​for syntax elements representing encoding / decoding decisions (e.g., encoding / decoding modes).

[0032] A point cloud can contain a set of points in 3D space and can have attributes associated with those points. Attributes can be color information, such as R, G, B or Y, Cb, Cr, or reflectivity information, or other attributes. Point clouds can be acquired by various cameras or sensors, such as LiDAR sensors and 3D scanners, and can also be computer-generated. Point cloud data is used in a variety of applications, including but not limited to architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LiDAR sensors for navigation aids).

[0033] Figure 2 This is a block diagram illustrating an example of a GPCC encoder 200 according to some embodiments of the present disclosure. The GPCC encoder 200 may be... Figure 1 An example of a GPCC encoder 116 in system 100 is shown. Figure 3 This is a block diagram illustrating an example of a GPCC decoder 300 according to some embodiments of the present disclosure. The GPCC decoder 300 may be... Figure 1 An example of the GPCC decoder 126 in the system 100 shown.

[0034] In both the GPCC encoder 200 and GPCC decoder 300, point cloud locations are encoded and decoded first. Attribute encoding and decoding depend on the decoded geometry. Figure 2 and Figure 3 In this configuration, Region Adaptive Hierarchical Transformation (RAHT) unit 218, Surface Approximation Analysis unit 212, RAHT unit 314, and Surface Approximation Synthesis unit 310 are options typically used for Category 1 data. Level of Detail (LOD) Generation unit 220, Lifting unit 222, LOD Generation unit 316, and Inverse Lifting unit 318 are options typically used for Category 3 data. All other units are common between Category 1 and Category 3.

[0035] For Category 3 data, the compressed geometry is typically represented as an octree from the root down to the leaf level of each voxel. For Category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root down to the leaf level of blocks larger than voxels) plus a model for approximating the surface within each leaf node of the pruned octree. In this way, both Category 1 and Category 3 data share the octree encoding / decoding mechanism, while Category 1 data can additionally utilize the surface model to approximate the voxels within each leaf node. The surface model used is a triangulation of each block comprising 1 to 10 triangles, producing a triangle soup. Therefore, the Category 1 geometry codec is called a triangle soup geometry codec, while the Category 3 geometry codec is called an octree geometry codec.

[0036] exist Figure 2 In the example, the GPCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometric reconstruction unit 216, a RAHT unit 218, a LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, and an arithmetic coding unit 226.

[0037] like Figure 2 As shown in the example, the GPCC encoder 200 can receive a set of locations and a set of attributes. Locations can include the coordinates of points in the point cloud. Attributes can include information about the points in the point cloud, such as the colors associated with those points.

[0038] The coordinate transformation unit 202 can apply transformations to the coordinates of a point to transform the coordinates from the initial domain to the transformation domain. The transformed coordinates can be referred to as transformed coordinates. The color transformation unit 204 can apply transformations to convert the color information of an attribute to different domains. For example, the color transformation unit 204 can convert color information from the RGB color space to the YCbCr color space.

[0039] In addition, Figure 2 In the example, voxelization unit 206 can voxelize the transformed coordinates. Voxelization of the transformed coordinates can include quantization and removal of some points in the point cloud. In other words, multiple points in the point cloud can be grouped into a single "voxel," which can then be considered a point in some respects. Furthermore, octree analysis unit 210 can generate an octree based on the voxelized transformed coordinates. Additionally, in Figure 2 In the example, the surface approximation analysis unit 212 can analyze points to potentially determine a surface representation of the set of points. The arithmetic coding unit 214 can perform arithmetic coding on syntax elements representing information about an octree and / or information about the surface determined by the surface approximation analysis unit 212. The GPCC encoder 200 can output these syntax elements in a geometric bitstream.

[0040] The geometric reconstruction unit 216 can reconstruct the transformed coordinates of points in the point cloud based on an octree, data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometric reconstruction unit 216 may differ from the original number of points in the point cloud. The resulting points may be referred to as reconstructed points. The attribute transfer unit 208 can transfer attributes of the original points in the point cloud to the reconstructed points in the point cloud data.

[0041] Furthermore, RAHT unit 218 can apply RAHT encoding to the attributes of the reconstructed points. Alternatively or additionally, LOD generation unit 220 and lifting unit 222 can apply LOD processing and lifting to the attributes of the reconstructed points, respectively. RAHT unit 218 and lifting unit 222 can generate coefficients based on the attributes. Coefficient quantization unit 224 can quantize the coefficients generated by RAHT unit 218 or lifting unit 222. Arithmetic encoding unit 226 can apply arithmetic encoding to the syntax elements representing the quantized coefficients. GPCC encoder 200 can output these syntax elements in the attribute bitstream.

[0042] exist Figure 3 In the example, the GPCC decoder 300 may include a geometric arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometric reconstruction unit 312, a RAHT unit 314, an LOD generation unit 316, an inverse lifting unit 318, a coordinate inverse transformation unit 320, and a color inverse transformation unit 322.

[0043] The GPCC decoder 300 can obtain a geometric bitstream and an attribute bitstream. The geometric arithmetic decoding unit 302 of the decoder 300 can apply arithmetic decoding (e.g., CABAC or other types of arithmetic decoding) to the syntax elements in the geometric bitstream. Similarly, the attribute arithmetic decoding unit 304 can apply arithmetic decoding to the syntax elements in the attribute bitstream.

[0044] Octree synthesis unit 306 can synthesize octrees based on syntax elements parsed from the geometric bitstream. In the case of using surface approximation in the geometric bitstream, surface approximation synthesis unit 310 can determine the surface model based on syntax elements parsed from the geometric bitstream and based on the octree.

[0045] Furthermore, the geometric reconstruction unit 312 can perform reconstruction to determine the coordinates of points in the point cloud. The inverse coordinate transformation unit 320 can apply an inverse transformation to the reconstructed coordinates to transform the reconstructed coordinates (positions) of points in the point cloud from the transformation domain back to the initial domain.

[0046] Additionally, in Figure 3 In the example, the dequantization unit 308 can dequantize the attribute value. The attribute value can be based on syntax elements obtained from the attribute bitstream (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0047] Depending on how the attribute values ​​are encoded, RAHT unit 314 can perform RAHT decoding to determine the color value for a point in the point cloud based on the dequantized attribute values. Alternatively, LOD generation unit 316 and inverse boosting unit 318 can use level-of-detail (LOD) based techniques to determine the color value for a point in the point cloud.

[0048] In addition, Figure 3 In the example, the color inverse transformation unit 322 can apply an inverse color transformation to color values. The inverse color transformation can be the inverse of the color transformation applied by the color transformation unit 204 of the encoder 200. For example, the color transformation unit 204 can transform color information from the RGB color space to the YCbCr color space. Correspondingly, the color inverse transformation unit 322 can transform color information from the YCbCr color space to the RGB color space.

[0049] Figure 2 and Figure 3 Various units are shown to aid in understanding the operations performed by encoder 200 and decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit is a circuit that provides a specific function and is preset with respect to the operations that can be performed. A programmable circuit is a circuit that can be programmed to perform various tasks and provides flexible functionality in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions in the software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more of these units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more of these units can be integrated circuits.

[0050] Some exemplary embodiments of this disclosure will be described in detail below. It should be understood that section headings are used in this document for ease of understanding and not to limit the embodiments disclosed in a section to that section. Furthermore, while some embodiments are described with reference to GPCC or other specific point cloud codecs, the disclosed techniques are also applicable to other point cloud codec techniques. Additionally, although some embodiments describe point cloud codec steps in detail, it should be understood that the corresponding decoding steps for decoding will be implemented by the decoder.

[0051] 1. Brief Overview This disclosure relates to point cloud encoding and decoding techniques. Specifically, it relates to inter-frame prediction of point cloud attributes in region adaptive hierarchical transformation. This idea can be applied alone or in various combinations to any standard or non-standard point cloud codec, such as geometry-based point cloud compression (G-PCC) under development.

[0052] 2. Abbreviations G-PCC is a geometry-based point cloud compression technology. MPEG Moving Picture Experts Group 3DG3D Graphics Encoding and Decoding Team CFP Proposal Solicitation V-PCC is a video-based point cloud compression technology. RAHT region adaptive hierarchical transformation 3. Introduction MPEG, short for Moving Picture Experts Group, is one of the leading standardization organizations for multimedia processing. In 2017, the MPEG 3D Graphics Codec Group (3DG) released a Call for Requests (CFP) document to begin development of a point cloud encoding and decoding standard. The final standard will include two types of solutions. Video-based point cloud compression (V-PCC) is suitable for point sets with a relatively uniform point distribution. Geometry-based point cloud compression (G-PCC) is suitable for sparser distributions. Both V-PCC and G-PCC support encoding and decoding of single point clouds and sequences of point clouds.

[0053] A point cloud can contain geometric information and attribute information. Geometric information describes the geometric location of the data points. Attribute information records details of the data points, such as texture, normal vectors, and reflections.

[0054] 3.1 Regional Adaptive Hierarchical Transformation In G-PCC, one of the important point cloud attribute encoding and decoding tools is RAHT. It's a transformation that uses attributes associated with nodes in lower levels of an octree to predict the attributes of nodes in the next level. It assumes that the location of the points is given at both the encoder and decoder. RAHT scans the octree backward from the leaf nodes to the root node, recombining nodes into larger nodes at each step until the root node is reached. At each level of the octree, nodes are processed in Morton order. Instead of combining all eight nodes at once during each decomposition, RAHT performs this in three steps along each dimension (e.g., along z, then y, then x). If the octree has... If there are 3L levels, then RAHT needs 3L levels to traverse the tree backward.

[0055] For integers x, y, z, define the hierarchy. The node at is . Through the and This is obtained through combination, with combinations along the first dimension as an example. RAHT only processes occupied nodes. If one node in a pair is not occupied, the other node is promoted to the next level and not processed; that is, if the latter is the occupied node in the pair, then... Repeat this combination process until the root is reached. Note that the combination process generates lower-level nodes, which are the result of combining different numbers of voxels in this way. The nodes are combined to generate the nodes. The number of nodes is the weight of that node. .

[0056] In two nodes (e.g.) and In each combination, utilize their respective weights. and RAHT applies the following transformations:

[0057] in and and

[0058] Note that the transformation matrix is ​​constantly changing to adapt to the weights, that is, to adapt to each... The actual number of leaf nodes represented. It is used to combine and form further nodes at lower levels. These are the actual high-pass coefficients generated by the transformations to be encoded and transmitted. Furthermore, the weights accumulate over the layers above. In the example above,

[0059] In the final stage, the roots of the tree, the remaining two voxels and Transformed into the final two coefficients:

[0060] in .

[0061] 3.3 Upsampled Transform Domain Prediction in RAHT Transform domain prediction is introduced to improve the encoding and decoding efficiency of RAHT. It consists of two parts.

[0062] First, the RAHT tree traversal is changed from the previous ascending method to a descending one; that is, a tree is constructed by summing attributes and weights. Then, RAHT is performed from the root to the leaf for both the encoder and decoder. The transformation is also performed in octree node transformation units with 2×2×2 child nodes. Within a node, the encoder transformation order is from leaf to root.

[0063] Secondly, for each child node of the transform unit, the corresponding prediction child node is generated by upsampling the previous transform level. In practice, only child nodes containing at least one point will generate a corresponding prediction child node. The transform unit containing 2×2×2 prediction child nodes is transformed and subtracted from the transformed attributes on the encoder side.

[0064] Each child node of a transformation unit node is predicted by 7 parent level nodes, including 3 collinear parent level nearest neighbors, 3 coplanar parent level nearest neighbors, and 1 parent node. Coplanar nearest neighbors and collinear nearest neighbors are the nearest neighbors that share a face and an edge with the current transformation unit node, respectively. Figure 4 The diagram shows the seven parent-level nodes for each child node of the transformation unit node.

[0065] Attributes of each child node The distance between it and its parent level node is predicted as follows.

[0066]

[0067] in It is an attribute of one of its parent level nodes, and It depends on the distance weight. In G-PCC, .

[0068] For the AC coefficients, the predicted residuals will be transmitted via signal transmission.

[0069] For DC coefficients, the coefficients are inherited from previous levels, which means that DC coefficients are transmitted through the signal without prediction.

[0070] 3.4 Attribute Inter-Frame Prediction in RAHT We propose applying inter-frame prediction to the DC and AC coefficients in RAHT.

[0071] For layers with inter-frame prediction enabled, an octree decomposition is performed on the two frames. The decomposition is derived based on the geometric information of each frame.

[0072] For each node in the current octree decomposition, it can be matched with a node in the reference decomposition. The Morton value of the matched node should be the same as the Morton value of the current node. For the AC and DC coefficients of these nodes, reference coefficients are generated in the same manner as the current coefficients. The reference coefficients are used to predict the current coefficients.

[0073] For example, for the AC coefficients, the prediction residuals are transmitted via signal as follows: , .

[0074] if If the value is 0, then apply. As a prediction in the original transform domain.

[0075] For each layer, there is a flag indicating whether inter-frame prediction is enabled. This flag is derived based on a rate-distortion optimization method.

[0076] 3.5 AC coefficient quantization offset A method is proposed to transmit a 3D quantization matrix via the signal, allowing the QP offset to be applied to each AC coefficient of the transformed 2×2×2 residual. Note that the DC coefficients are inherited, and therefore no residual DC component is transmitted to the decoder via the signal.

[0077] 4. Question Existing designs for inter-frame prediction of point cloud attributes in RAHT have the following problems: 1. In the current design, the flag used to indicate whether inter-frame prediction is enabled is determined for each layer. However, this layer-level determination may not be optimal, considering that the best choice for each node can vary.

[0078] 2. In the current design, reference information comes from only one reference frame, and there can be at most one reference node for each node. However, this reference information may be insufficient to provide efficient prediction.

[0079] 3. In the current design, the predicted values ​​are derived from intra-frame prediction or inter-frame prediction. However, this prediction method may not be optimal.

[0080] 4. In the current design, the indicator used to indicate whether the AC coefficient quantization matrix is ​​transmitted via signal is always transmitted via signal. However, the AC coefficient quantization matrix is ​​only used in the RAHT encoding / decoding method.

[0081] 5. Detailed Solution To address the above-mentioned problems and some other issues not mentioned, the methods outlined below are disclosed. The embodiments should be considered as examples for explaining general concepts and should not be interpreted in a narrow sense. Furthermore, these embodiments can be applied individually or in any combination.

[0082] 1) A prediction method for each node is proposed and the prediction method for each node is transmitted via signal transmission.

[0083] a. In one example, the prediction method could be no prediction.

[0084] b. In one example, the prediction method could be intra-frame prediction.

[0085] c. In one example, the prediction method could be inter-frame prediction.

[0086] d. In one example, the prediction method could be to derive the predicted value using intra-frame prediction and inter-frame prediction.

[0087] i. In one example, the predicted value could be a weighted average of inter-frame and intra-frame predictions.

[0088] e. In one example, the prediction method can be determined based on some eligibility criteria.

[0089] i. In one example, eligibility criteria can be based on the layer depth of a node.

[0090] ii. In one example, the eligibility criteria can be based on the geometric location of the node.

[0091] iii. In one example, eligibility criteria can be based on the node's attribute information.

[0092] iv. In one example, eligibility criteria can be based on a node's nearest neighbor information.

[0093] f. In one example, the prediction method can be determined based on a rate-distortion optimization method.

[0094] i. In one example, bit rate and / or distortion can be estimated.

[0095] ii. Alternatively, bit rate and / or distortion can be calculated based on the reconstructed values.

[0096] g. In one example, the above determination methods can be combined to determine the prediction method.

[0097] h. In one example, for each node, there may be an indicator to indicate which prediction method is determined to be applied to a node.

[0098] i. In one example, the indicator can be transmitted to the decoder via a signal.

[0099] 1. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0100] 2. In one example, the indicator can be encoded and decoded in a predictive manner.

[0101] 2) A prediction method for each region / level is proposed and the prediction method for each region / level is transmitted via signal transmission.

[0102] a. In one example, the prediction method could be no prediction.

[0103] b. In one example, the prediction method could be intra-frame prediction.

[0104] c. In one example, the prediction method could be inter-frame prediction.

[0105] d. In one example, the prediction method could be to derive the predicted value using intra-frame prediction and inter-frame prediction.

[0106] i. In one example, the predicted value could be a weighted average of inter-frame and intra-frame predictions.

[0107] e. In one example, the prediction method can be a combination of the above methods.

[0108] i. In one example, if the predicted value of method A is not zero, then the predicted value can be the predicted value of method A; otherwise, the predicted value can be the predicted value of method B.

[0109] f. In one example, the prediction method can be applied to all nodes in the region / hierarchy.

[0110] g. In one example, the prediction method can be determined based on some eligibility criteria.

[0111] i. In one example, eligibility criteria can be based on the layer depth of the region / level.

[0112] 1. In one example, there may be at least one indicator to indicate the layer depth(s) or the area(s) allowed in the qualifying conditions.

[0113] a. In one example, the indicator can be transmitted to the decoder via a signal.

[0114] i. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0115] ii. In one example, the indicator can be encoded and decoded in a predictive manner.

[0116] ii. In one example, eligibility criteria can be based on the geometric location of a region / hierarchy.

[0117] iii. In one example, eligibility criteria can be based on region / hierarchy attribute information.

[0118] iv. In one example, eligibility criteria can be based on the nearest neighbor information of nodes in a region / hierarchy.

[0119] h. In one example, the prediction method can be determined based on a rate-distortion optimization method.

[0120] i. In one example, bit rate and / or distortion can be estimated.

[0121] ii. Alternatively, bit rate and / or distortion can be calculated based on the reconstructed values.

[0122] i. In one example, the above determination methods can be combined to determine the prediction method.

[0123] j. In one example, for each region / level, there can be an indicator to indicate which prediction method is determined to be applied to the nodes in that region / level.

[0124] i. In one example, the indicator can be transmitted to the decoder via a signal.

[0125] 1. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0126] 2. In one example, the indicator can be encoded and decoded in a predictive manner.

[0127] 3) A method for determining the parameters of the prediction method by means of signal transmission is proposed.

[0128] a. In one example, there may be parameters indicating the eligibility criteria for a prediction method.

[0129] b. In one example, there may be some parameters to derive the eligibility criteria for a prediction method.

[0130] c. In one example, there may be parameters to indicate the area / level where the eligibility criteria need to be checked.

[0131] i. In one example, for a prediction method, only certain layers may need to check the eligibility criteria; other layers may not need to apply the prediction method.

[0132] ii. Alternatively, for a prediction method, only certain specific layers may require checking the qualification criteria; other layers may be eligible for the prediction method.

[0133] iii. In one example, there may be some parameters to derive a specific layer.

[0134] d. In one example, the above parameters can be transmitted to the decoder via a signal.

[0135] i. In one example, the parameters can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0136] ii. In one example, the parameters can be encoded and decoded in a predictive manner.

[0137] 4) A method for generating prediction results based on reference information from multiple reference frames is proposed.

[0138] a. In one example, for each node, there can be multiple reference nodes from multiple reference frames.

[0139] b. In one example, there may be one inter-frame prediction value for each reference node.

[0140] c. In one example, for each reference node, there can be an indicator to indicate the reference node.

[0141] d. In one example, for the current node, the prediction can be derived from the intra-frame prediction and multiple inter-frame predictions.

[0142] e. In one example, the predicted value can be selected from the predicted value candidates.

[0143] i. In one example, a prediction candidate can be an intra-frame prediction.

[0144] ii. In one example, if there is a reference node, a prediction candidate can be an inter-frame prediction of the reference node.

[0145] iii. In one example, if there are multiple reference nodes, a prediction candidate can be a weighted average of intra-frame predictions and multiple inter-frame predictions.

[0146] iv. In one example, a candidate prediction can be a weighted average of an intra-frame prediction and an inter-frame prediction.

[0147] v. In one example, if there are multiple reference nodes, a candidate prediction can be a weighted average of multiple inter-frame predictions.

[0148] f. In one example, the choice can be determined based on the rate-distortion optimization method.

[0149] i. In one example, bit rate and / or distortion can be estimated.

[0150] ii. Alternatively, bit rate and / or distortion can be calculated based on the reconstructed values.

[0151] g. In one example, there could be an indicator to show which predicted value candidate was selected.

[0152] i. In one example, the indicator can be transmitted to the decoder via a signal.

[0153] 1. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0154] 2. In one example, the indicator can be encoded and decoded in a predictive manner.

[0155] 5) A method was proposed to fix / derive / transmit the weight values ​​used in the above weighted average calculation.

[0156] a. In one example, the weights can be fixed at the decoder.

[0157] i. In one example, the weights can vary for each layer.

[0158] ii. In one example, the weights can vary for each node.

[0159] iii. In one example, the weights can be consistent across all nodes.

[0160] b. In one example, the weights can be derived at the decoder.

[0161] i. In one example, the weights can be derived for each layer.

[0162] 1. In one example, weights can be derived based on the candidate selection results of the predicted values ​​of nodes in (multiple) previous layers.

[0163] 2. In one example, the weights can be derived based on the depth level.

[0164] ii. In one example, the weights can be derived for each node.

[0165] 1. In one example, the weights can be derived based on the predicted candidate selection results of some previously encoded nodes (such as encoded neighbor nodes or parent nodes).

[0166] c. In one example, weights can be transmitted to the decoder via signals.

[0167] i. In one example, weights can be transmitted to the decoder via signals for each layer.

[0168] 1. In one example, the weights can be encoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0169] 2. In one example, the weights can be encoded and decoded in a predictive manner.

[0170] d. In one example, weights can be selected from a predefined list of weight candidates.

[0171] i. In one example, weights can be selected using a rate-distortion optimization / rate-distortion optimization method.

[0172] ii. In one example, there may be at least one indicator to indicate the selection result.

[0173] 1. In one example, the indicator can be transmitted to the decoder via a signal.

[0174] a. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0175] b. In one example, the indicator can be encoded and decoded in a predictive manner.

[0176] iii. In one example, selection can be performed on each layer.

[0177] iv. In one example, selection can be performed on a per-node basis.

[0178] 6) A weighted average of intra-frame prediction and inter-frame prediction is proposed to determine whether signal transmission uses this method.

[0179] a. In one example, there could be an indicator to show whether weighted averaging of intra-frame and inter-frame predictions can be enabled for the entire sequence.

[0180] b. In one example, there could be an indicator to indicate whether a weighted average of intra-frame and inter-frame predictions is used for a PC sample.

[0181] c. In one example, there could be an indicator to indicate whether a weighted average of intra-frame prediction and inter-frame prediction is used for a layer / region.

[0182] d. In one example, all or some of the above indicators may be transmitted to the decoder via signals.

[0183] i. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0184] ii. In one example, the indicator can be encoded and decoded in a predictive manner.

[0185] 7) A method is proposed to transmit parameters for the RAHT coefficient quantization matrix via signal transmission when RAHT attribute encoding / decoding is enabled.

[0186] a. In one example, the RAHT coefficient can be the AC coefficient.

[0187] b. In one example, the RAHT coefficient can be the DC coefficient.

[0188] c. In one example, the RAHT coefficient can be either the AC coefficient or the DC coefficient.

[0189] d. In one example, there may be a quantization matrix for the AC coefficients.

[0190] e. In one example, there could be a quantization matrix for the DC coefficients.

[0191] f. In one example, there may be at least one indicator to indicate whether the RAHT coefficient quantization matrix is ​​transmitted via signaling.

[0192] i. In one example, the indicator can only be signaled to the decoder if RAHT attribute encoding / decoding is enabled.

[0193] ii. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0194] iii. In one example, the indicator can be encoded and decoded in a predictive manner.

[0195] 8) Whether and / or how the methods disclosed above can be applied to transmit signals from the encoder to the decoder in bitstream / frame / slice / segment / octree / etc.

[0196] 9) Whether and / or how to apply the methods disclosed above may depend on the encoded / decoded information, such as dimensions, color format, color components, and slice / image type.

[0197] 6. Example Figure 5 An example of an encoding / decoding stream 500 for improved inter-frame prediction of coefficients is depicted when two reference frames are present. As shown, multiple reference frames (such as reference frame 1 and reference frame 2) can be used to encode / decode the current frame.

[0198] Embodiments of this disclosure relate to encoding and decoding for point cloud encoding and decoding. As used herein, the term "point cloud sequence" may refer to a sequence of one or more point clouds. The term "frame" may refer to a point cloud in a point cloud sequence. The term "point cloud" may refer to a frame in a point cloud sequence. The term "node" represents a spatial segmentation of the current frame.

[0199] Figure 6 A flowchart of a method 600 for point cloud encoding / decoding according to some embodiments of the present disclosure is shown. Method 600 can be implemented during the conversion between the current frame of a point cloud sequence and a bitstream of the point cloud sequence.

[0200] like Figure 6 As shown, method 600 begins at box 610, wherein a prediction mode for the region of the current frame is determined by a conversion between the current frame of the point cloud sequence and the bitstream of the point cloud sequence. The prediction mode includes at least a first mode based on intra-frame prediction and inter-frame prediction. As used herein, the prediction mode may also be referred to as a prediction method or prediction tool.

[0201] At box 620, the prediction of the region is determined based on the prediction pattern.

[0202] At box 630, a transformation is performed based on the prediction. In some embodiments, the transformation may include encoding the current frame into a bitstream. Alternatively or additionally, the transformation may include decoding the current frame from the bitstream.

[0203] Method 600 enables the determination of prediction patterns for regions such as at least one node. Point cloud encoding and decoding can therefore be enhanced.

[0204] In some embodiments, in a first mode, the prediction of a region includes a weighted average of intra-frame prediction and inter-frame prediction of the region.

[0205] In some embodiments, the prediction mode is indicated in the bitstream.

[0206] In some embodiments, the prediction mode further includes at least one of the following: no prediction mode, intra-frame prediction mode, or inter-frame prediction mode.

[0207] In some embodiments, the region includes at least one of the following: a node of the current frame or a hierarchy of the current frame.

[0208] In some embodiments, the region includes nodes of the current frame, and the prediction mode is determined based on at least one qualification condition, wherein the at least one qualification condition is based on at least one of the following: the layer depth of the node, the geometric position of the node, the attribute information of the node, or the neighbor information of the node.

[0209] In some embodiments, the region includes nodes of the current frame, and the prediction mode is determined based on rate-distortion optimization.

[0210] In some embodiments, at least one of bit rate or distortion for rate-distortion optimization is estimated.

[0211] In some embodiments, at least one of the bit rate or distortion for rate-distortion optimization is determined based on at least one reconstructed value.

[0212] In some embodiments, the region includes nodes of the current frame, and the indication associated with the node indicates the prediction pattern to be applied to the node.

[0213] In some embodiments, the indication is included in the bitstream.

[0214] In some embodiments, the instruction is to use one of the following for encoding and decoding: fixed-length encoding and decoding, unary encoding and decoding, or rounding unary encoding and decoding, or to use predictive encoding and decoding.

[0215] In some embodiments, the prediction mode includes a combination of a first prediction mode and a second prediction mode, and wherein if a first prediction value of a region based on the first prediction mode is not zero, the prediction value of the region is determined as the first prediction value, or wherein if a first prediction value of a region based on the first prediction mode is zero, the second prediction value of the region based on the second prediction mode is determined as the prediction value of the region.

[0216] In some embodiments, the region comprises multiple nodes, and the determined prediction pattern is applied to the multiple nodes.

[0217] In some embodiments, the prediction pattern is determined based on at least one qualification criterion, wherein the at least one qualification criterion is based on at least one of the following: the layer depth of the region, the geometric location of the region, the attribute information of the region, or the nearest neighbor information of the nodes in the region.

[0218] In some embodiments, at least one indicator indicates at least one permitted layer depth or at least one permitted region in at least one qualifying condition.

[0219] In some embodiments, at least one indicator is indicated to the decoder.

[0220] In some embodiments, at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0221] In some embodiments, at least one indicator is encoded and decoded in a predictive manner.

[0222] In some embodiments, the prediction pattern is determined based on rate-distortion optimization.

[0223] In some embodiments, at least one of rate-distortion optimized bit rate or distortion is estimated.

[0224] In some embodiments, at least one of the bit rate or distortion in rate-distortion optimization is calculated based on at least one reconstructed value.

[0225] In some embodiments, the indicator indicates that the prediction pattern is determined to be applied to nodes in the region.

[0226] In some embodiments, the indicator is given to the decoder.

[0227] In some embodiments, the indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0228] In some embodiments, the indicator is encoded and decoded predictively.

[0229] In some embodiments, at least one parameter for determining the prediction mode is indicated in the bitstream.

[0230] In some embodiments, at least one parameter indicates the eligibility criteria for the predicted pattern.

[0231] In some embodiments, the eligibility criteria for a prediction pattern are determined based on at least one parameter.

[0232] In some embodiments, at least one parameter indicates an area that needs to be checked for at least one qualifying condition.

[0233] In some embodiments, for the first prediction mode, the first group of layers needs to check at least one qualifying condition, and the second group of layers does not apply the first prediction mode.

[0234] In some embodiments, for a first prediction mode, a first set of layers needs to check at least one qualifying condition, and a second set of layers applies the first prediction mode.

[0235] In some embodiments, at least one of the first group of layers or the second group of layers is determined based on at least one parameter.

[0236] In some embodiments, at least one parameter is indicated to the decoder.

[0237] In some embodiments, at least one parameter is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0238] In some embodiments, at least one parameter is encoded and decoded in a predictive manner.

[0239] In some embodiments, the predicted value of the current node in a region is determined based on reference information from multiple reference frames.

[0240] In some embodiments, for each node in a region, multiple reference nodes come from multiple reference frames.

[0241] In some embodiments, inter-frame prediction values ​​are determined for each reference node.

[0242] In some embodiments, for each reference node, an indicator points to the reference node.

[0243] In some embodiments, for the current node, the prediction value is determined based on the intra-frame prediction value of the current node and multiple inter-frame prediction values ​​of the current node.

[0244] In some embodiments, for the current node, the predicted value is selected from a plurality of predicted value candidates.

[0245] In some embodiments, the plurality of prediction value candidates include intra-frame prediction values.

[0246] In some embodiments, if a reference node exists, the plurality of prediction candidates include inter-frame predictions of the reference node.

[0247] In some embodiments, if multiple reference nodes exist, the multiple prediction candidates include intra-frame predictions and a weighted average of multiple inter-frame predictions from the multiple reference nodes.

[0248] In some embodiments, the plurality of prediction candidates includes a weighted average of intra-frame predictions and inter-frame predictions.

[0249] In some embodiments, if multiple reference nodes exist, the multiple prediction candidates include a weighted average of multiple inter-frame predictions.

[0250] In some embodiments, rate-distortion optimization is selected to determine this.

[0251] In some embodiments, at least one of rate-distortion optimized bit rate or distortion is estimated.

[0252] In some embodiments, at least one of the bit rate or distortion in rate-distortion optimization is calculated based on at least one reconstructed value.

[0253] In some embodiments, the indicator indicates the selected predictive value candidate from a plurality of predictive value candidates.

[0254] In some embodiments, the indicator is given to the decoder.

[0255] In some embodiments, the indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0256] In some embodiments, the indicator is encoded and decoded predictively.

[0257] In some embodiments, multiple weights used to determine the predicted value of the current node in a region are fixed, indicated in the bitstream, or determined during the transformation.

[0258] In some embodiments, multiple weights are fixed at the decoder.

[0259] In some embodiments, multiple weights vary for each layer or for each node.

[0260] In some embodiments, multiple weights are consistent across multiple nodes.

[0261] In some embodiments, multiple weights are derived at the decoder.

[0262] In some embodiments, multiple weights are derived for each layer.

[0263] In some embodiments, multiple weights are derived based on the candidate selection results of the predicted values ​​of at least one node in the previous layer.

[0264] In some embodiments, multiple weights are derived based on depth levels.

[0265] In some embodiments, multiple weights are derived for each node.

[0266] In some embodiments, multiple weights are derived based on the candidate selection results of the predicted values ​​of at least one previously encoded node, which includes at least one of the following: an encoded neighbor node or a parent node.

[0267] In some embodiments, multiple weights are assigned to the decoder.

[0268] In some embodiments, multiple weights are assigned to the decoder for each layer.

[0269] In some embodiments, multiple weights are encoded and decoded using one of the following: fixed-length encoding and decoding, unary encoding and decoding, or rounding unary encoding and decoding.

[0270] In some embodiments, multiple weights are encoded and decoded in a predictive manner.

[0271] In some embodiments, multiple weights are selected from a predefined list of weight candidates.

[0272] In some embodiments, multiple weights are selected based on one of the following: rate-distortion optimization, rate optimization, or distortion optimization.

[0273] In some embodiments, at least one indicator indicates the selection result from a predefined list of weighted candidates.

[0274] In some embodiments, at least one indicator is indicated to the decoder.

[0275] In some embodiments, at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0276] In some embodiments, at least one indicator is encoded and decoded in a predictive manner.

[0277] In some embodiments, the action is performed either per layer or per node.

[0278] In some embodiments, a weighted average of intra-frame prediction and inter-frame prediction is indicated.

[0279] In some embodiments, the first indicator indicates whether weighted averaging of intra-frame prediction and inter-frame prediction is enabled for the entire sequence.

[0280] In some embodiments, the second indicator indicates whether a weighted average of intra-frame and inter-frame predictions is enabled for point cloud (PC) samples.

[0281] In some embodiments, the third indicator indicates whether weighted averaging of intra-frame prediction and inter-frame prediction is enabled for a hierarchy or region.

[0282] In some embodiments, at least one of a first indicator, a second indicator, or a third indicator relating to the use of a weighted average is indicated to the decoder.

[0283] In some embodiments, at least one of the first indicator, the second indicator, or the third indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0284] In some embodiments, at least one of the first indicator, the second indicator, or the third indicator is encoded and decoded in a predictive manner.

[0285] According to embodiments of this disclosure, a non-transitory computer-readable recording medium is proposed. A bitstream of a point cloud sequence is stored in the non-transitory computer-readable recording medium. The bitstream of the point cloud sequence is generated by a method performed by a point cloud sequence processing apparatus. According to the method, a prediction mode for a region of the current frame of the point cloud sequence is determined. The prediction mode includes at least a first mode based on intra-frame prediction and inter-frame prediction. A prediction of the region is determined based on the prediction mode. A bitstream is generated based on the prediction.

[0286] According to embodiments of this disclosure, a method for storing a bitstream of a point cloud sequence is proposed. In this method, a prediction mode for a region of the current frame of the point cloud sequence is determined. The prediction mode includes at least a first mode based on intra-frame prediction and inter-frame prediction. A prediction for the region is determined based on the prediction mode. A bitstream is generated based on the prediction. The bitstream is stored in a non-transitory computer-readable recording medium.

[0287] Figure 7 A flowchart of a method 700 for point cloud encoding / decoding according to some embodiments of the present disclosure is shown. Method 700 can be implemented during the conversion between the current frame of a point cloud sequence and a bitstream of the point cloud sequence. Figure 7 As shown, method 700 begins at box 710, where for the conversion between the current frame of the point cloud sequence and the bitstream of the point cloud sequence, the Region Adaptive Layer Transform (RAHT) attribute encoding and decoding is enabled for the current frame.

[0288] At box 720, a transformation is performed based on RAHT attribute encoding / decoding. If RAHT attribute encoding / decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream. If RAHT attribute encoding / decoding is disabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is not included in the bitstream. In some embodiments, the transformation may include encoding the current frame into a bitstream. Alternatively or additionally, the transformation may include decoding the current frame from the bitstream.

[0289] Method 700 enables the transmission of (multiple) parameters for the RAHT coefficient quantization matrix via signaling within the bitstream. Point cloud encoding and decoding can therefore be improved.

[0290] In some embodiments, the RAHT coefficient includes at least one of the following: the AC coefficient or the DC coefficient.

[0291] In some embodiments, at least one quantization matrix includes at least one of the following: at least one quantization matrix for AC coefficients or at least one quantization matrix for DC coefficients.

[0292] In some embodiments, at least one indicator indicates whether at least one quantization matrix for RAHT coefficients is indicated in the bitstream.

[0293] In some embodiments, if RAHT attribute encoding / decoding is enabled, at least one indicator is indicated to the decoder.

[0294] In some embodiments, at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0295] In some embodiments, at least one indicator is encoded and decoded in a predictive manner.

[0296] According to embodiments of this disclosure, a non-transitory computer-readable recording medium is proposed. A bitstream of a point cloud sequence is stored in the non-transitory computer-readable recording medium. The bitstream of the point cloud sequence is generated by a method performed by a point cloud sequence processing apparatus. According to the method, it is determined whether Region Adaptive Layer Transform (RAHT) attribute encoding / decoding is enabled for the current frame of the point cloud sequence. The bitstream is generated based on the RAHT attribute encoding / decoding. At least one parameter of at least one quantization matrix for RAHT coefficients is indicated in the bitstream. If RAHT attribute encoding / decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is indicated in the bitstream. If RAHT attribute encoding / decoding is disabled for the point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is not included in the bitstream.

[0297] According to embodiments of this disclosure, a method for storing a bitstream of a point cloud sequence is proposed. In this method, it is determined whether Region Adaptive Layered Transform (RAHT) attribute encoding / decoding is enabled for the current frame of the point cloud sequence. A bitstream is generated based on the RAHT attribute encoding / decoding. If RAHT attribute encoding / decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream. If RAHT attribute encoding / decoding is disabled for the point cloud sequence, at least one parameter of at least one quantization matrix for the RAHT coefficients is not included in the bitstream. The bitstream is stored in a non-transitory computer-readable recording medium.

[0298] It should be understood that methods 600 and / or 700 above can be used in combination or individually. Any suitable combination of these methods can be applied. The scope of this disclosure is not limited in this respect.

[0299] In some embodiments, an indicator indicating whether method 600 and / or method 700 are applied may be included in the bitstream. For example, the indicator may be included in one of the following from the encoder to the decoder: bitstream, frame, slice, slice, or octree.

[0300] Alternatively or additionally, in some embodiments, whether and / or how to apply method 600 and / or method 700 can be determined based on encoded / decoded information. For example, encoded / decoded information may include at least one of the following: dimension, color format, color components, slice type, or image type.

[0301] By using method 600 and / or method 700 alone or in combination, the encoding / decoding effectiveness and efficiency of point cloud encoding / decoding can be improved.

[0302] The embodiments of this disclosure can be described according to the following entries, and their features can be combined in any reasonable manner.

[0303] Item 1. A method for point cloud encoding and decoding, comprising: a conversion between a current frame of a point cloud sequence and a bitstream of the point cloud sequence; determining a prediction mode for a region of the current frame, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; determining a prediction of the region based on the prediction mode; and performing the conversion based on the prediction.

[0304] Item 2. The method according to Item 1, wherein in the first mode, the prediction of the region comprises a weighted average of the intra-frame prediction and the inter-frame prediction of the region.

[0305] Item 3. The method according to Item 1 or 2, wherein the prediction mode is indicated in the bitstream.

[0306] Item 4. The method according to any one of Items 1 to 3, wherein the prediction mode further comprises at least one of the following: no prediction mode, intra-frame prediction mode, or inter-frame prediction mode.

[0307] Item 5. The method according to any one of items 1 to 4, wherein the region includes at least one of the following: a node of the current frame, or a hierarchy of the current frame.

[0308] Item 6. The method according to any one of items 1 to 5, wherein the region includes nodes of the current frame, and the prediction mode is determined based on at least one qualifying condition, wherein the at least one qualifying condition is based on at least one of the following: the layer depth of the node, the geometric position of the node, attribute information of the node, or the nearest neighbor information of the node.

[0309] Item 7. The method according to any one of items 1 to 6, wherein the region includes nodes of the current frame, and the prediction mode is determined based on rate-distortion optimization.

[0310] Item 8. The method according to Item 7, wherein at least one of the bit rate or distortion optimized for said rate-distortion is estimated.

[0311] Item 9. The method according to Item 7, wherein at least one of the bit rate or distortion optimized for said rate-distortion is determined based on at least one reconstructed value.

[0312] Item 10. The method according to any one of items 1 to 9, wherein the region includes a node of the current frame, and an indication associated with the node is used to indicate the prediction mode to be applied to the node.

[0313] Item 11. The method according to Item 10, wherein the indication is included in the bit stream.

[0314] Item 12. The method according to Item 10 or 11, wherein the indication is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding, or the indication is encoded or decoded in a predictive manner.

[0315] Item 13. The method according to any one of items 1 to 12, wherein the prediction mode includes a combination of a first prediction mode and a second prediction mode, and wherein if a first prediction value of the region based on the first prediction mode is not zero, then the prediction value of the region is determined as the first prediction value, or wherein if the first prediction value of the region based on the first prediction mode is zero, then a second prediction value of the region based on the second prediction mode is determined as the prediction value of the region.

[0316] Item 14. The method according to any one of items 1 to 13, wherein the region comprises a plurality of nodes, and the determined prediction pattern is applied to the plurality of nodes.

[0317] Item 15. The method according to any one of Items 1 to 14, wherein the prediction pattern is determined based on at least one qualifying condition, wherein the at least one qualifying condition is based on at least one of the following: the layer depth of the region, the geometric location of the region, attribute information of the region, or the nearest neighbor information of the nodes in the region.

[0318] Item 16. The method according to Item 15, wherein at least one indicator indicates at least one permitted layer depth or at least one permitted region in the at least one qualifying condition.

[0319] Item 17. The method according to Item 16, wherein the at least one indicator is indicated to the decoder.

[0320] Item 18. The method according to Item 16 or 17, wherein the at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0321] Item 19. The method according to Item 16 or 17, wherein the at least one indicator is encoded and decoded in a predictive manner.

[0322] Item 20. The method according to any one of items 1 to 19, wherein the prediction mode is determined based on rate-distortion optimization.

[0323] Item 21. The method according to Item 20, wherein at least one of the rate-distortion optimized bit rate or distortion is estimated.

[0324] Item 22. The method according to Item 20, wherein at least one of the rate-distortion optimized bit rate or distortion is calculated based on at least one reconstructed value.

[0325] Item 23. The method according to any one of items 1 to 22, wherein the indicator indicates that the prediction pattern is determined to be applied to nodes in the region.

[0326] Item 24. The method according to Item 23, wherein the indicator is indicated to the decoder.

[0327] Item 25. The method according to Item 23 or 24, wherein the indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0328] Item 26. The method according to Item 23 or 24, wherein the indicator is encoded and decoded in a predictive manner.

[0329] Item 27. The method according to any one of items 1 to 26, wherein at least one parameter for determining the prediction mode is indicated in the bitstream.

[0330] Item 28. The method according to Item 27, wherein the at least one parameter indicates a qualifying condition for the prediction mode.

[0331] Item 29. The method according to Item 27, wherein the eligibility criteria for the prediction mode are determined based on the at least one parameter.

[0332] 30. The method according to item 27, wherein the at least one parameter indicates the area for which at least one qualification condition needs to be checked.

[0333] Item 31. The method according to Item 30, wherein for a first prediction mode, a first set of layers needs to check the at least one qualifying condition, and a second set of layers does not apply the first prediction mode.

[0334] Item 32. The method according to Item 30, wherein for a first prediction mode, a first set of layers needs to check the at least one qualifying condition, and a second set of layers applies the first prediction mode.

[0335] Item 33. The method according to Item 31 or 32, wherein at least one of the first group of layers or the second group of layers is determined based on the at least one parameter.

[0336] Item 34. The method according to any one of items 27 to 33, wherein the at least one parameter is indicated to the decoder.

[0337] Item 35. The method according to Item 34, wherein the at least one parameter is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0338] Item 36. The method according to Item 34, wherein the at least one parameter is encoded and decoded in a predictive manner.

[0339] Item 37. The method according to any one of items 1 to 36, wherein the predicted value of the current node of the region is determined based on reference information from multiple reference frames.

[0340] Item 38. The method according to Item 37, wherein for each node of the region, a plurality of reference nodes are derived from the plurality of reference frames.

[0341] Item 39. The method according to Item 38, wherein for each reference node, an inter-frame prediction value is determined.

[0342] Item 40. The method according to Item 38 or 39, wherein for each reference node, an indicator indicates the reference node.

[0343] Item 41. The method according to any one of items 37 to 40, wherein, for the current node, the prediction value is determined based on the intra-frame prediction value of the current node and a plurality of inter-frame prediction values ​​of the current node.

[0344] Item 42. The method according to any one of items 37 to 40, wherein, for the current node, the predicted value is selected from a plurality of predicted value candidates.

[0345] Item 43. The method according to Item 42, wherein the plurality of prediction candidates includes intra-frame prediction values.

[0346] Item 44. The method according to Item 43, wherein if a reference node exists, the plurality of prediction candidates include the inter-frame prediction values ​​of the reference node.

[0347] Item 45. The method according to Item 42, wherein if multiple reference nodes exist, the multiple prediction candidates include intra-frame predictions and a weighted average of multiple inter-frame predictions of the multiple reference nodes.

[0348] Item 46. The method according to Item 42, wherein the plurality of prediction candidates includes a weighted average of intra-frame predictions and inter-frame predictions.

[0349] Item 47. The method according to Item 42, wherein if multiple reference nodes exist, the multiple prediction candidates include a weighted average of multiple inter-frame predictions.

[0350] Item 48. The method according to any one of items 42 to 47, wherein the selection is determined based on rate-distortion optimization.

[0351] Item 49. The method according to Item 48, wherein at least one of the rate-distortion optimized bit rate or distortion is estimated.

[0352] Item 50. The method according to Item 48, wherein at least one of the rate-distortion optimized bit rate or distortion is calculated based on at least one reconstructed value.

[0353] Item 51. The method according to any one of items 42 to 50, wherein the indicator indicates the selected predictive value candidate among the plurality of predictive value candidates.

[0354] Item 52. The method according to Item 51, wherein the indicator is indicated to the decoder.

[0355] Item 53. The method according to Item 51 or 52, wherein the indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0356] Item 54. The method according to Item 51 or 52, wherein the indicator is encoded and decoded in a predictive manner.

[0357] Item 55. The method according to any one of items 1 to 54, wherein multiple weights for determining the predicted value of the current node of the region are fixed, or indicated in the bitstream, or determined during the transformation.

[0358] Item 56. The method according to Item 55, wherein the plurality of weights are fixed at the decoder.

[0359] Item 57. The method according to Item 56, wherein the plurality of weights are variable for each layer or for each node.

[0360] Item 58. The method according to Item 56, wherein the plurality of weights are consistent for the plurality of nodes.

[0361] Item 59. The method according to Item 55, wherein the plurality of weights are derived at the decoder.

[0362] Item 60. The method according to Item 59, wherein the plurality of weights are derived for each layer.

[0363] Item 61. The method according to Item 60, wherein the plurality of weights are derived based on the candidate selection result of the predicted value of at least one node in the previous layer.

[0364] Item 62. The method according to Item 60, wherein the plurality of weights are derived based on the depth level.

[0365] Item 63. The method according to Item 59, wherein the plurality of weights are derived for each node.

[0366] Item 64. The method according to Item 63, wherein the plurality of weights are derived based on the prediction candidate selection result of at least one previously encoded node, the at least one previously encoded node including at least one of the following: an encoded neighbor node, or a parent node.

[0367] Item 65. The method according to Item 55, wherein the plurality of weights are indicated to the decoder.

[0368] Item 66. The method according to Item 65, wherein the plurality of weights are indicated to the decoder for each layer.

[0369] Item 67. The method according to Item 65 or 66, wherein the plurality of weights are encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0370] Item 68. The method according to Item 65 or 66, wherein the plurality of weights are encoded and decoded in a predictive manner.

[0371] Item 69. The method according to Item 55, wherein the plurality of weights are selected from a predefined list of weight candidates.

[0372] Item 70. The method according to Item 69, wherein the plurality of weights are selected based on one of: rate-distortion optimization, rate optimization, or distortion optimization.

[0373] Item 71. The method according to Item 69, wherein at least one indicator indicates the selection result from the predefined list of weighted candidates.

[0374] Item 72. According to the method of Item 71, wherein the at least one indicator is indicated to the decoder.

[0375] Item 73. The method according to Item 72, wherein the at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0376] Item 74. The method according to Item 72, wherein the at least one indicator is encoded and decoded in a predictive manner.

[0377] Item 75. The method according to Item 69, wherein the selection is performed for each layer or for each node.

[0378] Item 76. The method according to any one of items 1 to 75, wherein whether a weighted average of intra-frame prediction and inter-frame prediction is used is indicated.

[0379] Item 77. The method according to Item 76, wherein a first indicator indicates whether the weighted average of the intra-frame prediction and the inter-frame prediction is enabled for the entire sequence.

[0380] Item 78. The method according to Item 76, wherein the second indicator indicates whether the weighted average of the intra-frame prediction and the inter-frame prediction is enabled for point cloud (PC) samples.

[0381] Item 79. The method according to Item 76, wherein a third indicator indicates whether the weighted average of the intra-frame prediction and the inter-frame prediction is enabled for a layer or region.

[0382] Item 80. The method according to any one of items 77 to 79, wherein at least one of a first indicator, a second indicator, or a third indicator relating to the use of the weighted average is indicated to the decoder.

[0383] Item 81. The method according to Item 80, wherein at least one of the first indicator, the second indicator, or the third indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0384] Item 82. The method according to Item 80, wherein at least one of the first indicator, the second indicator, or the third indicator is encoded / decoded in a predictive manner.

[0385] Item 83. A method for point cloud encoding and decoding, comprising: for a conversion between a current frame of a point cloud sequence and a bitstream of the point cloud sequence, determining that a Region Adaptive Layered Transform (RAHT) attribute encoding and decoding is enabled for the current frame; and performing the conversion based on the RAHT attribute encoding and decoding, wherein if the RAHT attribute encoding and decoding is enabled for the point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is indicated in the bitstream, and wherein if the RAHT attribute encoding and decoding is disabled for the point cloud sequence, the at least one parameter of the at least one quantization matrix for RAHT coefficients is not included in the bitstream.

[0386] Item 84. The method according to Item 83, wherein the RAHT coefficient includes at least one of the following: AC coefficient or DC coefficient.

[0387] Item 85. The method according to Item 83 or 84, wherein the at least one quantization matrix comprises at least one of the following: at least one quantization matrix for AC coefficients, or at least one quantization matrix for DC coefficients.

[0388] Item 86. The method according to any one of items 83 to 85, wherein at least one indicator indicates whether the at least one quantization matrix for RAHT coefficients is indicated in the bitstream.

[0389] Item 87. The method according to Item 86, wherein if the RAHT attribute encoding / decoding is enabled, the at least one indicator is indicated to the decoder.

[0390] Item 88. The method according to Item 86 or 87, wherein the at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

[0391] Item 89. The method according to Item 86 or 87, wherein the at least one indicator is encoded and decoded in a predictive manner.

[0392] Item 90. The method according to any one of items 1 to 89, wherein an indicator indicating whether and / or how the method is applied is indicated in one of the following: the bitstream, frame, slice, slice, or octree.

[0393] Item 91. The method according to any one of items 1 to 89 further comprises: determining whether and / or how to apply the method based on encoded or decoded information, wherein the encoded or decoded information includes at least one of the following: dimension, color format, color components, slice type, or image type.

[0394] Item 92. The method according to any one of items 1 to 91, wherein the conversion includes encoding the current frame into the bitstream.

[0395] Item 93. The method according to any one of items 1 to 91, wherein the conversion includes decoding the current frame from the bitstream.

[0396] Item 94. An apparatus for processing point cloud data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of items 1 to 93.

[0397] Item 95. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of items 1 to 93.

[0398] Item 96. A non-transitory computer-readable recording medium storing a bitstream of a point cloud sequence generated by a method performed by a point cloud processing apparatus, wherein the method includes: determining a prediction mode for a region of a current frame of the point cloud sequence, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; determining a prediction of the region based on the prediction mode; and generating the bitstream based on the prediction.

[0399] Item 97. A method for storing a bitstream of a point cloud sequence, comprising: determining a prediction mode for a region of a current frame of the point cloud sequence, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; determining a prediction of the region based on the prediction mode; generating the bitstream based on the prediction; and storing the bitstream in a non-transitory computer-readable recording medium.

[0400] Item 98. A non-transitory computer-readable recording medium storing a bitstream of a point cloud sequence generated by a method performed by a point cloud processing apparatus, wherein the method comprises: determining that a Region Adaptive Layer Transform (RAHT) attribute codec is enabled for a current frame of the point cloud sequence; and generating the bitstream based on the RAHT attribute codec, wherein if the RAHT attribute codec is enabled for the current point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is indicated in the bitstream, and wherein if the RAHT attribute codec is disabled for the current point cloud sequence, the bitstream does not include the at least one parameter of the at least one quantization matrix for RAHT coefficients.

[0401] Item 99. A method for storing a bitstream of a point cloud sequence, comprising: determining that a Region Adaptive Layered Transform (RAHT) attribute codec is enabled for a current frame of the point cloud sequence; generating the bitstream based on the RAHT attribute codec; and storing the bitstream in a non-transitory computer-readable recording medium, wherein if the RAHT attribute codec is enabled for the current point cloud sequence, at least one parameter of at least one quantization matrix for RAHT coefficients is indicated in the bitstream, and wherein if the RAHT attribute codec is disabled for the current point cloud sequence, the bitstream does not include the at least one parameter of the at least one quantization matrix for RAHT coefficients.

[0402] Example device Figure 8 A block diagram of a computing device 800 in which various embodiments of the present disclosure may be implemented is shown. The computing device 800 may be implemented as a source device 110 (or GPCC encoder 116 or 200) or a target device 120 (or GPCC decoder 126 or 300), or may be included in a source device 110 (or GPCC encoder 116 or 200) or a target device 120 (or GPCC decoder 126 or 300).

[0403] It should be understood that, Figure 8 The computing device 800 shown is for illustrative purposes only and is not intended to imply any limitation on the functionality and scope of the embodiments of this disclosure.

[0404] like Figure 8 As shown, the computing device 800 includes a general-purpose computing device 800. The computing device 800 may include at least one or more processors or processing units 810, memory 820, storage units 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860.

[0405] In some embodiments, the computing device 800 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, and includes accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 800 can support any type of interface to the user (such as "wearable" circuitry devices, etc.).

[0406] Processing unit 810 can be a physical processor or a virtual processor, and can perform various processes based on programs stored in memory 820. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of computing device 800. Processing unit 810 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.

[0407] Computing device 800 typically includes various computer storage media. Such media can be any media accessible by computing device 800, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 830 can be any removable or non-removable media and may include machine-readable media, such as memory, flash drives, disks, or other media that can be used to store information and / or data and can be accessed within computing device 800.

[0408] The computing device 800 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Although in Figure 8 Not shown, but may provide disk drives for reading from and / or writing to removable non-volatile disks, and optical disc drives for reading from and / or writing to removable non-volatile optical discs. In this case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0409] The communication unit 840 communicates with another computing device via a communication medium. Furthermore, the functionality of the components in the computing device 800 can be implemented by a single computing cluster or by multiple computing machines communicating via communication connections. Therefore, the computing device 800 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0410] Input device 850 can be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 860 can be one or more of various output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 840, computing device 800 can also communicate with one or more external devices (not shown), such as storage devices and display devices, one or more devices that enable a user to interact with computing device 800, or any device that enables computing device 800 to communicate with one or more other computing devices (e.g., network card, modem, etc.), if needed. This communication can be performed via an input / output (I / O) interface (not shown).

[0411] In some embodiments, some or all components of computing device 800 may be arranged in a cloud computing architecture, rather than integrated into a single device. In a cloud computing architecture, components may be provided remotely and work together to achieve the functionality described herein. In some embodiments, cloud computing provides computing, software, data access, and storage services without requiring the end user to know the physical location or configuration of the system or hardware providing these services. In various embodiments, cloud computing is provided via a wide area network (WAN) such as the Internet using suitable protocols. For example, a cloud computing provider provides applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at a remote location. Computing resources in a cloud computing environment may be consolidated or distributed at locations in remote data centers. Cloud computing infrastructure may be provided through shared data centers, although they may appear as a single access point to the user. Thus, cloud computing architectures can be used to provide the components and functionality described herein from service providers at remote locations. Alternatively, they may be provided from traditional servers or installed directly or otherwise on client devices.

[0412] In embodiments of this disclosure, computing device 800 can be used to implement point cloud encoding / decoding. Memory 820 may include one or more point cloud encoding / decoding modules 825 having one or more program instructions. These modules can be accessed and executed by processing unit 810 to perform the functions of the various embodiments described herein.

[0413] In an example embodiment of point cloud encoding, input device 850 may receive point cloud data as input 870 to be encoded. The point cloud data may be processed, for example, by point cloud encoding / decoding module 825 to generate an encoded bitstream. The encoded bitstream may be provided as output 880 via output device 860.

[0414] In an example embodiment of point cloud decoding, input device 850 may receive an encoded bitstream as input 870. The encoded bitstream may be processed, for example, by a point cloud encoding / decoding module 825 to generate decoded point cloud data. The decoded point cloud data may be provided as output 880 via output device 860.

[0415] While this disclosure has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application as defined by the appended claims. These changes are intended to be covered by the scope of this application. Therefore, the foregoing description of embodiments of this application is not intended to be limiting.

Claims

1. A method for point cloud encoding and decoding, comprising: For the conversion between the current frame of the point cloud sequence and the bitstream of the point cloud sequence, a prediction mode for the region of the current frame is determined, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction. The prediction for the region is determined based on the prediction model; as well as The transformation is performed based on the prediction.

2. The method of claim 1, wherein in the first mode, the prediction of the region comprises a weighted average of the intra-frame prediction and the inter-frame prediction of the region.

3. The method according to claim 1 or 2, wherein the prediction mode is indicated in the bitstream.

4. The method according to any one of claims 1 to 3, wherein the prediction mode further comprises at least one of the following: No predictive pattern Intra-frame prediction mode, or Inter-frame prediction mode.

5. The method according to any one of claims 1 to 4, wherein the region includes at least one of the following: a node of the current frame, or a layer of the current frame.

6. The method according to any one of claims 1 to 5, wherein the region includes nodes of the current frame, and the prediction mode is determined based on at least one qualifying condition, wherein the at least one qualifying condition is based on at least one of the following: The layer depth of the node, The geometric position of the node, The attribute information of the node, or The nearest neighbor information of the node.

7. The method according to any one of claims 1 to 6, wherein the region includes nodes of the current frame, and the prediction mode is determined based on rate-distortion optimization.

8. The method of claim 7, wherein at least one of the bit rate or distortion optimized for the rate-distortion is estimated.

9. The method of claim 7, wherein at least one of the bit rate or distortion optimized for the rate-distortion is determined based on at least one reconstruction value.

10. The method of any one of claims 1 to 9, wherein the region includes a node of the current frame, and an indication associated with the node indicates the prediction mode to be applied to the node.

11. The method of claim 10, wherein the indication is included in the bit stream.

12. The method of claim 10 or 11, wherein the indication is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding, or the indication is encoded or decoded in a predictive manner.

13. The method according to any one of claims 1 to 12, wherein the prediction mode comprises a combination of a first prediction mode and a second prediction mode, and If the first predicted value of the region based on the first prediction model is not zero, then the predicted value of the region is determined to be the first predicted value, or If the first predicted value of the region based on the first prediction mode is zero, then the second predicted value of the region based on the second prediction mode is determined as the predicted value of the region.

14. The method according to any one of claims 1 to 13, wherein the region comprises a plurality of nodes, and the determined prediction pattern is applied to the plurality of nodes.

15. The method according to any one of claims 1 to 14, wherein the prediction pattern is determined based on at least one qualifying condition, wherein the at least one qualifying condition is based on at least one of the following: the layer depth of the region, the geometric location of the region, attribute information of the region, or the nearest neighbor information of the nodes in the region.

16. The method of claim 15, wherein at least one indicator indicates at least one permitted layer depth or at least one permitted region in the at least one qualifying condition.

17. The method of claim 16, wherein the at least one indicator is indicated to the decoder.

18. The method of claim 16 or 17, wherein the at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

19. The method of claim 16 or 17, wherein the at least one indicator is encoded and decoded in a predictive manner.

20. The method according to any one of claims 1 to 19, wherein the prediction mode is determined based on rate-distortion optimization.

21. The method of claim 20, wherein at least one of the rate-distortion optimized bit rate or distortion is estimated.

22. The method of claim 20, wherein at least one of the rate-distortion optimized bit rate or distortion is calculated based on at least one reconstructed value.

23. The method according to any one of claims 1 to 22, wherein the indicator indicates that the prediction pattern is determined to be applied to nodes in the region.

24. The method of claim 23, wherein the indicator is indicated to the decoder.

25. The method of claim 23 or 24, wherein the indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

26. The method of claim 23 or 24, wherein the indicator is encoded and decoded in a predictive manner.

27. The method according to any one of claims 1 to 26, wherein at least one parameter for determining the prediction mode is indicated in the bitstream.

28. The method of claim 27, wherein the at least one parameter indicates a qualifying condition for the prediction mode.

29. The method of claim 27, wherein the eligibility criteria for the predicted pattern are determined based on the at least one parameter.

30. The method of claim 27, wherein the at least one parameter indicates the area for which at least one qualification condition needs to be checked.

31. The method of claim 30, wherein for the first prediction mode, the first group of layers needs to check the at least one qualifying condition, and the second group of layers does not apply the first prediction mode.

32. The method of claim 30, wherein for the first prediction mode, the first group of layers needs to check the at least one qualification condition, and the second group of layers applies the first prediction mode.

33. The method of claim 31 or 32, wherein at least one of the first group of layers or the second group of layers is determined based on the at least one parameter.

34. The method according to any one of claims 27 to 33, wherein the at least one parameter is indicated to the decoder.

35. The method of claim 34, wherein the at least one parameter is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

36. The method of claim 34, wherein the at least one parameter is encoded and decoded in a predictive manner.

37. The method according to any one of claims 1 to 36, wherein the predicted value of the current node of the region is determined based on reference information from a plurality of reference frames.

38. The method of claim 37, wherein for each node of the region, a plurality of reference nodes are derived from the plurality of reference frames.

39. The method of claim 38, wherein for each reference node, an inter-frame prediction value is determined.

40. The method of claim 38 or 39, wherein for each reference node, an indicator indicates the reference node.

41. The method according to any one of claims 37 to 40, wherein, for the current node, the prediction value is determined based on intra-frame prediction values ​​of the current node and a plurality of inter-frame prediction values ​​of the current node.

42. The method according to any one of claims 37 to 40, wherein, for the current node, the predicted value is selected from a plurality of predicted value candidates.

43. The method of claim 42, wherein the plurality of prediction candidates includes intra-frame prediction values.

44. The method of claim 43, wherein if a reference node exists, the plurality of prediction candidates include inter-frame predictions of the reference node.

45. The method of claim 42, wherein if multiple reference nodes exist, the multiple prediction candidates include intra-frame predictions and a weighted average of multiple inter-frame predictions of the multiple reference nodes.

46. ​​The method of claim 42, wherein the plurality of prediction candidates comprises a weighted average of intra-frame predictions and inter-frame predictions.

47. The method of claim 42, wherein if multiple reference nodes exist, the multiple prediction candidates comprise a weighted average of multiple inter-frame predictions.

48. The method according to any one of claims 42 to 47, wherein the selection is determined based on rate-distortion optimization.

49. The method of claim 48, wherein at least one of the rate-distortion optimized bit rate or distortion is estimated.

50. The method of claim 48, wherein at least one of the rate-distortion optimized bit rate or distortion is calculated based on at least one reconstructed value.

51. The method according to any one of claims 42 to 50, wherein the indicator indicates the selected prediction candidate among the plurality of prediction candidates.

52. The method of claim 51, wherein the indicator is indicated to the decoder.

53. The method of claim 51 or 52, wherein the indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

54. The method of claim 51 or 52, wherein the indicator is encoded and decoded in a predictive manner.

55. The method according to any one of claims 1 to 54, wherein the weights for determining the predicted value of the current node of the region are fixed, or indicated in the bitstream, or determined during the conversion.

56. The method of claim 55, wherein the plurality of weights are fixed at the decoder.

57. The method of claim 56, wherein the plurality of weights are variable for each layer or for each node.

58. The method of claim 56, wherein the plurality of weights are consistent for the plurality of nodes.

59. The method of claim 55, wherein the plurality of weights are derived at the decoder.

60. The method of claim 59, wherein the plurality of weights are derived for each layer.

61. The method of claim 60, wherein the plurality of weights are derived based on the candidate selection result of the predicted value of at least one node in the previous layer.

62. The method of claim 60, wherein the plurality of weights are derived based on the depth level.

63. The method of claim 59, wherein the plurality of weights are derived for each node.

64. The method of claim 63, wherein the plurality of weights are derived based on the candidate selection result of the predicted value of at least one previously encoded node, the at least one previously encoded node including at least one of the following: an encoded neighbor node, or a parent node.

65. The method of claim 55, wherein the plurality of weights are indicated to the decoder.

66. The method of claim 65, wherein the plurality of weights are indicated to the decoder for each layer.

67. The method of claim 65 or 66, wherein the plurality of weights are encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

68. The method of claim 65 or 66, wherein the plurality of weights are encoded and decoded in a predictive manner.

69. The method of claim 55, wherein the plurality of weights are selected from a predefined list of weight candidates.

70. The method of claim 69, wherein the plurality of weights are selected based on one of: rate-distortion optimization, rate optimization, or distortion optimization.

71. The method of claim 69, wherein at least one indicator indicates the selection result from the predefined list of weighted candidates.

72. The method of claim 71, wherein the at least one indicator is indicated to the decoder.

73. The method of claim 72, wherein the at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

74. The method of claim 72, wherein the at least one indicator is encoded and decoded in a predictive manner.

75. The method of claim 69, wherein the selection is performed for each layer or for each node.

76. The method according to any one of claims 1 to 75, wherein whether a weighted average of intra-frame prediction and inter-frame prediction is used is indicated.

77. The method of claim 76, wherein the first indicator indicates whether the weighted average of the intra-frame prediction and the inter-frame prediction is enabled for the entire sequence.

78. The method of claim 76, wherein the second indicator indicates whether the weighted average of the intra-frame prediction and the inter-frame prediction is enabled for point cloud (PC) samples.

79. The method of claim 76, wherein a third indicator indicates whether the weighted average of the intra-frame prediction and the inter-frame prediction is enabled for a layer or region.

80. The method according to any one of claims 77 to 79, wherein at least one of a first indicator, a second indicator, or a third indicator relating to the use of the weighted average is indicated to the decoder.

81. The method of claim 80, wherein at least one of the first indicator, the second indicator, or the third indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

82. The method of claim 80, wherein at least one of the first indicator, the second indicator, or the third indicator is encoded / decoded in a predictive manner.

83. A method for point cloud encoding and decoding, comprising: For the conversion between the current frame of the point cloud sequence and the bitstream of the point cloud sequence, it is determined that the Region Adaptive Hierarchical Transform (RAHT) attribute encoding and decoding is enabled for the current frame; as well as The conversion is performed based on the RAHT attribute encoding / decoding. If the RAHT attribute encoding / decoding is enabled for the point cloud sequence, then at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream, and If the RAHT attribute encoding / decoding is disabled for the point cloud sequence, then the bitstream does not include the at least one parameter of the at least one quantization matrix for the RAHT coefficients.

84. The method of claim 83, wherein the RAHT coefficient comprises at least one of the following: AC coefficient or DC coefficient.

85. The method according to claim 83 or 84, wherein the at least one quantization matrix comprises at least one of the following: at least one quantization matrix for AC coefficients, or at least one quantization matrix for DC coefficients.

86. The method according to any one of claims 83 to 85, wherein at least one indicator indicates whether the at least one quantization matrix for RAHT coefficients is indicated in the bitstream.

87. The method of claim 86, wherein if the RAHT attribute encoding / decoding is enabled, the at least one indicator is indicated to the decoder.

88. The method of claim 86 or 87, wherein the at least one indicator is encoded or decoded using one of the following: fixed-length encoding / decoding, unary encoding / decoding, or rounding unary encoding / decoding.

89. The method of claim 86 or 87, wherein the at least one indicator is encoded and decoded in a predictive manner.

90. The method according to any one of claims 1 to 89, wherein an indicator indicating whether and / or how the method is applied is indicated in one of the following: the bitstream, frame, slice, slice, or octree.

91. The method according to any one of claims 1 to 89, further comprising: The method is determined based on encoded and decoded information, which includes at least one of the following: dimension, color format, color components, slice type, or image type.

92. The method according to any one of claims 1 to 91, wherein the conversion comprises encoding the current frame into the bitstream.

93. The method according to any one of claims 1 to 91, wherein the conversion comprises decoding the current frame from the bitstream.

94. An apparatus for processing point cloud data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 93.

95. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1 to 93.

96. A non-transitory computer-readable recording medium storing a bitstream of a point cloud sequence generated by a method performed by a point cloud processing apparatus, wherein the method includes: Determine a prediction mode for the region of the current frame of the point cloud sequence, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; The prediction for the region is determined based on the prediction model; as well as The bitstream is generated based on the prediction.

97. A method for storing a bitstream of a point cloud sequence, comprising: Determine a prediction mode for the region of the current frame of the point cloud sequence, the prediction mode including at least a first mode based on intra-frame prediction and inter-frame prediction; The prediction for the region is determined based on the prediction model; The bitstream is generated based on the prediction; as well as The bitstream is stored in a non-transitory computer-readable recording medium.

98. A non-transitory computer-readable recording medium for storing a bitstream of a point cloud sequence generated by a method performed by a point cloud processing apparatus, wherein the method includes: The Region Adaptive Hierarchical Transformation (RAHT) attribute encoding and decoding is enabled for the current frame of the point cloud sequence; as well as The bitstream is generated based on the RAHT attribute encoding and decoding. If the RAHT attribute encoding / decoding is enabled for the current point cloud sequence, then at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream, and If the RAHT attribute encoding / decoding is disabled for the current point cloud sequence, then the bitstream does not include the at least one parameter of the at least one quantization matrix for the RAHT coefficients.

99. A method for storing a bitstream of a point cloud sequence, comprising: The Region Adaptive Hierarchical Transformation (RAHT) attribute encoding and decoding is enabled for the current frame of the point cloud sequence; The bitstream is generated by encoding and decoding based on the RAHT attribute; as well as The bitstream is stored in a non-transitory computer-readable recording medium. If the RAHT attribute encoding / decoding is enabled for the current point cloud sequence, then at least one parameter of at least one quantization matrix for the RAHT coefficients is indicated in the bitstream, and If the RAHT attribute encoding / decoding is disabled for the current point cloud sequence, then the bitstream does not include the at least one parameter of the at least one quantization matrix for the RAHT coefficients.

Images