Methods, apparatus and media for point cloud encoding and decoding

Abstract

Embodiments of this disclosure provide a solution for point cloud encoding and decoding. In the method for point cloud encoding and decoding, for the conversion between the current frame of a point cloud sequence and a bitstream of the point cloud sequence, a prediction mode for the current node of the current frame is determined based on an indication in the bitstream. The indication is encoded and decoded using context-based adaptive binary arithmetic encoding and decoding. The prediction for the current node is determined based on the prediction mode. The conversion is performed based on the prediction.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to point cloud technology, and more specifically, to prediction pattern determination for point cloud encoding and decoding. Background Technology

[0002] A point cloud is a collection of data points in a three-dimensional (3D) plane, where each point has defined coordinates on the X, Y, and Z axes. Therefore, point clouds can be used to represent the physical content of three-dimensional space. For a wide range of immersive applications, from augmented reality to autonomous vehicles, point clouds have proven to be a promising way to represent 3D visual data.

[0003] Point cloud encoding and decoding standards have largely evolved from the well-known MPEG organization. MPEG stands for Moving Picture Experts Group, one of the main standardization groups for multimedia processing. In 2017, the MPEG 3D Graphics Codec Group (3DG) released a Call for Proposals (CFP) document to begin developing point cloud encoding and decoding standards. The final standard will encompass two categories of solutions. Video-based point cloud compression (V-PCC or VPCC) is suitable for point sets with relatively uniform point distribution. Geometry-based point cloud compression (G-PCC or GPCC) is suitable for sparser distributions. However, the overall expectation is to further improve the encoding and decoding efficiency of conventional point cloud encoding and decoding techniques. Summary of the Invention

[0004] Embodiments of this disclosure provide a solution for point cloud encoding and decoding.

[0005] In a first aspect, a method for point cloud encoding and decoding is proposed. The method includes: a conversion between a current frame of a point cloud sequence and a bitstream of the point cloud sequence; determining a prediction mode for a current node in the current frame based on an indication in the bitstream, wherein the indication is encoded and decoded using context-based adaptive binary arithmetic encoding and decoding; determining a prediction for the current node based on the prediction mode; and performing a conversion based on the prediction. The method according to the first aspect of this disclosure can determine predictions more accurately, thus improving the encoding and decoding efficiency of point cloud encoding and decoding.

[0006] In a second aspect, an apparatus for point cloud encoding and decoding is proposed. The apparatus includes a processor and a non-transitory memory having instructions thereon. When executed by the processor, the instructions cause the processor to perform the method according to the first aspect of this disclosure.

[0007] In a third aspect, a non-transitory computer-readable storage medium is proposed. This non-transitory computer-readable storage medium stores instructions that cause a processor to execute the method according to the first aspect of this disclosure.

[0008] In a fourth aspect, another non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of a point cloud sequence generated by a method performed by means of a point cloud encoding / decoding apparatus. The method includes: determining a prediction mode for the current node of the current frame of the point cloud sequence based on an indication in the bitstream, wherein the indication is encoded / decoded via context-based adaptive binary arithmetic encoding / decoding; determining a prediction for the current node based on the prediction mode; and generating a bitstream based on the prediction.

[0009] In a fifth aspect, a method for storing a bitstream of a point cloud sequence is proposed. The method includes: determining a prediction mode for the current node of the current frame of the point cloud sequence based on an indication in the bitstream, wherein the indication is encoded / decoded via context-based adaptive binary arithmetic encoding / decoding; determining a prediction for the current node based on the prediction mode; generating a bitstream based on the prediction; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] This summary aims to present, in a simplified form, the selected concepts further described below in the detailed embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0011] The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become clearer from the following detailed description with reference to the accompanying drawings, in which the same reference numerals generally refer to the same parts.

[0012] Figure 1 A block diagram of an example point cloud encoding / decoding system according to some embodiments of the present disclosure is shown; Figure 2 A block diagram illustrating an example of a GPCC encoder according to some embodiments of the present disclosure is shown; Figure 3 A block diagram illustrating an example of a GPCC decoder according to some embodiments of the present disclosure is shown; Figure 4 The parent node for each child node of the transformation unit node is shown; Figure 5 An example flowchart illustrating improved attribute prediction when there are two reference frames, according to some embodiments of the present disclosure; Figure 6 A flowchart of a method for point cloud encoding / decoding according to some embodiments of the present disclosure is shown; and Figure 7 A block diagram of a computing device in which various embodiments of the present disclosure may be implemented is shown.

[0013] In all accompanying drawings, the same or similar reference numerals usually refer to the same or similar elements. Detailed Implementation

[0014] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement this disclosure, and do not imply any limitation on the scope of this disclosure. In addition to the methods described below, the disclosure described herein can be implemented in various other ways.

[0015] In the following description and claims, unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0016] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment is required to include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, whether explicitly described or not, it is believed that such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0017] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” “having,” “having,” “containing,” and / or “comprising” as used herein indicate the presence of the said features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0019] Example Environment Figure 1This is a block diagram illustrating an example point cloud encoding / decoding system 100 from which the techniques of this disclosure can be utilized. As shown, the point cloud encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a point cloud encoding device, and the destination device 120 may also be referred to as a point cloud decoding device. In operation, the source device 110 may be configured to generate encoded point cloud data, and the destination device 120 may be configured to decode the encoded point cloud data generated by the source device 110. The techniques of this disclosure are generally directed to encoding and / or decoding point cloud data, i.e., supporting point cloud compression. Encoding and decoding can be effective in compressing and / or decompressing point cloud data.

[0020] Source device 100 and destination device 120 may include any of a variety of devices, including desktop computers, laptops, tablets, set-top boxes, handsets (such as smartphones and mobile phones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, vehicles (e.g., land or sea vehicles, spacecraft, aircraft, etc.), robots, LiDAR devices, satellites, extended reality devices, etc. In some cases, source device 100 and destination device 120 may be equipped for wireless communication.

[0021] Source device 100 may include a data source 112, a memory 114, a GPCC encoder 116, and an input / output (I / O) interface 118. Destination device 120 may include an input / output (I / O) interface 128, a GPCC decoder 126, a memory 124, and a data consumer 122. According to this disclosure, the GPCC encoder 116 of source device 100 and the GPCC decoder 126 of destination device 120 may be configured to apply the point cloud encoding / decoding techniques of this disclosure. Therefore, source device 100 represents an example of an encoding device, and destination device 120 represents an example of a decoding device. In other examples, source device 100 and destination device 120 may include other components or arrangements. For example, source device 100 may receive data (e.g., point cloud data) from an internal or external source. Similarly, destination device 120 may interface with an external data consumer rather than including the data consumer in the same device.

[0022] Generally, data source 112 represents a source of point cloud data (i.e., raw, unencoded point cloud data) and can provide a continuous series of "frames" of point cloud data to GPCC encoder 116, which encodes the point cloud data for each frame. In some examples, data source 112 generates point cloud data. The data source 112 of source device 100 may include point cloud acquisition devices, such as any of various cameras or sensors, such as one or more cameras, an archive containing previously acquired point cloud data, a 3D scanner or light detection and ranging (LIDAR) device, and / or a data feed interface that receives point cloud data from a data content provider. Thus, in some examples, data source 112 may generate point cloud data based on signals from a LIDAR device. Alternatively or additionally, point cloud data may be generated by a computer from scanners, cameras, sensors, or other data. For example, data source 112 may generate point cloud data, or produce a combination of real-time point cloud data, archived point cloud data, and computer-generated point cloud data. In each case, the GPCC encoder 116 encodes the acquired, pre-acquired, or computer-generated point cloud data. The GPCC encoder 116 can rearrange the frames of the point cloud data from the receiving order (sometimes referred to as the "display order") to an encoding / decoding order for encoding and decoding. The GPCC encoder 116 can generate one or more bitstreams comprising the encoded point cloud data. The source device 100 can then output the encoded point cloud data via I / O interface 118 for reception and / or retrieval by, for example, the I / O interface 128 of the destination device 120. The encoded point cloud data can be directly transmitted to the destination device 120 via I / O interface 118 through network 130A. The encoded point cloud data can also be stored on storage medium / server 130B for access by the destination device 120.

[0023] The memory 114 of the source device 100 and the memory 124 of the destination device 120 may represent general-purpose memory. In some examples, memory 114 and memory 124 may store raw point cloud data, such as raw point cloud data from data source 112 and raw, decoded point cloud data from GPCC decoder 126. Additionally or alternatively, memory 114 and memory 124 may store software instructions executable by, for example, GPCC encoder 116 and GPCC decoder 126. Although memory 114 and memory 124 are shown separately from GPCC encoder 116 and GPCC decoder 126 in this example, it should be understood that GPCC encoder 116 and GPCC decoder 126 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 114 and memory 124 may store encoded point cloud data, such as encoded point cloud data output from GPCC encoder 116 and input to GPCC decoder 126. In some examples, portions of memory 114 and memory 124 may be allocated as one or more caches, for example, to store raw point cloud data, decoded and / or encoded point cloud data. For example, memory 114 and memory 124 may store point cloud data.

[0024] I / O interfaces 118 and 128 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where I / O interfaces 118 and 128 include wireless components, they may be configured to transmit data, such as encoded point cloud data, according to cellular communication standards such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc. In some examples where I / O interface 118 includes a wireless transmitter, they may be configured to transmit data, such as encoded point cloud data, according to other wireless standards such as the IEEE 802.11 specification. In some examples, source device 100 and / or destination device 120 may include corresponding system-on-chip (SoC) devices. For example, source device 100 may include a SoC device for performing functions belonging to GPCC encoder 116 and / or I / O interface 118, and destination device 120 may include a SoC device for performing functions belonging to GPCC decoder 126 and / or I / O interface 128.

[0025] The techniques disclosed herein can be applied to encoding and decoding to support any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices (e.g., local or remote servers), geographic mapping, or other applications.

[0026] The I / O interface 128 of the destination device 120 receives an encoded bitstream from the source device 110. The encoded bitstream may include signaling information defined by the GPCC encoder 116, which is also used by the GPCC decoder 126, such as syntax elements having values ​​representing the point cloud. The data consumer 122 uses the decoded data. For example, the data consumer 122 may use the decoded point cloud data to determine the location of physical objects. In some examples, the data consumer 122 may include a display for presenting images based on the point cloud data.

[0027] The GPCC encoder 116 and GPCC decoder 126 can each be implemented as any of a variety of suitable encoder circuitry and / or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the GPCC encoder 116 and GPCC decoder 126 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the GPCC encoder 116 and / or GPCC decoder 126 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0028] The GPCC encoder 116 and GPCC decoder 126 can operate according to encoding / decoding standards such as the Video Point Cloud Compression (VPCC) standard or the Geometric Point Cloud Compression (GPCC) standard. Generally, this disclosure may refer to the encoding and decoding of frames (e.g., encoding and decoding) to include the process of encoding or decoding data. Encoded bitstreams typically include a series of values ​​for syntax elements representing encoding / decoding decisions (e.g., encoding / decoding modes).

[0029] A point cloud can contain a set of points in 3D space and can have attributes associated with those points. Attributes can be color information, such as R, G, B or Y, Cb, Cr, or reflectivity information, or other attributes. Point clouds can be acquired by various cameras or sensors, such as LiDAR sensors and 3D scanners, and can also be computer-generated. Point cloud data is used in a variety of applications, including but not limited to architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LiDAR sensors for navigation aids).

[0030] Figure 2 This is a block diagram illustrating an example of a GPCC encoder 200 according to some embodiments of the present disclosure. The GPCC encoder 200 may be... Figure 1 An example of a GPCC encoder 116 in system 100 is shown. Figure 3 This is a block diagram illustrating an example of a GPCC decoder 300 according to some embodiments of the present disclosure. The GPCC decoder 300 may be... Figure 1 An example of the GPCC decoder 126 in the system 100 shown.

[0031] In both the GPCC encoder 200 and GPCC decoder 300, point cloud locations are encoded and decoded first. Attribute encoding and decoding depend on the decoded geometry. Figure 2 and Figure 3 In this configuration, Region Adaptive Hierarchical Transformation (RAHT) unit 218, Surface Approximation Analysis unit 212, RAHT unit 314, and Surface Approximation Synthesis unit 310 are options typically used for Category 1 data. Level of Detail (LOD) generation unit 220, Lifting unit 222, LOD generation unit 316, and Inverse Lifting unit 318 are options typically used for Category 3 data. All other units are common to both Category 1 and Category 3 data.

[0032] For Category 3 data, the compressed geometry is typically represented as an octree from the root down to the leaf level of each voxel. For Category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root down to the leaf level of blocks larger than voxels) plus a model for approximating the surface within each leaf node of the pruned octree. In this way, both Category 1 and Category 3 data share the octree encoding / decoding mechanism, while Category 1 data can additionally utilize the surface model to approximate the voxels within each leaf node. The surface model used is a triangulation of each block comprising 1 to 10 triangles, producing a triangle soup. Therefore, the Category 1 geometry codec is called a triangle soup geometry codec, while the Category 3 geometry codec is called an octree geometry codec.

[0033] exist Figure 2 In the example, the GPCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometric reconstruction unit 216, a RAHT unit 218, a LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, and an arithmetic coding unit 226.

[0034] like Figure 2 As shown in the example, the GPCC encoder 200 can receive a set of locations and a set of attributes. Locations can include the coordinates of points in the point cloud. Attributes can include information about the points in the point cloud, such as the colors associated with those points.

[0035] The coordinate transformation unit 202 can apply transformations to the coordinates of a point to transform the coordinates from the initial domain to the transformation domain. The transformed coordinates can be referred to as transformed coordinates. The color transformation unit 204 can apply transformations to convert the color information of an attribute to different domains. For example, the color transformation unit 204 can convert color information from the RGB color space to the YCbCr color space.

[0036] In addition, Figure 2 In the example, voxelization unit 206 can voxelize the transformed coordinates. Voxelization of the transformed coordinates can include quantization and removal of some points in the point cloud. In other words, multiple points in the point cloud can be grouped into a single "voxel," which can then be considered a point in some respects. Furthermore, octree analysis unit 210 can generate an octree based on the voxelized transformed coordinates. Additionally, in Figure 2 In the example, the surface approximation analysis unit 212 can analyze points to potentially determine a surface representation of the set of points. The arithmetic coding unit 214 can perform arithmetic coding on syntax elements representing information about an octree and / or information about the surface determined by the surface approximation analysis unit 212. The GPCC encoder 200 can output these syntax elements in a geometric bitstream.

[0037] The geometric reconstruction unit 216 can reconstruct the transformed coordinates of points in the point cloud based on an octree, data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometric reconstruction unit 216 may differ from the original number of points in the point cloud. The resulting points may be referred to as reconstructed points. The attribute transfer unit 208 can transfer attributes of the original points in the point cloud to the reconstructed points in the point cloud data.

[0038] Furthermore, RAHT unit 218 can apply RAHT encoding to the attributes of the reconstructed points. Alternatively or additionally, LOD generation unit 220 and lifting unit 222 can apply LOD processing and lifting to the attributes of the reconstructed points, respectively. RAHT unit 218 and lifting unit 222 can generate coefficients based on the attributes. Coefficient quantization unit 224 can quantize the coefficients generated by RAHT unit 218 or lifting unit 222. Arithmetic encoding unit 226 can apply arithmetic encoding to the syntax elements representing the quantized coefficients. GPCC encoder 200 can output these syntax elements in the attribute bitstream.

[0039] exist Figure 3 In the example, the GPCC decoder 300 may include a geometric arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometric reconstruction unit 312, a RAHT unit 314, an LOD generation unit 316, an inverse lifting unit 318, a coordinate inverse transformation unit 320, and a color inverse transformation unit 322.

[0040] The GPCC decoder 300 can obtain a geometric bitstream and an attribute bitstream. The geometric arithmetic decoding unit 302 of the decoder 300 can apply arithmetic decoding (e.g., CABAC or other types of arithmetic decoding) to the syntax elements in the geometric bitstream. Similarly, the attribute arithmetic decoding unit 304 can apply arithmetic decoding to the syntax elements in the attribute bitstream.

[0041] Octree synthesis unit 306 can synthesize octrees based on syntax elements parsed from the geometric bitstream. In the case of using surface approximation in the geometric bitstream, surface approximation synthesis unit 310 can determine the surface model based on syntax elements parsed from the geometric bitstream and based on the octree.

[0042] Furthermore, the geometric reconstruction unit 312 can perform reconstruction to determine the coordinates of points in the point cloud. The inverse coordinate transformation unit 320 can apply an inverse transformation to the reconstructed coordinates to transform the reconstructed coordinates (positions) of points in the point cloud from the transformation domain back to the initial domain.

[0043] Additionally, in Figure 3 In the example, the dequantization unit 308 can dequantize the attribute value. The attribute value can be based on syntax elements obtained from the attribute bitstream (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0044] Depending on how the attribute values ​​are encoded, RAHT unit 314 can perform RAHT decoding to determine the color value for a point in the point cloud based on the dequantized attribute values. Alternatively, LOD generation unit 316 and inverse boosting unit 318 can use level-of-detail (LOD) based techniques to determine the color value for a point in the point cloud.

[0045] In addition, Figure 3 In the example, the color inverse transformation unit 322 can apply an inverse color transformation to color values. The inverse color transformation can be the inverse of the color transformation applied by the color transformation unit 204 of the encoder 200. For example, the color transformation unit 204 can transform color information from the RGB color space to the YCbCr color space. Correspondingly, the color inverse transformation unit 322 can transform color information from the YCbCr color space to the RGB color space.

[0046] Figure 2 and Figure 3 Various units are shown to aid in understanding the operations performed by encoder 200 and decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit is a circuit that provides a specific function and is preset with respect to the operations that can be performed. A programmable circuit is a circuit that can be programmed to perform various tasks and provides flexible functionality in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions in the software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more of these units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more of these units can be integrated circuits.

[0047] Some exemplary embodiments of this disclosure will be described in detail below. It should be understood that section headings are used in this document for ease of understanding and not to limit the embodiments disclosed in a section to that section. Furthermore, while some embodiments are described with reference to GPCC or other specific point cloud codecs, the disclosed techniques are also applicable to other point cloud codec techniques. Additionally, although some embodiments describe point cloud codec steps in detail, it should be understood that the corresponding decoding steps for decoding will be implemented by the decoder.

[0048] 1. Brief Overview This disclosure relates to point cloud encoding and decoding techniques. Specifically, it relates to inter-frame prediction of point cloud attributes in region adaptive hierarchical transformation. This idea can be applied alone or in various combinations to any standard or non-standard point cloud codec, such as geometry-based point cloud compression (G-PCC) under development.

[0049] 2. Abbreviation G-PCC is a geometry-based point cloud compression technology. MPEG Moving Picture Experts Group 3DG3D graphics codec group CFP Proposal Solicitation V-PCC is a video-based point cloud compression technology. RAHT region adaptive hierarchical transformation 3. Introduction MPEG, short for Moving Picture Experts Group, is one of the leading standardization organizations for multimedia processing. In 2017, the MPEG 3D Graphics Codec Group (3DG) released a Call for Requests (CFP) document to begin development of a point cloud encoding and decoding standard. The final standard will include two types of solutions. Video-based point cloud compression (V-PCC) is suitable for point sets with a relatively uniform point distribution. Geometry-based point cloud compression (G-PCC) is suitable for sparser distributions. Both V-PCC and G-PCC support encoding and decoding of single point clouds and sequences of point clouds.

[0050] A point cloud can contain geometric information and attribute information. Geometric information describes the geometric location of the data points. Attribute information records details of the data points, such as texture, normal vectors, and reflections.

[0051] 3.1 Regional Adaptive Hierarchical Transformation In G-PCC, one of the important point cloud attribute encoding and decoding tools is RAHT. It's a transformation that uses attributes associated with nodes in lower levels of an octree to predict the attributes of nodes in the next level. It assumes that the location of the points is given at both the encoder and decoder. RAHT scans the octree backward from the leaf nodes to the root node, recombining nodes into larger nodes at each step until the root node is reached. At each level of the octree, nodes are processed in Morton order. Instead of combining all eight nodes at once during each decomposition, RAHT performs this in three steps along each dimension (e.g., along z, then y, then x). If the octree has... If there are 3L levels, then RAHT needs 3L levels to traverse the tree backward.

[0052] For integers x, y, z, define the hierarchy. The node at is . Through the and This is obtained through combination, with combinations along the first dimension as an example. RAHT only processes occupied nodes. If one node in a pair is not occupied, the other node is promoted to the next level and not processed; that is, if the latter is the occupied node in the pair, then... Repeat this combination process until the root is reached. Note that the combination process generates lower-level nodes, which are the result of combining different numbers of voxels in this way. The nodes are combined to generate the nodes. The number of nodes is the weight of that node. .

[0053] In two nodes (e.g.) and In each combination, utilize their respective weights. and RAHT applies the following transformation: , in and as well as .

[0054] Note that the transformation matrix is ​​constantly changing to adapt to the weights, that is, to adapt to each... The actual number of leaf nodes represented. It is used to combine and form further nodes at lower levels. These are the actual high-pass coefficients generated by the transformations to be encoded and transmitted. Furthermore, the weights accumulate over the layers above. In the example above, .

[0055] In the final stage, the roots of the tree, the remaining two voxels and Transformed into the final two coefficients: , in .

[0056] 3.3 Upsampled Transform Domain Prediction in RAHT Transform domain prediction is introduced to improve the encoding and decoding efficiency of RAHT. It consists of two parts.

[0057] First, the RAHT tree traversal is changed from the previous ascending method to a descending one; that is, a tree is constructed by summing attributes and weights. Then, RAHT is performed from the root to the leaf for both the encoder and decoder. The transformation is also performed in octree node transformation units with 2×2×2 child nodes. Within a node, the encoder transformation order is from leaf to root.

[0058] Secondly, for each child node of the transform unit, the corresponding prediction child node is generated by upsampling the previous transform level. In practice, only child nodes containing at least one point will generate a corresponding prediction child node. The transform unit containing 2×2×2 prediction child nodes is transformed and subtracted from the transformed attributes on the encoder side.

[0059] Each child node of a transformation unit node is predicted by 7 parent level nodes, including 3 collinear parent level nearest neighbors, 3 coplanar parent level nearest neighbors, and 1 parent node. Coplanar nearest neighbors and collinear nearest neighbors are the nearest neighbors that share a face and an edge with the current transformation unit node, respectively. Figure 4 Seven parent nodes are shown for each child node of the transformation unit node. For example, node 410 (such as the current node) can be divided or segmented into multiple child nodes, such as child node 420. Node 410 can be referred to as the parent node of child node 420. Node 410 can have multiple neighboring nodes (also called parent neighboring nodes), such as neighboring node 430. As used herein, a neighboring node can be a node that shares at least one of a face, edge, or vertex with a particular node.

[0060] Attributes of each child node The prediction is based on the distance between it and its parent node as follows.

[0061] , It is a property of one of its parent nodes, and It depends on the distance weight. In G-PCC, .

[0062] For the AC coefficients, the residuals will be predicted through signal transmission.

[0063] For DC coefficients, the coefficients are inherited from previous levels, which means that DC coefficients are transmitted through the signal without prediction.

[0064] 3.4 Attribute Inter-Frame Prediction in RAHT We propose applying inter-frame prediction to the DC and AC coefficients in RAHT.

[0065] For layers with inter-frame prediction enabled, an octree decomposition is performed on the two frames. The decomposition is derived based on the geometric information of each frame.

[0066] For each node in the current octree decomposition, it can be matched with a node in the reference decomposition. The Morton value of the matched node should be the same as the Morton value of the current node. For the AC and DC coefficients of these nodes, reference coefficients are generated in the same manner as the current coefficients. The reference coefficients are used to predict the current coefficients.

[0067] For example, for the AC coefficients, the prediction residuals are transmitted via signal as follows: , .

[0068] if If it equals 0, then It is used for prediction in the original transform domain.

[0069] Each layer has a flag indicating whether inter-frame prediction is enabled. This flag is derived based on a rate-distortion optimization method.

[0070] 3.5 AC coefficient quantization offset A method is proposed to transmit a 3D quantization matrix via the signal, allowing the QP offset to be applied to each AC coefficient of the transformed 2×2×2 residual. Note that the DC coefficients are inherited, and therefore no residual DC component is transmitted to the decoder via the signal.

[0071] 4. Problem The existing design for inter-frame prediction of point cloud attributes in RAHT has the following problems.

[0072] In the current design, the flag used to indicate whether inter-frame prediction is enabled is determined for each layer. However, this layer-by-layer determination may not be optimal, given that the best choice for each node can vary.

[0073] In the current design, reference information comes from only one reference frame, and there can be at most one reference node per node. However, this reference information may be insufficient to provide efficient prediction.

[0074] In the current design, predictions are derived from intra-frame or inter-frame predictions. However, this prediction determination may not be optimal.

[0075] In the current design, the indicator used to indicate whether the AC coefficient quantization matrix is ​​transmitted via signaling is always transmitted via signaling. However, the AC coefficient quantization matrix is ​​only used in the RAHT encoding / decoding method.

[0076] 5. Detailed Solution 1) A prediction method for each node is proposed and transmitted via signal transmission.

[0077] a. In one example, the prediction method could be no prediction.

[0078] b. In one example, the prediction method could be intra-frame prediction.

[0079] c. In one example, the prediction method could be inter-frame prediction.

[0080] d. In one example, the prediction method could be to derive the predicted value using intra-frame prediction and inter-frame prediction.

[0081] i. In one example, the predicted value could be a weighted average of inter-frame and intra-frame predictions.

[0082] e. In one example, the prediction method for each node can be selected from a number of candidate prediction methods.

[0083] i. In one example, the candidate prediction method could be no prediction.

[0084] ii. In one example, the candidate prediction method could be intra-frame prediction.

[0085] iii. In one example, the candidate prediction method could be inter-frame prediction.

[0086] iv. In one example, the candidate prediction method could be a weighted average prediction.

[0087] f. In one example, the prediction method can be determined based on some eligibility criteria.

[0088] i. In one example, eligibility criteria can be based on the layer depth of a node.

[0089] ii. In one example, the eligibility criteria can be based on the geometric location of the node.

[0090] iii. In one example, eligibility criteria can be based on the node's attribute information.

[0091] iv. In one example, eligibility criteria can be based on a node's nearest neighbor information.

[0092] v. In one example, the eligibility criteria for a candidate prediction method can be based on the eligibility of other candidate prediction methods.

[0093] 1. In one example, the eligibility criteria for weighted average prediction can be that the eligibility criteria for both inter-frame prediction and intra-frame prediction are met.

[0094] vi. In one example, the above eligibility criteria can be combined to determine the eligibility of a prediction method.

[0095] g. In one example, the prediction method can be determined based on a rate-distortion optimization method.

[0096] i. In one example, the rate and / or distortion can be estimated.

[0097] ii. Alternatively, the rate and / or distortion can be calculated based on the reconstructed values.

[0098] h. In one example, the above determination methods can be combined to determine the prediction method.

[0099] i. In one example, the prediction method can be selected from the list of prediction method candidates based on the rate-distortion optimization method.

[0100] ii. In one example, the candidate list of prediction methods can be derived based on the eligibility of each prediction method.

[0101] 1. In one example, the candidate list of prediction methods can contain up to N prediction methods.

[0102] a. In one example, N can be predefined, for example, N = 3.

[0103] b. In one example, N can be transmitted to the decoder via a signal.

[0104] 2. In one example, a prediction method can be included in the list when the eligibility criteria for that method are met.

[0105] 3. In one example, a prediction method can be included in the list when its candidate priority is higher than that of a method not in the list.

[0106] a. In one example, the candidate priority of weighted average prediction is higher than no prediction / inter-frame prediction / intra-frame prediction; the candidate priority of no prediction is higher than inter-frame prediction and intra-frame prediction; the candidate priority of intra-frame prediction is higher than inter-frame prediction.

[0107] 4. In one example, the prediction method can be included in the list when all of the above conditions are met.

[0108] i. In one example, for each node, there could be an indicator to determine which prediction method to apply to that node.

[0109] i. In one example, the indicator may consist of at least one bit.

[0110] 1. In one example, the indicator can consist of up to N bits.

[0111] a. In one example, when there are more than two prediction methods in the candidate prediction method list, the indicator can consist of a maximum of two bits.

[0112] b. In one example, when there are fewer than two prediction methods in the candidate prediction method list, the indicator can consist of a maximum of one bit.

[0113] 2. In one example, the representation of N bits can be different in different situations.

[0114] a. In one example, the representation of N bits can be chosen based on the PC sample type of the current PC sample.

[0115] i. In one example, the PC sample type can be deduced based on whether and how inter-frame prediction is enabled on PC samples.

[0116] ii. In one example, the PC sample type can be deduced based on which reference GOF structure is used for the PC sample.

[0117] iii. In one example, the PC sample type may include I-strip / I-frame, P-strip / P-frame, and B-strip / B-frame.

[0118] b. In one example, the representation of N bits can be chosen based on how many prediction methods and which prediction methods are enabled for the current node.

[0119] 3. In one example, when there are three prediction methods (A, B, C) in the candidate list of prediction methods for the current node, the first bit can be used to indicate whether the prediction method is prediction method A.

[0120] a. When the prediction method is method A, the indicator is the first bit.

[0121] b. When the prediction method is not method A, the second bit can be used to indicate whether the prediction method is prediction method B. The indicator consists of two bits.

[0122] c. In one example, prediction method A could be inter-frame prediction.

[0123] d. Alternatively, prediction method A can be intra-frame prediction.

[0124] e. For alternative sites, forecasting method A can be a weighted average forecast.

[0125] 4. In one example, when there are prediction methods (A, B) in the candidate list of prediction methods for the current node, a bit can be used to indicate whether the prediction method is prediction method A.

[0126] a. Indicators are bits.

[0127] 5. In one example, when there are four prediction methods (A, B, C, D) in the candidate list of prediction methods for the current node, the first bit can be used to indicate whether the prediction method is prediction method A or prediction method B.

[0128] a. When the prediction method is method A or prediction method B, the second bit can be used to indicate whether the prediction method is prediction method A. The indicator consists of two bits.

[0129] b. When the prediction method is not method A or prediction method B, the second bit can be used to indicate whether the prediction method is prediction method C. The indicator consists of two bits.

[0130] c. In one example, prediction method A could be inter-frame prediction.

[0131] d. Alternatively, prediction method A can be intra-frame prediction.

[0132] e. For alternative sites, forecasting method A can be a weighted average forecast.

[0133] ii. In one example, the indicator can be transmitted to the decoder via a signal.

[0134] 1. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0135] 2. In one example, the indicator can be encoded and decoded in a predictive manner.

[0136] 3. In one example, the indicator can be transmitted via signaling using a context-based adaptive binary arithmetic encoding / decoding method.

[0137] a. In one example, the context can be inferred from multiple elements.

[0138] b. In one example, an element can be derived based on the primary prediction method of the current node's neighboring nodes.

[0139] i. In one example, a neighboring node can be an uncle node.

[0140] ii. In one example, a neighboring node can be a chapel node.

[0141] iii. In one example, the primary prediction method for neighboring nodes can be derived based on the prediction method for neighboring nodes.

[0142] iv. In one example, the primary prediction method for neighboring nodes could be inter-frame prediction, intra-frame prediction, no prediction, or no prediction found.

[0143] 1. When a primary prediction method is not found for a neighboring node, the primary prediction method is indicated as not found.

[0144] v. In one example, the primary prediction methods for neighboring nodes can be classified into N types.

[0145] 1. In one example, N can be 3.

[0146] a. In one example, one type could be inter-frame prediction.

[0147] b. In one example, one type could be intra-frame prediction.

[0148] c. In one example, a type can be unpredictable or not found.

[0149] vi. In one example, an element can be determined in different cases depending on which type of primary prediction method it is a neighboring node of.

[0150] 2) A prediction method for each node is proposed to be derived at the decoder.

[0151] a. In one example, the prediction method for each node can be selected from a number of candidate prediction methods.

[0152] i. In one example, the candidate prediction method could be no prediction.

[0153] ii. In one example, the candidate prediction method could be intra-frame prediction.

[0154] iii. In one example, the candidate prediction method could be inter-frame prediction.

[0155] iv. In one example, the candidate prediction method could be a weighted average prediction.

[0156] b. In one example, the prediction method can be derived based on some qualifying conditions.

[0157] i. In one example, eligibility criteria can be based on the layer depth of a node.

[0158] ii. In one example, the eligibility criteria can be based on the geometric location of the node.

[0159] iii. In one example, eligibility criteria can be based on the node's attribute information.

[0160] iv. In one example, eligibility criteria can be based on a node's nearest neighbor information.

[0161] v. In one example, the eligibility criteria for a candidate prediction method can be based on the eligibility of other candidate prediction methods.

[0162] 1. In one example, the eligibility criteria for weighted average prediction can be that the eligibility criteria for both inter-frame prediction and intra-frame prediction are met.

[0163] vi. In one example, the above eligibility criteria can be combined to determine the eligibility of a prediction method.

[0164] c. In one example, the prediction method can be derived based on the prediction priority of each candidate prediction method.

[0165] i. In one example, the candidate priority of weighted average prediction is higher than no prediction / inter-frame prediction / intra-frame prediction; the candidate priority of inter-frame prediction is higher than intra-frame prediction and no prediction; the candidate priority of intra-frame prediction is higher than no prediction.

[0166] d. In one example, the above determination methods can be combined to derive a prediction method for a single node.

[0167] i. In one example, the prediction method can be selected from the list of prediction method candidates based on the prediction priority of each candidate prediction method.

[0168] ii. In one example, the candidate list of prediction methods can be derived based on the eligibility of each prediction method.

[0169] 1. In one example, the candidate list of prediction methods can contain up to N prediction methods.

[0170] a. In one example, N can be predefined.

[0171] b. In one example, N can be transmitted to the decoder via a signal.

[0172] 2. In one example, a prediction method can be included in the list when the eligibility criteria for that method are met.

[0173] 3. In one example, a prediction method can be included in the list when its candidate priority is higher than that of a method not in the list.

[0174] a. In one example, the candidate priority of weighted average prediction is higher than no prediction / inter-frame prediction / intra-frame prediction; the candidate priority of no prediction is higher than inter-frame prediction and intra-frame prediction; the candidate priority of intra-frame prediction is higher than inter-frame prediction.

[0175] 4. In one example, the prediction method can be included in the list when all of the above conditions are met.

[0176] 3) A method combining the above methods to determine the prediction of each node is proposed.

[0177] a. A method is proposed to determine and transmit predictions for each node at some specific layer (i.e., layer group A).

[0178] i. In one example, the prediction method for a node in layer group A can be indicated by some indicators, and the indicators can be transmitted to the decoder via signals.

[0179] b. A method for predicting each node at some other specific layers (i.e., layer group B) is proposed.

[0180] i. In one example, the prediction method for nodes in layer group B can be derived at the decoder.

[0181] c. In one example, layer group A and layer group B can be derived from some parameters.

[0182] i. In one example, the parameters can be transmitted to the decoder via a signal.

[0183] 4) A prediction method for each region / level was proposed, which is determined and transmitted via signal.

[0184] a. In one example, the prediction method could be no prediction.

[0185] b. In one example, the prediction method could be intra-frame prediction.

[0186] c. In one example, the prediction method could be inter-frame prediction.

[0187] d. In one example, the prediction method could be to derive the predicted value using intra-frame prediction and inter-frame prediction.

[0188] i. In one example, the predicted value could be a weighted average of inter-frame and intra-frame predictions.

[0189] ii. In one example, the above prediction method could be a weighted average prediction.

[0190] e. In one example, the prediction method can be a combination of the methods described above.

[0191] i. In one example, if the predicted value of method A is not zero, then the predicted value can be the predicted value of method A; otherwise, the predicted value can be the predicted value of method B.

[0192] f. In one example, the prediction method can be applied to all nodes in the region / hierarchy.

[0193] g. In one example, the prediction method can be determined based on some eligibility criteria.

[0194] i. In one example, eligibility criteria can be based on the layer depth of the region / level.

[0195] 1. In one example, there may be at least one indicator to indicate the allowed layer depth(s) or region(s) in the qualification criteria.

[0196] a. In one example, the indicator can be transmitted to the decoder via a signal.

[0197] i. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0198] ii. In one example, the indicator can be encoded and decoded in a predictive manner.

[0199] ii. In one example, eligibility criteria can be based on the geometric location of a region / hierarchy.

[0200] iii. In one example, eligibility criteria can be based on region / hierarchy attribute information.

[0201] iv. In one example, eligibility criteria can be based on the nearest neighbor information of nodes in a region / hierarchy.

[0202] v. In one example, the eligibility criteria for a prediction method can be based on the eligibility criteria of other prediction methods.

[0203] 1. In one example, the eligibility criteria for weighted average prediction can be that the eligibility criteria for both inter-frame prediction and intra-frame prediction are met.

[0204] vi. In one example, the above eligibility criteria can be combined to determine the eligibility of a prediction method.

[0205] h. In one example, the prediction method can be determined based on a rate-distortion optimization method.

[0206] i. In one example, the rate and / or distortion can be estimated.

[0207] ii. Alternatively, the rate and / or distortion can be calculated based on the reconstructed values.

[0208] i. In one example, the above determination methods can be combined to determine the prediction method.

[0209] j. In one example, for each region / level, there could be an indicator to specify which prediction method should be applied to the nodes in that region / level.

[0210] i. In one example, the indicator can be transmitted to the decoder via a signal.

[0211] 1. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0212] 2. In one example, the indicator can be encoded and decoded in a predictive manner.

[0213] 5) A prediction method based on signal transmission parameters was proposed.

[0214] a. In one example, there may be parameters to indicate the eligibility criteria for a prediction method.

[0215] b. In one example, there may be some parameters to derive the eligibility criteria for a prediction method.

[0216] c. In one example, there may be parameters to indicate the area / level / layer where the qualification criteria need to be checked.

[0217] i. In one example, for a prediction method, only certain layers may need to check the eligibility criteria; other layers may not need to apply the prediction method.

[0218] ii. Alternatively, for a prediction method, only certain specific layers may require checking eligibility criteria; other layers may be eligible for the prediction method.

[0219] iii. In one example, there may be some parameters to derive a specific layer.

[0220] 1. In one example, some parameters can be transmitted via signaling to derive a specific layer for weighted average prediction.

[0221] a. In one example, there could be an indicator to indicate the smallest layer within a specific layer. .

[0222] i. In one example, the indicator could be depth, or The depth difference between the root node and the root node, or The depth difference between the bottom layer and the bottom layer.

[0223] ii. In one example, the indicator could be With another specific layer The depth difference between them.

[0224] 1. In one example, It can be derived from other parameters.

[0225] 2. In one example, the indicator could be... .

[0226] b. In one example, there could be an indicator to indicate the maximum layer within a specific layer. .

[0227] i. In one example, the indicator could be depth, or The depth difference between the root node and the root node, or The depth difference between the bottom layer and the bottom layer.

[0228] ii. In one example, the indicator could be With another specific layer The depth difference between them.

[0229] 1. In one example, It can be derived from other parameters.

[0230] 2. In one example, the indicator could be... .

[0231] d. In one example, the above parameters can be transmitted to the decoder via a signal.

[0232] i. In one example, the parameters can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0233] ii. In one example, parameters can be encoded and decoded in a predictive manner.

[0234] 6) A method for generating prediction results based on reference information from multiple reference frames is proposed.

[0235] a. In one example, for each node, there can be multiple reference nodes from multiple reference frames.

[0236] b. In one example, there may be one inter-frame prediction value for each reference node.

[0237] c. In one example, for each reference node, there can be an indicator to indicate the reference node.

[0238] d. In one example, for the current node, the prediction can be derived from the intra-frame prediction and multiple inter-frame predictions.

[0239] e. In one example, the predicted value can be selected from the predicted value candidates.

[0240] i. In one example, a prediction candidate can be an intra-frame prediction.

[0241] ii. In one example, if there is a reference node, a prediction candidate can be an inter-frame prediction of the reference node.

[0242] iii. In one example, if there are multiple reference nodes, a prediction candidate can be a weighted average of intra-frame predictions and multiple inter-frame predictions.

[0243] iv. In one example, a candidate prediction can be a weighted average of an intra-frame prediction and an inter-frame prediction.

[0244] v. In one example, if there are multiple reference nodes, a candidate prediction can be a weighted average of multiple inter-frame predictions.

[0245] f. In one example, the choice can be determined based on the rate-distortion optimization method.

[0246] i. In one example, the rate and / or distortion can be estimated.

[0247] ii. Alternatively, the rate and / or distortion can be calculated based on the reconstructed values.

[0248] g. In one example, there could be an indicator to indicate which predicted value candidate to select.

[0249] i. In one example, the indicator can be transmitted to the decoder via a signal.

[0250] 1. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0251] 2. In one example, the indicator can be encoded and decoded in a predictive manner.

[0252] 7) The weight values ​​used in the above weighted average calculation are proposed to be fixed / derived / transmitted via signal transmission.

[0253] a. In one example, the weights can be fixed at the decoder.

[0254] i. In one example, the weights can be different for each layer.

[0255] ii. In one example, the weights can be different for each node.

[0256] iii. In one example, the weights can be consistent across all nodes.

[0257] b. In one example, the weights can be derived at the decoder.

[0258] i. In one example, the weights can be derived for each layer.

[0259] 1. In one example, weights can be derived based on the candidate selection results of the predicted values ​​of nodes in (multiple) previous layers.

[0260] 2. In one example, the weights can be derived based on the depth level.

[0261] ii. In one example, the weights can be derived for each node.

[0262] 1. In one example, the weights can be derived based on the predicted candidate selection results of some previously encoded nodes (such as encoded neighboring nodes or parent nodes).

[0263] 2. In one example, the weights can be derived based on the predicted candidate selection results of some previously encoded nodes (such as parent nodes, uncle nodes, and paternal nodes).

[0264] a. In one example, an uncle node is a neighboring node of a parent node in the parent hierarchy; a paternal uncle node is a child node of an uncle node.

[0265] b. In one example, the weights for intra / inter-frame prediction can be derived based on the number of uncle nodes that select intra / inter / no prediction results as the final prediction value, the number of aunt nodes that select intra / inter / no prediction results as the final prediction value, and whether the parent node selects intra / inter / no prediction results as the final prediction value.

[0266] c. In one example, the amount of intra / inter-frame prediction can be calculated to derive the weights for intra / inter-frame prediction.

[0267] i. In one example, the amount of intra / inter-frame prediction can be calculated as / , / , / Linear combinations, such as:

[0268]

[0269] in / It is the number of uncle nodes that select intra / inter-frame prediction results as the final prediction value; / It is the number of nodes that select intra / inter-frame prediction results as the final prediction value; / This is used to indicate whether the parent node selects the intra / inter-frame prediction result as the final prediction value; if the parent node selects the intra-frame prediction result as the final prediction value, then Set to 1, It is set to 0; if the parent node selects the inter-frame prediction result as the final prediction value, then Set to 0, It is set to 1. , and It is an impact factor.

[0270] 1. In one example, the impact factor can be predefined.

[0271] a. In one example, one or more impact factors can be set to positive values, for example, , and They were set to 1, 3, and 6 respectively.

[0272] b. In one example, one or more impact factors can be set to 0.

[0273] c. In one example, one or more impact factors can be set to negative values.

[0274] 2. In one example, the influence factor can be transmitted to the decoder via a signal.

[0275] ii. In one example, the amount of intra / inter-frame prediction can be calculated as / / , / / , / / Linear combinations, such as:

[0276] , in / / It is the number of uncle nodes that are selected as the final prediction value based on intra-frame / inter-frame / no prediction results; / / It is the number of nodes that select intra-frame / inter-frame / no prediction results as the final prediction value; / / This is used to indicate which prediction result is selected as the final prediction value for the parent node; if the parent node selects the intra-frame prediction result as the final prediction value, then Set to 1, and It is set to 0; if the parent node selects the inter-frame prediction result as the final prediction value, then Set to 1, and It is set to 0; if the parent node selects no prediction result as the final prediction value, then Set to 1, and Set to 0; , , , , and It is an impact factor.

[0277] 1. In one example, the impact factor can be predefined.

[0278] a. In one example, one or more impact factors can be set to positive values, for example, , , , , and They were set to 2, 6, 12, 1, 3, and 6 respectively.

[0279] b. In one example, one or more impact factors can be set to 0.

[0280] c. In one example, one or more impact factors can be set to negative values.

[0281] 2. In one example, the influence factor can be transmitted to the decoder via a signal.

[0282] iii. In one example, the prediction weights for inter-frame / intra-frame prediction can be a transformation of the quantities for inter-frame / intra-frame prediction.

[0283] 1. In one example, a transformation may include bit shifting operations, fixed-point implementations of division operations, and other operations.

[0284] c. In one example, weights can be transmitted to the decoder via signals.

[0285] i. In one example, weights can be transmitted to the decoder via signals for each layer.

[0286] 1. In one example, the weights can be encoded and decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0287] 2. In one example, weights can be encoded and decoded in a predictive manner.

[0288] d. In one example, weights can be selected from a predefined list of weight candidates.

[0289] i. In one example, weights can be selected using a rate-distortion optimization / rate-distortion optimization method.

[0290] ii. In one example, there may be at least one indicator to indicate the selection result.

[0291] 1. In one example, the indicator can be transmitted to the decoder via a signal.

[0292] a. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0293] b. In one example, the indicator can be encoded and decoded in a predictive manner.

[0294] iii. In one example, selection can be performed on each layer.

[0295] iv. In one example, selection can be performed on a per-node basis.

[0296] 8) A weighted average of intra-frame prediction and inter-frame prediction was proposed to determine whether signal transmission uses intra-frame prediction and inter-frame prediction.

[0297] a. In one example, there could be an indicator to indicate whether weighted averaging of intra-frame and inter-frame predictions can be enabled for the entire sequence.

[0298] b. In one example, there could be an indicator to indicate whether a weighted average of intra-frame prediction and inter-frame prediction is used for a PC sample.

[0299] c. In one example, there could be an indicator to indicate whether a weighted average of intra-frame prediction and inter-frame prediction is used for a layer / region.

[0300] d. In one example, all or some of the above indicators may be transmitted to the decoder via signaling.

[0301] i. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0302] ii. In one example, the indicator can be encoded and decoded in a predictive manner.

[0303] 9) A method for transmitting the parameters of the RAHT coefficient quantization matrix via signal transmission when RAHT attribute encoding / decoding is enabled is proposed.

[0304] a. In one example, the RAHT coefficient can be the AC coefficient.

[0305] b. In one example, the RAHT coefficient can be the DC coefficient.

[0306] c. In one example, the RAHT coefficient can be either the AC coefficient or the DC coefficient.

[0307] d. In one example, there may be a quantization matrix for the AC coefficients.

[0308] e. In one example, there could be a quantization matrix for the DC coefficients.

[0309] f. In one example, there may be at least one indicator to indicate whether the RAHT coefficient quantization matrix is ​​transmitted via signaling.

[0310] i. In one example, the indicator can only be signaled to the decoder if RAHT attribute encoding / decoding is enabled.

[0311] ii. In one example, the indicator can be encoded or decoded using fixed-length encoding / decoding, unary encoding / decoding, rounding unary encoding / decoding, etc.

[0312] iii. In one example, the indicator can be encoded and decoded in a predictive manner.

[0313] 10) Whether and / or how the methods disclosed above can be applied to transmit signals from the encoder to the decoder in a bitstream / frame / slice / segment / octree / etc.

[0314] 11) Whether and / or how the methods disclosed above are applied may depend on the encoded / decoded information, such as dimensions, color format, color components, slice / image type.

[0315] 6. Examples Figure 5Example of an encoding / decoding stream 500 for improved inter-frame prediction of coefficients when there are two reference frames is depicted. As shown, multiple reference frames (such as reference frame 1 and reference frame 2) can be used to encode / decode the current frame. For example, an octree scan is performed backward on the current frame, reference frame 1, and / or reference frame 2. Coefficients for each octree depth level can be derived for the current frame. If a node is not in the first N depth levels (N is a positive integer), intra-frame predictions can be used to predict coefficients. Reference node 1 and reference node 2 can be searched. If reference node 1 or reference node 2 is found, reference coefficients for reference node 1 or reference node 2 are derived. If reference node 1 or reference node 2 is not found, inter-frame prediction 1 or inter-frame prediction 2 can be used to predict coefficients for each octree depth level. Alternatively, the RDO method can be used to select predictions to predict coefficients. The residuals of the transformed AC coefficients can be calculated and indicated in the bitstream.

[0316] Embodiments of this disclosure relate to encoding and decoding for point cloud encoding and decoding. As used herein, the term "point cloud sequence" may refer to a sequence of one or more point clouds. The term "frame" may refer to a point cloud in a point cloud sequence. The term "point cloud" may refer to a frame in a point cloud sequence. The term "node" represents a spatial segmentation of the current frame.

[0317] Figure 6 A flowchart of a method 600 for point cloud encoding / decoding according to an embodiment of the present disclosure is shown. Method 600 is implemented during the conversion between the current frame of a point cloud sequence and a bitstream of the point cloud sequence.

[0318] At box 610, for the conversion between the current frame of the point cloud sequence and the bitstream of the point cloud sequence, a prediction mode for the current node of the current frame is determined based on an indication in the bitstream, wherein the indication is encoded or decoded by context-based adaptive binary arithmetic encoding / decoding. For example, the determined prediction mode may be a combination of inter-frame prediction and intra-frame prediction, or it may be any other suitable prediction mode.

[0319] At box 620, the prediction for the current node is determined based on the prediction pattern. At box 630, a transformation is performed based on the prediction.

[0320] Method 600 enables the determination of a prediction pattern for the current node based on instructions from context-based adaptive binary arithmetic encoding / decoding. By using the determined prediction pattern, more accurate predictions can be obtained, thus improving the encoding / decoding efficiency of point cloud encoding / decoding.

[0321] In some embodiments, the indicated context can be derived from at least one element. For example, the indicated context can be derived from multiple elements. An element of at least one element can be derived based on the dominant prediction patterns of at least one of the current node's neighboring nodes. For example, an element can be derived from the dominant prediction patterns of multiple neighboring nodes. The dominant prediction pattern can be the prediction pattern used by the majority of the multiple neighboring nodes.

[0322] In some embodiments, at least one neighboring node can be an uncle node. An uncle node can be a neighboring node of the current node's parent node in the parent hierarchy. In some other embodiments, at least one neighboring node can be a hall code. A hall code can be a child node of an uncle node.

[0323] In some embodiments, the primary prediction mode of at least one neighboring node is derived based on at least one prediction mode of at least one neighboring node. For example, the primary prediction mode can be determined from multiple prediction modes of multiple neighboring nodes. The primary prediction mode can be an inter-frame prediction mode, an intra-frame prediction mode, or a no-prediction mode.

[0324] In some other embodiments, the primary prediction pattern is not found. If the primary prediction pattern is not found, it is indicated as not found, or as no prediction pattern.

[0325] In some embodiments, the primary prediction method for neighboring nodes can be categorized into N types, where N is a positive integer. For example, N can be 3 or any other suitable integer.

[0326] In some embodiments, the type of primary prediction mode can be an inter-frame prediction mode type, an intra-frame prediction mode type, or a no-prediction mode type. In some other embodiments, the type of primary prediction can be unfound.

[0327] In some embodiments, at least one element includes multiple elements determined under multiple conditions. The condition of an element may be based on the type of the dominant prediction pattern of at least one neighboring node of the current node.

[0328] In some embodiments, the prediction mode of the current node can be an inter-frame prediction mode, an intra-frame prediction mode, or a mode combining inter-frame and intra-frame prediction. In some other embodiments, the prediction mode of the current node can be a no-prediction mode.

[0329] In some embodiments, an indicator indicating whether method 600 is applied may be included in the bitstream. For example, the indicator may be included from the encoder to the decoder in a bitstream, frame, slice, strip, or octree.

[0330] Alternatively or additionally, in some embodiments, whether and / or how to apply method 600 can be determined based on the encoded / decoded information. For example, the encoded / decoded information may include at least one of the following: dimension, color format, color components, stripe type, or image type. By using method 600, the encoding / decoding effectiveness and efficiency of point cloud encoding / decoding can be improved.

[0331] In some embodiments, the conversion includes encoding the current frame into a bitstream. Alternatively or additionally, in some embodiments, the conversion includes decoding the current frame from the bitstream.

[0332] According to another embodiment of this disclosure, a non-transitory computer-readable recording medium is provided. This non-transitory computer-readable recording medium stores a bitstream of a point cloud sequence generated by a method performed by means of a point cloud encoding / decoding apparatus. In this method, a prediction mode for the current node of the current frame of the point cloud sequence is determined based on an indication in the bitstream, wherein the indication is encoded / decoded by context-based adaptive binary arithmetic encoding / decoding. A prediction for the current node is determined based on the prediction mode. A bitstream is generated based on the prediction.

[0333] According to further embodiments of this disclosure, a method for storing a bitstream of a point cloud sequence is provided. In this method, a prediction mode for the current node of the current frame of the point cloud sequence is determined based on an indication in the bitstream, wherein the indication is encoded / decoded using context-based adaptive binary arithmetic encoding / decoding. A prediction for the current node is determined based on the prediction mode. A bitstream is generated based on the prediction. The bitstream is stored in a non-transitory computer-readable recording medium.

[0334] The embodiments of this disclosure can be described according to the following entries, and their features can be combined in any reasonable manner.

[0335] Item 1. A method for point cloud encoding / decoding, comprising: a conversion between a current frame of a point cloud sequence and a bitstream of the point cloud sequence; determining a prediction mode for a current node of the current frame based on an indication in the bitstream, wherein the indication is encoded / decoded by context-based adaptive binary arithmetic encoding / decoding; determining a prediction for the current node based on the prediction mode; and performing the conversion based on the prediction.

[0336] Item 2. According to the method described in Item 1, the context of the indication is derived from at least one element.

[0337] Item 3. According to the method described in Item 2, wherein an element of the at least one element is derived based on the main prediction pattern of at least one neighboring node of the current node.

[0338] Item 4. The method according to Item 3, wherein the at least one neighboring node includes at least one of the following: an uncle node, which is a neighboring node of the parent node of the current node in the parent hierarchy, or a hall code, which is a child node of the uncle node.

[0339] Item 5. The method according to Item 3 or 4, wherein the primary prediction mode of the at least one neighboring node is derived based on at least one prediction mode of the at least one neighboring node.

[0340] Item 6. The method according to any one of Items 3-5, wherein the primary prediction mode includes at least one of the following: inter-frame prediction mode, intra-frame prediction mode, or no prediction mode, or wherein the primary prediction mode is not found.

[0341] Item 7. The method according to Item 6, wherein the primary prediction pattern is indicated as not found in response to the primary prediction pattern not being found.

[0342] Item 8. The method according to any one of items 3-7, wherein the type of the primary prediction mode is one of N types, where N is a positive integer.

[0343] Item 9. The method according to Item 8, wherein the type of the primary prediction mode is at least one of the following: the type of inter-frame prediction mode, the type of intra-frame prediction mode, or the type of no prediction mode or not found.

[0344] Item 10. The method according to any one of Items 2-9, wherein the at least one element comprises a plurality of elements determined in a plurality of cases, the cases of which are based on the type of the primary prediction pattern of at least one neighboring node of the current node.

[0345] Item 11. The method according to any one of Items 1-10, wherein the prediction mode of the current node includes at least one of the following: inter-frame prediction mode, intra-frame prediction mode, a mode combining inter-frame prediction and intra-frame prediction, or no prediction mode.

[0346] Item 12. The method according to any one of items 1-11, wherein the conversion includes encoding the current frame into a bitstream.

[0347] Item 13. The method according to any one of items 1-11, wherein the conversion includes decoding the current frame from the bitstream.

[0348] Item 14. An apparatus for point cloud encoding and decoding, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of items 1-13.

[0349] Item 15. A non-transitory computer-readable storage medium storing instructions that cause a processor to execute the method according to any one of items 1-13.

[0350] Item 16. A non-transitory computer-readable recording medium storing a bitstream of a point cloud sequence generated by a method performed by means for point cloud encoding / decoding, wherein the method includes: determining a prediction mode for a current node of a current frame of the point cloud sequence based on an indication in the bitstream, wherein the indication is encoded / decoded by context-based adaptive binary arithmetic encoding / decoding; determining a prediction of the current node based on the prediction mode; and generating the bitstream based on the prediction.

[0351] Item 17. A method for storing a bitstream of a point cloud sequence, comprising: determining a prediction mode for a current node of a current frame of the point cloud sequence based on an indication in the bitstream, wherein the indication is encoded or decoded by context-based adaptive binary arithmetic encoding / decoding; determining a prediction for the current node based on the prediction mode; generating the bitstream based on the prediction; and storing the bitstream in a non-transitory computer-readable recording medium.

[0352] Example device Figure 7 A block diagram of a computing device 700 in which various embodiments of the present disclosure may be implemented is shown. The computing device 700 may be implemented as a source device 110 (or GPCC encoder 116 or 200) or a destination device 120 (or GPCC decoder 126 or 300), or may be included in a source device 110 (or GPCC encoder 116 or 200) or a destination device 120 (or GPCC decoder 126 or 300).

[0353] It should be understood that, Figure 7 The computing device 700 shown is for illustrative purposes only and is not intended to imply any limitation on the functionality and scope of the embodiments of this disclosure.

[0354] like Figure 7 As shown, the computing device 700 includes a general-purpose computing device 700. The computing device 700 may include at least one or more processors or processing units 710, memory 720, storage units 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760.

[0355] In some embodiments, the computing device 700 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, and includes accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 700 can support any type of interface to the user (such as "wearable" circuitry devices, etc.).

[0356] The processing unit 710 can be a physical processor or a virtual processor, and can perform various processes based on programs stored in the memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of the computing device 700. The processing unit 710 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0357] Computing device 700 typically includes various computer storage media. Such media can be any media accessible by computing device 700, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 730 can be any removable or non-removable media and can include machine-readable media, such as memory, flash drives, disks, or other media that can be used to store information and / or data and can be accessed within computing device 700.

[0358] The computing device 700 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Although in Figure 7 Not shown, but may provide disk drives for reading from and / or writing to removable non-volatile disks, and optical disc drives for reading from and / or writing to removable non-volatile optical discs. In this case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0359] The communication unit 740 communicates with another computing device via a communication medium. Furthermore, the functionality of the components in the computing device 700 can be implemented by a single computing cluster or by multiple computing machines communicating via communication connections. Therefore, the computing device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0360] Input device 750 can be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 760 can be one or more of various output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 740, computing device 700 can also communicate with one or more external devices (not shown), such as storage devices and display devices. Computing device 700 can also communicate with one or more devices that enable a user to interact with computing device 700, or any device that enables computing device 700 to communicate with one or more other computing devices (e.g., network card, modem, etc.), if needed. This communication can be performed via an input / output (I / O) interface (not shown).

[0361] In some embodiments, some or all components of computing device 700 may not be integrated into a single device, but may be deployed in a cloud computing architecture. In a cloud computing architecture, components may be provided remotely and may work together to achieve the functionality described herein. In some embodiments, cloud computing provides computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing is provided via a wide area network (WAN) such as the Internet using suitable protocols. For example, a cloud computing provider provides applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at a remote location. Computing resources in a cloud computing environment may be consolidated or distributed at locations in remote data centers. Cloud computing infrastructure may be provided through shared data centers, although they may appear as a single access point to a user. Therefore, cloud computing architectures can be used to provide the components and functionality described herein from service providers at remote locations. Alternatively, they may be provided from traditional servers or installed directly or otherwise on client devices.

[0362] The computing device 700 can be used to implement point cloud encoding / decoding in embodiments of this disclosure. The memory 720 may include one or more point cloud encoding / decoding modules 725 having one or more program instructions. These modules can be accessed and executed by the processing unit 710 to perform the functions of the various embodiments described herein.

[0363] In an example embodiment of point cloud encoding, input device 750 may receive point cloud data as input 770 to be encoded. The point cloud data may be processed, for example, by point cloud encoding / decoding module 725 to generate an encoded bitstream. The encoded bitstream may be provided as output 780 via output device 760.

[0364] In an example embodiment of point cloud decoding, input device 750 may receive an encoded bitstream as input 770. The encoded bitstream may be processed, for example, by a point cloud encoding / decoding module 725 to generate decoded point cloud data. The decoded point cloud data may be provided as output 780 via output device 760.

[0365] While this disclosure has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application as defined by the appended claims. These changes are intended to be covered by the scope of this application. Therefore, the foregoing description of embodiments of this application is not intended to be limiting.

Claims

1. A method for point cloud encoding and decoding, comprising: For the conversion between the current frame of a point cloud sequence and the bitstream of the point cloud sequence, a prediction mode for the current node of the current frame is determined based on an indication in the bitstream, wherein the indication is encoded or decoded by context-based adaptive binary arithmetic encoding and decoding. The prediction for the current node is determined based on the prediction model; as well as The transformation is performed based on the prediction.

2. The method of claim 1, wherein the context of the indication is derived from at least one element.

3. The method of claim 2, wherein the element of the at least one element is derived based on the main prediction pattern of at least one neighboring node of the current node.

4. The method of claim 3, wherein the at least one neighboring node comprises at least one of the following: Uncle node, where the uncle node is the neighboring node of the current node's parent node in the parent hierarchy, or The ancestor is a child node of the uncle node.

5. The method according to claim 3 or 4, wherein the primary prediction pattern of the at least one neighboring node is derived based on at least one prediction pattern of the at least one neighboring node.

6. The method according to any one of claims 3-5, wherein the primary prediction mode comprises at least one of the following: inter-frame prediction mode, intra-frame prediction mode, or no prediction mode, or The primary prediction pattern was not found.

7. The method of claim 6, wherein the primary prediction pattern is indicated as not found in response to the primary prediction pattern not being found.

8. The method according to any one of claims 3-7, wherein the type of the primary prediction mode is one of N types, where N is a positive integer.

9. The method of claim 8, wherein the type of the primary prediction mode is at least one of the following: Types of inter-frame prediction modes Type of intra-prediction mode, or Types with no predictive patterns or types not found.

10. The method according to any one of claims 2-9, wherein the at least one element comprises a plurality of elements determined in a plurality of cases, the cases of which are based on the type of the primary prediction pattern of at least one neighboring node of the current node.

11. The method according to any one of claims 1-10, wherein the prediction mode of the current node comprises at least one of the following: Inter-frame prediction mode, Intra-frame prediction mode, A mode that combines inter-frame prediction and intra-frame prediction, or No prediction mode.

12. The method according to any one of claims 1-11, wherein the conversion comprises encoding the current frame into the bitstream.

13. The method according to any one of claims 1-11, wherein the conversion comprises decoding the current frame from the bitstream.

14. An apparatus for point cloud encoding and decoding, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1-13.

15. A non-transitory computer-readable storage medium storing instructions that cause a processor to execute the method according to any one of claims 1-13.

16. A non-transitory computer-readable recording medium storing a bitstream of a point cloud sequence generated by a method performed by means of a point cloud encoding / decoding apparatus, wherein the method comprises: The prediction mode for the current node of the current frame of the point cloud sequence is determined based on an indication in the bitstream, wherein the indication is encoded or decoded by context-based adaptive binary arithmetic encoding and decoding. The prediction for the current node is determined based on the prediction model; as well as The bitstream is generated based on the prediction.

17. A method for storing a bitstream of a point cloud sequence, comprising: The prediction mode for the current node of the current frame of the point cloud sequence is determined based on an indication in the bitstream, wherein the indication is encoded or decoded by context-based adaptive binary arithmetic encoding and decoding. The prediction for the current node is determined based on the prediction model; The bitstream is generated based on the prediction; as well as The bitstream is stored in a non-transitory computer-readable recording medium.

Images