Improvements for quantization of point cloud attribute transform domain coefficients

Abstract

A method of processing video data is disclosed. The method includes: determining quantization based on a region adaptive hierarchical transform (RAHT) weight; and performing a conversion between the visual media data and the bitstream based on the quantization.

Description

[0001] Cross - reference to related applications

[0002] This patent application claims the benefit of International Patent Application No. PCT / CN2022 / 143472, filed on December 29, 2022, which is incorporated herein by reference. Technical field

[0003] This patent document relates to generating, storing, and using digital audio - visual media information in a file format. Background art

[0004] Digital video occupies the largest bandwidth used on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video may continue to grow. Summary of the invention

[0005] A first aspect relates to a method for processing media data, including: changing the quantization step size of region - adaptive hierarchical transform (RAHT) coefficients in a point - cloud video unit; and performing a conversion between the media data and a media data file based on the quantization step size.

[0006] Optionally, in another embodiment of any of the above aspects, the point - cloud video unit is one of a video sequence, a frame, a cube, and a block.

[0007] Optionally, in another embodiment of any of the above aspects, the quantization step size depends on the weight of the RAHT coefficient.

[0008] Optionally, in another embodiment of any of the above aspects, different quantization step sizes are used for RAHT coefficients with different weights.

[0009] Optionally, in another embodiment of any of the above aspects, one or more indicators are used to indicate the different quantization step sizes.

[0010] Optionally, in another embodiment of any of the above aspects, the one or more indicators are included in the bitstream.

[0011] Optionally, in another embodiment of any of the above aspects, the different weights are divided into different groups, and different quantization offsets are transmitted for the different groups.

[0012] Optionally, in another embodiment of any of the above aspects, the different weights are divided into different groups, and different quantization offsets are transmitted for a subset of the different groups.

[0013] Optionally, in another implementation of any of the above aspects, the quantization step size of the RAHT coefficient is determined based on a predefined function.

[0014] Optionally, in another implementation of any of the above aspects, the predefined function includes where Δ is the quantization step size, K is a constant, and w is the weight of the RAHT coefficient.

[0015] Optionally, in another implementation of any of the above aspects, the value of K is included in the bitstream.

[0016] Optionally, in another implementation of any of the above aspects, K varies based on the level of the point cloud video unit, and the level of the point cloud video unit is a slice or a picture.

[0017] Optionally, in another implementation of any of the above aspects, the predefined function includes where Δ is the quantization step size, K is a constant, and w is the weight of the RAHT coefficient.

[0018] Optionally, in another implementation of any of the above aspects, the predefined function is for different weights of the RAHT coefficients within a specific range, or for different weights of the RAHT coefficients with specific values.

[0019] Optionally, in another implementation of any of the above aspects, it further includes: restricting the quantization step size between a minimum quantization step size and a maximum quantization step size.

[0020] Optionally, in another implementation of any of the above aspects, it further includes: replacing the octree layer-based adaptive quantization with weight-based quantization.

[0021] Optionally, in another implementation of any of the above aspects, the weight-based quantization is one of multiple different types of quantization, and additional information can be included in the bitstream or derived by the decoder to indicate which quantization among the multiple different types of quantization is to be used.

[0022] Optionally, in another implementation of any of the above aspects, the quantization step size is based on both the layer of the RAHT coefficient and the weight of the RAHT coefficient.

[0023] Optionally, in another implementation of any of the above aspects, it further includes: determining a quantization parameter (QP) offset based on the layer of the RAHT coefficient, and modifying the QP offset based on the weight of the RAHT coefficient.

[0024] Optionally, in another implementation of any of the above aspects, it further includes: changing the quantization step based on a function, where the function includes both the layer of the RAHT coefficients and the weight of the RAHT coefficients.

[0025] Optionally, in another implementation of any of the above aspects, it further includes: applying different quantization methods to different attribute channels.

[0026] Optionally, in another implementation of any of the above aspects, layer-based adaptive quantization is applied to the luminance channel, and quantization depending on the weight of the RAHT coefficients is applied to the chrominance channel.

[0027] Optionally, in another implementation of any of the above aspects, different functions based on the weight of the RAHT coefficients are applied to the different attribute channels.

[0028] Optionally, in another implementation of any of the above aspects, it further includes: dividing the different attribute channels into different subgroups for the different attribute channels, where the offsets corresponding to the different subgroups are different for the different attribute channels.

[0029] Optionally, in another implementation of any of the above aspects, it further includes: dividing the point cloud video unit into different regions, and applying different quantization methods to the different regions.

[0030] Optionally, in another implementation of any of the above aspects, it further includes: changing the quantization step of the RAHT coefficients based on a function, where the function is based on each of the region, layer, and weight of the RAHT coefficients.

[0031] Optionally, in another implementation of any of the above aspects, it further includes: applying layer-based quantization to some regions of the point cloud video unit, and applying weight-based quantization to other regions of the point cloud video unit.

[0032] Optionally, in another implementation of any of the above aspects, it further includes: determining a region-based quantization parameter (QP) offset, and modifying the region-based QP offset based on the weight of the RAHT coefficients.

[0033] Optionally, in another implementation of any of the above aspects, it further includes: dividing the different regions into different subgroups for the different regions, where the offsets corresponding to the different subgroups are different for the different regions.

[0034] Optionally, in another implementation of any of the above aspects, it further includes: selectively enabling and disabling the method for a group of voxels.

[0035] Optionally, in another implementation of any of the above aspects, the group of voxels is a group of 2×2×2 voxels.

[0036] Optionally, in another implementation of any of the above aspects, it further includes: applying different quantization to different transform coefficients after the 2×2×2 transform residual block has been predicted by upsampling.

[0037] Optionally, in another implementation of any of the above aspects, it further includes: deriving the quantization step size using a 2×2×2 quantization parameter (QP) offset matrix.

[0038] Optionally, in another implementation of any of the above aspects, the 2×2×2 QP offset matrix is specified at the sequence level, slice level, or picture level of the bitstream.

[0039] Optionally, in another implementation of any of the above aspects, the 2×2×2 QP offset matrix is based on an octree.

[0040] Optionally, in another implementation of any of the above aspects, it further includes: restricting the use of the 2×2×2 QP offset matrix to partial layers in the octree.

[0041] Optionally, in another implementation of any of the above aspects, it further includes: using different three-dimensional (3D) quantization parameter (QP) matrices for different layers in the octree.

[0042] Optionally, in another implementation of any of the above aspects, the 2×2×2 QP offset matrix is based on the weights of the RAHT coefficients.

[0043] Optionally, in another implementation of any of the above aspects, it further includes: restricting the use of the 2×2×2 QP offset matrix to partial subbands of the RAHT coefficients, where each of the subbands represents a group of coefficients with the same weight of the RAHT coefficients.

[0044] Optionally, in another implementation of any of the above aspects, it further includes: using different three-dimensional (3D) quantization parameter (QP) matrices for different subbands of the RAHT coefficients.

[0045] Optionally, in another embodiment of any of the above aspects, the 2×2×2 QP offset matrix is based on an attribute channel.

[0046] Optionally, in another embodiment of any of the above aspects, further comprising: using the 2×2×2 QP offset matrix for a subset of the different attribute channels.

[0047] Optionally, in another embodiment of any of the above aspects, further comprising: using the 2×2×2 QP offset matrix for the different attribute channels.

[0048] Optionally, in another embodiment of any of the above aspects, the 2×2×2 QP offset matrix is based on different regions.

[0049] Optionally, in another embodiment of any of the above aspects, further comprising: using the 2×2×2 QP offset matrix for a subset of the different regions.

[0050] Optionally, in another embodiment of any of the above aspects, further comprising: using the 2×2×2 QP offset matrix for the different regions.

[0051] Optionally, in another embodiment of any of the above aspects, further comprising: modifying the quantization of a subset of the 2×2×2 transform coefficients, rather than the quantization of all 2×2×2 transform coefficients, wherein the 2×2×2 transform coefficients include eight transform coefficients.

[0052] Optionally, in another embodiment of any of the above aspects, further comprising: applying coarser quantization to high-frequency coefficients without modifying low-frequency coefficients.

[0053] Optionally, in another embodiment of any of the above aspects, modifying the quantization of a subset of the 2×2×2 transform coefficients based on one or more of an attribute channel, the weight of the RAHT coefficients, and a layer of an octree.

[0054] Optionally, in another embodiment of any of the above aspects, further comprising: partitioning the RAHT coefficients into different subgroups, and applying different quantization parameter (QP) offsets to the different subgroups.

[0055] Optionally, in another embodiment of any of the above aspects, the different subgroups are based on the frequency of the RAHT coefficients.

[0056] Optionally, in another implementation of any of the above aspects, it further includes: providing an option to use a complete 2×2×2 QP offset matrix, a partial 2×2×2 QP offset matrix, or subgroup-based offsets, where the option is based on one or more of the levels of the octree and the attribute channels.

[0057] Optionally, in another implementation of any of the above aspects, it further includes: using a function or a set of functions to associate a quantization parameter (QP) offset with a function of one or more of the levels of the octree and the transform coefficient matrix.

[0058] Optionally, in another implementation of any of the above aspects, the function or the set of functions is indicated to the decoder.

[0059] Optionally, in another implementation of any of the above aspects, the function or the set of functions is different for different attribute channels.

[0060] Optionally, in another implementation of any of the above aspects, the function or the set of functions is predefined.

[0061] Optionally, in another implementation of any of the above aspects, the quantization parameter (QP) offset depends on one or more of the original point cloud resolution or the display resolution at the client side.

[0062] Optionally, in another implementation of any of the above aspects, the quantization for a larger display size is finer than that for a smaller display size.

[0063] Optionally, in another implementation of any of the above aspects, the quantization parameter (QP) offset for chrominance is based on the chrominance subsampling employed.

[0064] Optionally, in another implementation of any of the above aspects, the quantization for 4:4:4 content is coarser than that for 4:4:4 content.

[0065] Optionally, in another implementation of any of the above aspects, it further includes: when a direct current (DC) coefficient has been inherited from a previous level of the octree, applying the previously disclosed offset only to the alternating current (AC) coefficients.

[0066] Optionally, in another implementation of any of the above aspects, it further includes: transmitting the previously disclosed offset to the decoder, where when a direct current (DC) coefficient has been inherited from a previous level of the octree, the previously disclosed offset is applied only to the alternating current (AC) coefficients.

[0067] Optionally, in another embodiment of any of the above aspects, one or more syntax elements are included in the bitstream to indicate the use of any method.

[0068] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements include flags.

[0069] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the syntax structure of a parameter set in the bitstream.

[0070] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the sequence parameter set (SPS) in the bitstream.

[0071] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the group parameter set (GPS) in the bitstream.

[0072] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the adaptive parameter set (APS) in the bitstream.

[0073] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the part of the bitstream corresponding to a slice.

[0074] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the part of the bitstream corresponding to a strip.

[0075] Optionally, in another embodiment of any of the above aspects, a strip includes an attribute strip or a geometry strip.

[0076] Optionally, in another embodiment of any of the above aspects, the one or more syntax elements are included in the strip header.

[0077] Optionally, in another embodiment of any of the above aspects, the strip in the strip header includes an attribute strip or a geometry strip.

[0078] Optionally, in another embodiment of any of the above aspects, one or more flags for quantization of the weighted RAHT coefficients are included in the bitstream, and the one or more flags include:

[0079] A flag indicating the QP offset for different sub-bands, where the QP offset is binarized using fixed-length coding, exponential Golomb coding, or truncated unary coding;

[0080] A flag indicating that the QP offset is coded progressively, where progressive coding is defined as the difference between the QP offset of the current sub-band and the QP offset of the previous sub-band;

[0081] A flag indicating whether the quantization method is enabled or disabled for different sub-bands;

[0082] A flag indicating whether adaptive quantization is enabled or disabled, where the flag is set based on one or more of the attribute channel, layer, region of the point cloud video unit, slice, or a group of voxels;

[0083] A flag indicating a function for different frames, sequences, strips, slices, attribute channels, or regions of the point cloud video unit, where a list or table of different functions providing weights based on the RAHT coefficients is provided; and

[0084] A flag indicating different function parameters for different frames, sequences, strips, slices, attribute channels, or regions of the point cloud video unit,

[0085] where the one or more flags are context-coded or bypass-coded.

[0086] Optionally, in another implementation of any of the above aspects, the bitstream includes one or more flags related to using different quantization for different transform coefficients in a 2×2×2 transform residual block after upsampled prediction, and the one or more flags include:

[0087] A flag indicating a 2×2×2 QP matrix or QP offset matrix corresponding to different RAHT coefficients of the 2×2×2 transform residual block, where the entries of the 2×2×2 QP matrix or the QP offset matrix are binarized using fixed-length coding, exponential Golomb coding, or truncated unary coding;

[0088] A flag indicating that the QP offset is coded progressively, where progressive coding is defined as the difference between the QP offset of the current RAHT coefficient and the QP offset of the previous RAHT coefficient;

[0089] A flag indicating a partial 2×2×2 matrix instead of a full 2×2×2 matrix;

[0090] A flag indicating the QP offset for each subgroup;

[0091] A flag indicating that the QP offset is progressively coded / decoded relative to the layers of the octree, where the progressive coding / decoding relative to the octree is defined as the difference between the QP offset of the current layer and the QP offset of the previous layer;

[0092] A flag indicating that the QP offset is progressively coded / decoded relative to the attribute channels, where the progressive coding / decoding relative to the attribute channels is defined as the difference between the QP offset of the current attribute channel and the QP offset of the previous attribute channel;

[0093] A flag indicating the use of a partial matrix or a complete matrix, where the flag is based on one or more of the layers of the octree, the region of the point cloud video unit, and the flag is included at the level of the bitstream corresponding to the sequence, slice, tile, or region of the point cloud video unit;

[0094] A flag indicating the application of a subset of the 2×2×2 coefficients of the QP offset; and

[0095] A flag indicating a function for determining the relationship between the QP offset and the layers of the octree or the transform coefficient matrix;

[0096] wherein the one or more flags are context-coded or bypass-coded;

[0097] When the DC coefficient is inherited from the previous level of the octree, the QP offset is signaled only for the AC coefficients; and

[0098] The one or more flags are set to values that enable or disable the use of the QP offset for different layers, attribute channels, or subbands of the octree.

[0099] Optionally, in another implementation of any of the above aspects, the specified 2×2×2 is replaced by M×N×K, where M, N, and K are all integers.

[0100] Optionally, in another implementation of any of the above aspects, one or more of the QP matrix and the QP offset are included at different levels of the bitstream.

[0101] Optionally, in another implementation of any of the above aspects, one or more of the first QP matrix are included at the first level of the bitstream, and the first QP offset is included at the second level of the bitstream, where the first level is higher than the second level.

[0102] Optionally, in another implementation of any of the above aspects, one or more of the second QP matrix and the second QP offset are predicted based on the first QP matrix and the first QP offset of the point cloud video unit at the second level of the bitstream.

[0103] Optionally, in another implementation of any of the above aspects, one or more of the first QP matrix and the first QP offset are covered by the second QP matrix and the second QP offset of the point cloud video unit in the second level of the bitstream.

[0104] Optionally, in another implementation of any of the above aspects, one or more of the first QP matrix and the first QP offset are used together with the second QP matrix and the second QP offset to derive the final QP matrix and the final QP offset of the point cloud video unit in the second level of the bitstream.

[0105] Optionally, in another implementation of any of the above aspects, the first QP matrix and the first QP offset are added to the second QP matrix and the second QP offset.

[0106] Optionally, in another implementation of any of the above aspects, the syntax elements in any of the above methods are binarized into flags, fixed - length coding / decoding, exponential Golomb (EG) coding / decoding, unary coding / decoding, truncated unary coding / decoding or truncated binary coding / decoding, and the syntax elements are signed or unsigned.

[0107] Optionally, in another implementation of any of the above aspects, the syntax elements in any of the above methods are coded or bypass - coded using at least one context model.

[0108] Optionally, in another implementation of any of the above aspects, the syntax elements in any of the above methods are conditionally included in the bitstream or signaled only when the function corresponding to the syntax element is applicable.

[0109] Optionally, in another implementation of any of the above aspects, the syntax elements in any of the above methods are included in the bitstream at the block level, octree level, sequence level, group of pictures level, picture level, slice level, point cloud video unit level region, or are included in the coding structure of the coding unit (CU), the coding structure of the prediction unit (PU), the sequence header, the picture header, the sequence parameter set (SPS), the picture parameter set (PPS), the adaptive parameter set (APS) or the slice header.

[0110] Optionally, in another embodiment of any of the above aspects, whether or how any one of the above methods is applied is included at the block level, octree level, sequence level, group of pictures level, picture level, slice level, point cloud video unit level region in the bitstream, or is included in the coding and decoding structure of a coding and decoding unit (CU), the coding and decoding structure of a prediction unit (PU), sequence header, picture header, sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), or slice header.

[0111] Optionally, in another embodiment of any of the above aspects, the conversion includes encoding the media data into a bitstream.

[0112] Optionally, in another embodiment of any of the above aspects, the conversion includes decoding the media data from the bitstream.

[0113] A second aspect relates to an apparatus for processing media data, including: a processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to execute the method in any one of the above aspects.

[0114] A third aspect relates to a non-transitory computer-readable medium, including a computer program product for use by a video coding and decoding device, wherein the computer program product includes computer-executable instructions stored on the non-transitory computer-readable medium, and the computer-executable instructions, when executed by a processor, cause the video coding and decoding device to execute the method in any one of the above aspects.

[0115] A fourth aspect relates to a non-transitory computer-readable recording medium storing a bitstream of a video generated by a method executed by a video processing device, wherein the method includes the method in any one of the above aspects.

[0116] A fifth aspect relates to a method of storing a bitstream of a video, including the method in any one of the above aspects.

[0117] A sixth aspect relates to a method, apparatus, or system described in the present disclosure.

[0118] For the purpose of clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to create new embodiments within the scope of the present disclosure.

[0119] These and other features will be more clearly understood from the following detailed description in conjunction with the drawings and the claims. Description of the Drawings

[0120] To more fully understand the present disclosure, reference is now made to the following brief description in conjunction with the accompanying drawings and specific embodiments, in which like reference numerals represent like parts.

[0121] Figure 1 is an example of the parent-level node of each child node of the transform unit node.

[0122] Figure 2 is an example operation of the incremental layer QP for RAHT.

[0123] Figure 3 is an example of a bitstream structure that supports modified layer incremental QP.

[0124] Figure 4 shows a block diagram of an example of a video processing system.

[0125] Figure 5 is a block diagram of an example of a video processing device.

[0126] Figure 6 is a flowchart of an example of a video processing method.

[0127] Figure 7 shows a block diagram of an example of a video codec system.

[0128] Figure 8 shows a block diagram of an example of an encoder.

[0129] Figure 9 shows a block diagram of an example of a decoder.

[0130] Figure 10 is a schematic diagram of an example of an encoder.

[0131] Figure 11 is an image or video method implemented by a codec device according to an embodiment of the present disclosure. Detailed Description

[0132] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or to be developed. The present disclosure should in no way be limited to the illustrative implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but may be modified without departing from the full scope of the appended claims and their equivalents.

[0133] 1. Preliminary Discussion

[0134] This document relates to media file formats. Specifically, the present disclosure relates to point cloud encoding and decoding techniques. Specifically, it relates to the quantization of points in a region adaptive hierarchical transform. This idea can be applied, alone or in various combinations, to any point cloud encoding and decoding standard or non-standard point cloud codec, such as the geometry-based point cloud compression (G-PCC) being developed.

[0135] 2. Abbreviations

[0136] G-PCC Geometry-based Point Cloud Compression

[0137] MPEG Moving Picture Experts Group

[0138] 3DG 3D Graphics Coding Group

[0139] CFP Call for Proposals

[0140] V-PCC Video-based Point Cloud Compression

[0141] RAHT Region Adaptive Hierarchical Transform

[0142] SPS Sequence Parameter Set

[0143] APS Attribute Parameter Set

[0144] GPS Geometry Parameter Set

[0145] 3. Further Discussion

[0146] MPEG is an abbreviation for the Moving Picture Experts Group, which is one of the main standardization organizations for dealing with multimedia. In 2017, the MPEG 3D Graphics Coding Group (3DG) released a Call for Proposals (CFP) document to start developing a point cloud encoding and decoding standard [1]. The final standard will include two types of solutions. Video-based Point Cloud Compression (V-PCC) is suitable for point sets with a relatively uniform point distribution [2]. Geometry-based Point Cloud Compression (G-PCC) is suitable for sparser distributions [3]. Both V-PCC and G-PCC support the encoding and decoding of single point clouds and point cloud sequences.

[0147] In a point cloud, there can be geometric information and attribute information. Geometric information is used to describe the geometric positions of data points. Attribute information is used to record some details of data points, such as texture, normal vector, reflection, etc.

[0148] 3.1 Octree Geometry Compression

[0149] Point cloud codecs can process various information in different ways. Generally, there are many optional tools in the codec to separately support the encoding and decoding of geometric information and attribute information. Among the geometric encoding and decoding tools in G-PCC, octree geometry compression has an important impact on the point cloud geometry encoding and decoding performance in G-PCC [4].

[0150] In G-PCC, one of the important point cloud geometry encoding and decoding tools is octree geometry compression, which makes full use of the spatial correlation of point cloud geometry. If the geometry encoding and decoding tool is enabled, the cube axis-aligned bounding box associated with the octree root node will be determined according to the point cloud geometry information. Then, the bounding box will be subdivided into 8 sub-cubes, which are associated with the 8 child nodes of the root node (the cube is equivalent to the node hereafter). Then, 8-bit codes are generated in a specific order to indicate whether the 8 child nodes contain points respectively, where 1 bit is associated with 1 child node. The bit associated with a child node is called an occupancy bit, and the generated 8-bit code is called an occupancy code. The generated occupancy code will be signaled according to the occupancy information of neighboring nodes. Then, only the nodes containing points will be further subdivided into 8 child nodes. This process will be executed recursively until the node size is 1. Therefore, the point cloud geometry information is converted into a sequence of occupancy codes.

[0151] On the decoder side, the sequence of occupancy codes will be decoded, and the point cloud geometry information can be reconstructed according to the sequence of occupancy codes.

[0152] The breadth-first scan order will be used for the octree. In one level of the octree, the octree nodes will be scanned in Morton order. If the coordinates of a node are represented by N bits, the coordinates (X, Y, Z) of the node can be represented as follows:

[0153] X = (x N-1 x N-2 …x1x0)

[0154] Y = (y N-1 y N-2 …y1y0)

[0155] Z = (z N-1 z N-2 …z1z0)

[0156] Its Morton code can be represented as follows:

[0157] M = (x N-1 y N-1 z N-1 x 1-2 y n-2 z M-2 …x1y1z1x0y0z0)

[0158] The Morton order is the order from smallest to largest according to the Morton code.

[0159] 3.2 Region-Adaptive Hierarchical Transform

[0160] In G-PCC, one of the important point cloud attribute encoding and decoding tools is RAHT. It is a transform that uses the attributes associated with nodes in the lower levels of an octree to predict the attributes of nodes in the next level [5]. It assumes that the positions of the points are given at both the encoder and the decoder. RAHT follows the octree scan in reverse, from the leaf nodes to the root node, recombining the nodes into larger nodes at each step until reaching the root node. At each level of the octree, the nodes are processed in Morton order. At each decomposition, instead of grouping eight nodes at a time, RAHT proceeds in three steps along each dimension (e.g., along z, then y, then x). If there are L levels in the octree, RAHT requires 3L levels to traverse the tree in reverse.

[0161] Let the node at level l be g l,x,y,z , for x, y, z integers. g l,x,y,z is obtained by grouping g l+1,2x,y,z and g l+1,2x+1,y,z , where the grouping along the first dimension is an example. RAHT only processes occupied nodes. If one of the nodes in a pair is unoccupied, the other node is promoted to the next level and not processed, i.e., if the latter is the occupied node in the pair, then g l-1,x,y,z = g l,2x,y,z . The grouping process is repeated until the root is reached. Note that the grouping process generates nodes at lower levels that are the result of grouping different numbers of voxels along the way. The number of nodes grouped to generate node g l,x,y,z is the weight ω l,x,y,z of that node.

[0162] At each grouping of two nodes (e.g., g l,2x,y,z and g l,2x+1,y,z ) and their respective weights ω l,2x,y,z and ω l,2x+1,y,z , RAHT applies the following transform:

[0163]

[0164] where ω1 = ω l,2x,y,z , ω2 = ω l,2x+1,y,z , and

[0165]

[0166] Note that the transformation matrix is always changing to adapt to the weights, i.e., to adapt to the number of leaf nodes that each g l,x,y,z actually represents. g l,x,y,z is used to group and combine nodes at lower levels. h l,x,y,zare the actual high-pass coefficients generated by the transform to be coded and transmitted. Additionally, the weights are accumulated at the levels above. In the example above,

[0167] ω l-1,2,y,z = ω l,2x,y,z + ω l,2x+1,y,z

[0168] In the final stage, the remaining two voxels g 1,0,0,0 and g 1,1,0,0 [[ID=1--5]]at the root of the tree are transformed into the final two coefficients:

[0169]

[0170] where g DC = g 0,0,0,0 .

[0171] 3.3 Upsampling transform domain prediction in RAHT

[0172] Figure 1 is an example of the parent-level node of each child node of the transform unit node.

[0173] Upsampling transform domain prediction is introduced to improve the coding and decoding efficiency of RAHT [6]. It consists of two parts.

[0174] First, the RAHT tree traversal is changed from the previous ascending method to descending-based, i.e., a tree of the sum of attributes and weights is constructed, and then RAHT is performed from the root to the leaves for both the encoder and the decoder. The transform is also performed in the octree node transform unit with 2×2×2 child nodes. Inside the node, the encoder transform order is from the leaves to the root.

[0175] Second, for each child node of the transform unit, a corresponding predicted child node is generated by upsampling the previous transform level. In fact, only the child node containing the last point will generate a corresponding predicted child node. The transform unit containing 2×2×2 predicted child nodes is transformed on the encoder side and subtracted from the transformed attributes. The residual of the AC coefficients will be signaled. Note that the prediction does not affect the DC coefficients.

[0176] Each child node of the transform unit is predicted by 7 parent-level nodes, including 3 collinear parent-level neighboring nodes, 3 coplanar parent-level neighboring nodes, and 1 parent node. The coplanar neighboring nodes and collinear neighboring nodes are the neighboring nodes sharing a face and an edge with the current transform unit node, respectively. Figure 1 shows the 7 parent-level nodes of each child node of the transform unit node.

[0177] The attribute a up of each child node is predicted as follows according to the distance between it and the parent-level nodes:

[0178] a up = ∑ω k a k / ∑ω k ,

[0179] a k is an attribute of one of its parent-level nodes, and ω k is a distance-dependent weight. In G-PCC, ω parrnt : ω coplane : ω coline = 4:2:1.

[0180] 3.4 Quantization

[0181] Figure 2 is an example operation for the incremental layer QP of RAHT. Figure 2 Shows RAHT with incremental QP for each layer, and specific incremental QP is applied to each layer. The effective QP value for each layer in a specific strip is added between the incremental QP of that layer and the QP for that strip. The final residual value will be the attribute divided by the square root of the sum of weights and the effective QP value.

[0182] Quantization parameter (QPAPS) is stored in the attribute parameter set (APS), and the parameter can be changed slice by slice and also layer by layer.

[0183] Figure 3 is an example of a bitstream structure that supports modified layer incremental QP.

[0184] By adding incremental QP layer values (ΔQP slic_Layer , ΔQP slice1_Laye …) in each attribute strip header, the QP value can be changed for each specific layer. The incremental QP layer value is the difference between the QP strip and the effective QP value of the layer of that strip. The incremental QP layer exists only when the layer_QP_present_flag in the attribute strip header ASH is set to 1.

[0185] 3.5 Coding and Decoding Parameter Classification

[0186] There are some coding and decoding parameters in the encoder to control the encoding of the point cloud. Some of them are signaled to the decoder to support the decoding process. These parameters can be classified and stored in several clusters according to the affected parts of each parameter, such as the geometry parameter set (GPS), the attribute parameter set (APS), and the sequence parameter set (SPS). The parameters that control the geometry coding and decoding tools are stored in GPS. The parameters that control the attribute coding and decoding tools are stored in APS. For example, the parameters that describe the attribute category of the point cloud sequence and the data precision of the coding and decoding process are stored in SPS.

[0187] 4. Technical problems solved by the disclosed technical solutions

[0188] Example designs for point cloud attribute quantization of Region Adaptive Hierarchical Transform (RAHT) coefficients have the following problems:

[0189] First, the quantization step size can vary for different layers. However, this is not always optimal. The weights of the RAHT coefficients can be better parameters for deriving or determining the quantization step size.

[0190] Second, after upsampled prediction, 2×2×2 residual blocks are transformed and quantized. However, during quantization, all eight transform coefficients are quantized in the same way, which may not be optimal.

[0191] 5. List of solutions and examples

[0192] To solve the above problems and some other problems not mentioned, the methods outlined below are disclosed. These items should be considered as examples for explaining general concepts and should not be construed in a narrow sense. In addition, these inventions can be applied individually or combined in any way.

[0193] 1) In one example, the quantization step size used in RAHT coefficients can be changed in a point cloud video unit, which can be a sequence, frame, cube, block, etc.

[0194] a. In one example, the quantization step size can depend on the weight of the RAHT coefficient.

[0195] b. In one example, different step sizes can be used for coefficients with different weights.

[0196] i. In one example, there can be at least one indication to indicate different step sizes, such as QP offset.

[0197] ii. In one example, the indication can be explicitly signaled to the decoder.

[0198] iii. In one example, the weights can be divided into different groups, and different quantization offsets can be transmitted for different groups.

[0199] iv. In one example, the weights can be divided into different groups, and different quantization offsets can be transmitted for subsets of the groups.

[0200] c. In one example, the determination of the step size can be based on some predefined functions.

[0201] i. In one example, the function can be where Δ is the quantization step size, K is a constant, and w is the weight of the RAHT coefficient.

[0202] 1. In one example, the value of K can be signaled to the decoder and can vary, for example, at the slice level, picture level, etc.

[0203] 2. In one example, the function can be replaced by other functions, such as w, etc.

[0204] 3. In one example, the function can be used only for specific ranges or values of w.

[0205] ii. In one example, the step size can be restricted between a minimum step size value and a maximum step size value.

[0206] d. In one example, weight-based quantization can completely replace octree-layer-based adaptive quantization.

[0207] e. In one example, weight-based quantization can be an additional quantization mode, and additional information about the use of the quantization type can be sent to the decoder.

[0208] f. In one example, quantization can be determined based on both the layer and the weight of the RAHT coefficient.

[0209] i. In one example, the QP offset can be determined first based on the layer, and the offset can be further modified based on the weight of the RAHT coefficient.

[0210] ii. In one example, the quantization step size can vary based on a function jointly determined by the layer and the weight of the RAHT coefficient.

[0211] g. In one example, different quantization methods can be applied to different attribute channels.

[0212] i. In one example, quantization of the luminance channel can be based on layer-based adaptive quantization, and quantization of the chrominance channel can be based on quantization depending on the weight of the RAHT coefficient.

[0213] ii. In one example, different functions based on the RAHT weight can be used for different attribute channels.

[0214] iii. In one example, for different channels, the partitioning into different subgroups can be different, and the corresponding offsets can also be different.

[0215] h. In one example, the point cloud can be segmented into regions, and different quantization methods can be applied to different regions.

[0216] i. In one example, the quantization step size may vary based on a function jointly determined by the region, layer, and weight of the RAHT coefficients.

[0217] ii. In one example, layer-based quantization may be used for some regions, and weight-based quantization may be used for other regions.

[0218] iii. In one example, the region-based QP offset may be determined first, and then the QP offset may be further modified based on the weight.

[0219] iv. In one example, for different regions, the division into different subgroups may be different, and the corresponding offsets transmitted through the signal may also be different.

[0220] i. In one example, the method may be selectively enabled / disabled for a group of voxels, such as a group of 2×2×2 voxels.

[0221] 2) In one example, for the residual block of the 2×2×2 transform after upsampled prediction, different quantization may be used for different transform coefficients.

[0222] a. In one example, a 2×2×2 QP offset matrix may be used to derive the quantization step size.

[0223] i. In one example, the matrix may be specified at the sequence level / strip level / slice level, etc.

[0224] b. In one example, the 2×2×2 QP offset matrix may depend on the octree layer.

[0225] i. In one example, the use of the matrix may be restricted to a specific layer in the octree.

[0226] ii. In one example, different 3D QP matrices may be used for different layers in the octree.

[0227] c. In one example, the 2×2×2 QP offset matrix may depend on the RAHT weight.

[0228] i. In one example, the use of the matrix may be restricted to some subbands of the RAHT, where a subband represents a group of coefficients with the same RAHT weight.

[0229] ii. In one example, different 3D QP matrices may be used for different subbands.

[0230] d. In one example, the 2×2×2 QP offset matrix may depend on the attribute channel.

[0231] i. In one example, the matrix can be used only for a subset of the attribute channels.

[0232] ii. In one example, different matrices can be used for different attribute channels.

[0233] e. In one example, the 2×2×2 QP matrix can depend on the region.

[0234] i. In one example, the QP matrix can be used only for some regions.

[0235] ii. In one example, different matrices can be used for different regions.

[0236] f. In one example, only a subset of the coefficients can be modified instead of modifying the quantization of all 8 (2×2×2) transform coefficients.

[0237] i. For example, coarser quantization can be applied to the high-frequency coefficients without modifying the low-frequency coefficients.

[0238] ii. For example, the use of such partial modification of the coefficients can depend on other factors such as the attribute channels, RAHT weights, octree levels, etc.

[0239] g. In one example, the RAHT coefficients can be partitioned into subgroups and different QP offsets can be applied to different subgroups.

[0240] i. For example, the subgroups can be based on the frequency of the coefficients.

[0241] h. In one example, there can be additional options regarding the use of the full 2×2×2 QP offset matrix or a partial matrix or subgroup-based offsets, which can depend on various parameters such as the octree level, attribute channels, etc.

[0242] i. In one example, a function or a set of functions can be used to represent the QP offset as a function of the octree level, transform coefficient index, etc.

[0243] i. In one example, the function or the functions can be transmitted to the decoder.

[0244] ii. In one example, the function or the functions can be different for different attribute channels.

[0245] iii. In one example, only predefined functions can be used.

[0246] j. In one example, the QP offset can depend on factors such as the original point cloud resolution, the display resolution at the client side, etc.

[0247] i. In one example, a larger display size may require finer quantization relative to a smaller display size.

[0248] k. In one example, the QP offset for chrominance may depend on the chrominance subsampling employed.

[0249] i. For example, coarser quantization may be applied to 4:4:4 content compared to 4:2:0 content.

[0250] l. In one example, the method may be selectively enabled / disabled for a set of voxels, such as a set of 2×2×2 voxels.

[0251] m. In one example, when the DC is inherited from a previous octree layer, the previously disclosed offset may be transmitted or applied only for AC coefficients.

[0252] 3) In one example, for each of the methods disclosed above, a syntax element (e.g., a flag) or syntax elements (flags) may be signaled in the bitstream to specify the use of the disclosed method.

[0253] a. In one example, the bitstream unit may be the bitstream of the syntax structure of a parameter set.

[0254] i. In one example, the syntax structure may be the SPS.

[0255] ii. In one example, the syntax structure may be the GPS.

[0256] iii. In one example, the syntax structure may be the APS.

[0257] b. In one example, the bitstream unit may be the bitstream of a slice.

[0258] c. In one example, the bitstream unit may be the bitstream of a strip.

[0259] i. In one example, the strip may be an attribute strip, a geometry strip.

[0260] d. In one example, the bitstream unit may be the strip header.

[0261] i. In one example, the strip may be an attribute strip, a geometry strip.

[0262] e. In one example, for RAHT coefficient quantization based on RAHT coefficient weights, the following flags may be signaled.

[0263] i. In one example, the QP offset can be signaled for different subbands. The offset can be binarized using fixed-length coding, exponential-Golomb (EG) coding, (truncated) unary coding, etc.

[0264] ii. Additionally, the QP offset can be coded progressively, i.e., the difference between the QP offset of the current subband and the QP offset of the previous subband can be signaled.

[0265] iii. In one example, an additional flag can be signaled to enable / disable the quantization method for different subbands.

[0266] iv. In one example, a flag can be signaled based on an attribute channel, layer, region, slice, a set of voxels, etc., to enable / disable adaptive quantization.

[0267] v. In one example, a list / table of different functions based on RAHT weights can be defined, and a flag can be signaled to specify the function for different frames, sequences, stripes, slices, attribute channels, regions, etc.

[0268] vi. Additionally, different function parameters can be signaled for different frames, sequences, stripes, slices, attribute channels, regions, etc.

[0269] vii. In one example, the above flag can be context-coded or bypass-coded.

[0270] f. In one example, for a method of using different quantization for different transform coefficients in a 2×2×2 transform residual block after upsampled prediction, the following flags can be signaled.

[0271] i. In one example, a (2×2×2) QP matrix or QP offset matrix can be signaled for different RAHT coefficients (corresponding to the transform of the 2×2×2 residual block). The entries of the matrix can be binarized using fixed-length coding, exponential-Golomb (EG) coding, (truncated) unary coding, etc.

[0272] ii. Additionally, the QP offset can be coded progressively, i.e., the difference between the QP offset of the current RAHT coefficient and the QP offset of the previous RAHT coefficient can be signaled.

[0273] iii. In one example, a partial matrix instead of the full 2×2×2 matrix can be signaled.

[0274] iv. In one example, the QP offset can be signaled for each subgroup.

[0275] v. In one example, the QP offset can be progressively coded / decoded relative to an octree layer, i.e., the difference between the QP offset of the current layer and the QP offset of the previous layer can be signaled.

[0276] vi. In one example, the QP offset can be progressively coded / decoded relative to an attribute channel, i.e., the difference between the QP offset of the current attribute channel and the QP offset of the previously coded channel can be coded / decoded.

[0277] vii. In one example, an additional flag can be signaled to specify the use of a full matrix or a partial matrix, and the flag can depend on the octree layer, region, etc., and can be signaled at, for example, the sequence / frame / strip / slice / region level, etc.

[0278] viii. Additionally, a flag can be signaled to signal a subset of the 2×2×2 coefficients to which the QP offset is applied.

[0279] ix. In one example, a flag can be signaled to convey a function that determines the relationship between the QP offset and the octree layer, transform coefficient index, etc.

[0280] x. In one example, the above-mentioned flag can be context-coded or bypass-coded.

[0281] xi. In one example, when the DC is inherited from the previous octree level, the QP offset is signaled only for the AC coefficients.

[0282] xii. In one example, a flag can be signaled to enable / disable the use of the QP offset for different octree layers, attribute channels, subbands, etc.

[0283] 2) In the disclosed method, "2×2×2" can be replaced by "M×N×K", where M, N, and K are integers.

[0284] 3) In one example, the QP matrix and / or offset can be signaled at different levels.

[0285] a. In one example, a first QP matrix and / or offset can be signaled at a first level, and a second QP matrix and / or offset can be signaled at a second level.

[0286] b. If the first level is higher than the second level (e.g., the first level is a sequence and the second level is a picture).

[0287] i. For example, the second QP matrix and / or offset can be predicted from the first QP matrix and / or offset of a specific unit in the second level.

[0288] ii. For example, the first QP matrix and / or offset may be overridden by the second QP matrix and / or offset of a specific unit in the second level.

[0289] iii. For example, the first QP matrix and / or offset may be used together with the second QP matrix and / or offset to derive the final QP matrix and / or offset of a specific unit in the second level.

[0290] 1. For example, the first QP matrix and / or offset may be added to the second QP matrix and / or offset.

[0291] 4) The above-mentioned disclosed syntax elements may be binarized into flags, fixed-length coding / decoding, EG(x) coding / decoding, unary coding / decoding, truncated unary coding / decoding, truncated binary coding / decoding, etc. It may be signed or unsigned.

[0292] 5) The above-mentioned disclosed syntax elements may be coded / decoded using at least one context model. Or it may be bypass coded / decoded.

[0293] 6) The above-mentioned disclosed syntax elements may be signaled in a conditional manner.

[0294] a. The SE is signaled only when the corresponding function applies.

[0295] 7) The above-mentioned disclosed syntax elements may be signaled at the block level / octree level / sequence level / group of pictures level / picture level / strip level / region level, such as in the coding / decoding structure of a CU / PU, or in the sequence header / picture header / SPS / APS / PPS / strip header.

[0296] 8) Whether and / or how to apply the method disclosed above may be signaled at the block level / octree level / sequence level / group of pictures level / picture level / strip level / region level / slice group level, such as in the coding / decoding structure of a CU / PB, or in the sequence header / picture header / SPS / APS / strip header / slice group header.

[0297] 6. Embodiments

[0298] 6.1 Summary

[0299] In the current G-PCC [1], for RAHT coefficient coding / decoding, layer-based QP offsets are signaled. In this proposal, a 3D quantization matrix for RAHT coding / decoding is proposed to signal the QP offset for each AC coefficient obtained by transforming the 2×2×2 residuals after upsampled prediction.

[0300] 6.2 Introduction

[0301] In the current G-PCC [1], layer-based quantization is employed. During RAHT encoding and decoding, the AC components of the residual 2×2×2 blocks generated after upsampled prediction are transformed and quantized. We note that the DC is simply inherited, so there is no residual DC component that needs to be sent to the decoder. When processing 2×2×2 residual blocks, all AC coefficients share the same quantizer.

[0302] 6.3 Proposed Syntax

[0303] Having the flexibility to vary quantization for different AC components can be beneficial, especially from a perceptual point of view. Additionally, AC coefficients in different octree layers can have different physical meanings and importance. Therefore, it can also be beneficial to further adapt to this by enabling different quantization for AC coefficients in different layers. To enable this, the proposal introduces some syntax elements presented next.

[0304]

[0305]

[0306] 6.4 Semantics

[0307] ash_attr_ACcomp_QP_luma_present_flag: Indicates whether the delta QP for the AC component exists for the luma channel

[0308] ash_attr_ACcomp_QP_chroma_present_flag: Indicates whether the delta QP for the AC component exists for the chroma channel

[0309] ash_attr_ACcomp_layer_QP_luma_present_flag[idx]: Indicates whether the delta QP for the AC component exists for the luma channel of the specific layer indexed by idx in the octree

[0310] ash_attr_ACcomp_layer_QP_chroma_present_flag[idx]: Indicates whether the delta QP for the AC component exists for the chroma channel of the specific layer indexed by idx in the octree

[0311] ash_attr_delta_ACcomp_layer_QP_luma[idx][compidx]: Indicates the delta QP of the AC component indexed by compidx for the luma channel of the specific layer indexed by idx in the octree

[0312] ash_attr_delta_ACcomp_layer_QP_chroma[idx][compidx]: Indicates the delta QP of the AC component indexed by compidx of the chroma channel of the specific layer indexed by idx in the octree

[0313] 6.5 Usage

[0314] The QP of each layer is first derived according to the current specification, as Figure 2 shown.

[0315] After deriving the QP of layer 'l', the QP of each AC coefficient indexed by 'i' is derived as

[0316] QP l,i = QP l + ΔQP(l,i); i = 0,1..6

[0317] where QP l,i is the final QP for layer l and AC component index i. ΔQP(l,i) is the delta value obtained from the previously presented syntax elements.

[0318] 7. References

[0319] [1] MPEG 3DG and Requirements, “Call for Proposals for Point Cloud Compression V2”, ISO / IEC JTC1 / SC29 WG11 N16763.

[0320] [2] ISO / IEC JTC 1 / SC 29 / WG 07, “Information technology—Coded Representation of Immersive Media—Part 5: Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC)”, ISO / IEC 23090-5.

[0321] [3]ISO / IEC JTC 1 / SC 29 / WG 11, “Information technology—MPEG-I (Coded Representation of Immersive Media)—Part 9: Geometry-based Point Cloud Compression”, ISO / IEC 23090-9:2020 (E).

[0322] [4]MPEG 3D Graphics Coding, “G-PCC codec description”, ISO / IEC JTC1 / SC29 WG07N0015.

[0323] [5]Ricardo L. De Queiroz and Philip A. Chou, “Compression of 3D Point Clouds Using a Region-Adaptive Hierarchical Transform”, IEEE Transactions on Image Processing.

[0324] [6]S. Lasserre, D. Flynn, “On an improvement of RAHT to exploit attribute correlation”, ISO / IEC JTC1 / SC29 / WG11 M47378.

[0325] Figure 4 FIG. 4000 is a block diagram illustrating an example video processing system 4000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all components of system 4000. System 4000 may include an input 4002 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8- or 10-bit multi-component pixel values, or may be received in a compressed or encoded format. Input 4002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), etc., and wireless interfaces such as Wireless Fidelity (Wi-Fi) or cellular interfaces.

[0326] System 4000 may include an encoding component 4004 that may implement various encoding or decoding methods described in this document. The encoding component 4004 may reduce the average bit rate of the video from input 4002 to the output of the encoding component 4004 to produce an encoded representation of the video. Encoding techniques are thus sometimes referred to as video compression or video transcoding techniques. The output of the encoding component 4004 may be stored or transmitted via a communication connection represented by component 4006. The stored or communicatively transmitted bitstream (or encoded) representation of the video received at input 4002 may be used by component 4008 to generate pixel values or a displayable video for transmission to a display interface 4010. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Additionally, while certain video processing operations are referred to as "encoding" operations or tools, it can be understood that encoding tools or operations are used by an encoder and the corresponding decoding tools or operations that reverse the encoding results will be performed by a decoder.

[0327] Examples of a peripheral bus interface or a display interface may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), Displayport, etc. Examples of a storage interface include Serial Advanced Technology Attachment (SATA), Peripheral Component Interconnect (PCI), Integrated Drive Electronics (IDE) interface, etc. The techniques described in this document may be embodied in various electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of performing digital data processing and / or video display.

[0328] Figure 5 is a block diagram of an example of a video processing apparatus 4100. The apparatus 4100 may be used to implement one or more methods described herein. The apparatus 4100 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 4100 may include one or more processors 4102, one or more memories 4104, and video processing circuitry 4106. The (one or more) processors 4102 may be configured to implement one or more methods described in this document. The (one or more) memories 4104 may be used to store data and code for implementing the methods and techniques described herein. The video processing circuitry 4106 may be used to implement some of the techniques described in this document in hardware circuitry. In some embodiments, the video processing circuitry 4106 may be at least partially included in the processor 4102, e.g., a graphics coprocessor.

[0329] Figure 6FIG. 4200 is an example flowchart of a video processing method 4200. At step 4202, method 4200 determines quantization based on region adaptive hierarchical transform (RAHT) weights. At step 4204, a conversion is performed between visual media data and a bitstream based on the quantization. The conversion at step 4204 can include encoding at an encoder or decoding at a decoder, depending on the example.

[0330] It should be noted that method 4200 can be implemented in a device for processing video data that includes a processor and a non-transitory memory having instructions thereon, such as video encoder 4400, video decoder 4500, and / or encoder 4600. In such a case, the instructions, when executed by the processor, cause the processor to perform method 4200. Additionally, method 4200 can be executed by a non-transitory computer-readable medium including a computer program product for use in a video codec device. The computer program product includes computer-executable instructions stored on a non-transitory computer-readable medium that, when executed by a processor, cause the video codec device to perform method 4200.

[0331] Figure 7 FIG. 4300 is a block diagram of an example of a video codec system 4300 that can utilize the techniques of the present disclosure. Video codec system 4300 can include a source device 4310 and a destination device 4320. Source device 4310 generates encoded video data, where the source device 4310 can be referred to as a video encoding device. Destination device 4320 can decode the encoded video data generated by source device 4310, where the destination device 4320 can be referred to as a video decoding device.

[0332] Source device 4310 can include a video source 4312, a video encoder 4314, and an input / output (I / O) interface 4316. Video source 4312 can include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data can include one or more pictures. Video encoder 4314 encodes the video data from video source 4312 to generate a bitstream. The bitstream can include a sequence of bits that forms an encoded representation of the video data. The bitstream can include encoded pictures and associated data. An encoded picture is an encoded representation of a picture. The associated data can include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 4316 can include a modulator / demodulator (modem) and / or a transmitter. The encoded video data can be transmitted directly to destination device 4320 via I / O interface 4316 over network 4330. The encoded video data can also be stored on a storage medium / server 4340 for access by destination device 4320.

[0333] The target device 4320 may include an I / O interface 4326, a video decoder 4324, and a display device 4322. The I / O interface 4326 may include a receiver and / or a modem. The I / O interface 4326 may obtain encoded video data from a source device 4310 or a storage medium / server 4340. The video decoder 4324 may decode the encoded video data. The display device 4322 may display the decoded video data to a user. The display device 4322 may be integrated with the target device 4320, or may be external to the target device 4320, which may be configured to interface with an external display device.

[0334] The video encoder 4314 and the video decoder 4324 may operate according to video compression standards, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other existing and / or further standards.

[0335] Figure 8 A block diagram of an example of a video encoder 4400 is shown, and the video encoder 4400 may be Figure 7 the video encoder 4314 in the system 4300 shown. The video encoder 4400 may be configured to perform any or all of the techniques of the present disclosure. The video encoder 4400 includes a plurality of functional components. The techniques described in the present disclosure may be shared among the various components of the video encoder 4400. In some examples, a processor may be configured to perform any or all of the techniques described in the present disclosure.

[0336] The functional components of the video encoder 4400 may include a splitting unit 4401, a prediction unit 4402, a residual generation unit 4407, a transform processing unit 4408, a quantization unit 4409, an inverse quantization unit 4410, an inverse transform unit 4411, a reconstruction unit 4412, a buffer 4413, and an entropy coding unit 4414, and the prediction unit 4402 may include a mode selection unit 4403, a motion estimation unit 4404, a motion compensation unit 4405, and an intra prediction unit 4406.

[0337] In other examples, the video encoder 4400 may include more, fewer, or different functional components. In one example, the prediction unit 4402 may include an Intra Block Copy (IBC) unit. The IBC unit may perform prediction in the IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0338] In addition, some components, such as the motion estimation unit 4404 and the motion compensation unit 4405, may be highly integrated, but for the purpose of explanation, these components are shown separately in the example of the video encoder 4400.

[0339] The splitting unit 4401 can split the picture into one or more video blocks. The video encoder 4400 and the video decoder 4500 can support various video block sizes.

[0340] The mode selection unit 4403 can select one of multiple coding / decoding modes (intra coding / decoding or inter coding / decoding), for example, based on the error result, and provide the generated intra-coded block or inter-coded block to the residual generation unit 4407 to generate residual block data, and provide it to the reconstruction unit 4412 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 4403 can select the combined intra / inter prediction (CIIP) mode, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 4403 can also select the resolution for the motion vector for the block (e.g., sub-pixel accuracy or integer pixel accuracy).

[0341] To perform inter prediction on the current video block, the motion estimation unit 4404 can generate motion information for the current video block by comparing one or more reference frames from the cache 4413 with the current video block. The motion compensation unit 4405 can determine a predicted video block for the current video block based on the motion information and the decoded samples of the pictures from the cache 4413 other than the picture associated with the current video block.

[0342] The motion estimation unit 4404 and the motion compensation unit 4405 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-slice, a P-slice, or a B-slice.

[0343] In some examples, the motion estimation unit 4404 can perform uni-directional prediction on the current video block, and the motion estimation unit 4404 can search the reference pictures in list 0 or list 1 to find a reference video block for the current video block. The motion estimation unit 4404 can then generate a reference index and a motion vector, where the reference index indicates the reference picture in list 0 or list 1 that contains the reference video block, and the motion vector indicates the spatial displacement between the current video block and the reference video block. The motion estimation unit 4404 can output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 4405 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0344] In other examples, the motion estimation unit 4404 may perform bidirectional prediction on a current video block. The motion estimation unit 4404 may search the reference pictures in list 0 to find a reference video block for the current video block, and may also search the reference pictures in list 1 to find another reference video block for the current video block. The motion estimation unit 4404 may then generate a reference index and a motion vector. The reference index indicates the reference pictures in list 0 and list 1 that contain the reference video blocks, and the motion vector indicates the spatial displacement between the reference video block and the current video block. The motion estimation unit 4404 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 4405 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0345] In some examples, the motion estimation unit 4404 may output a complete set of motion information for the decoding process of the decoder. In some examples, the motion estimation unit 4404 may not output a complete set of motion information for the current video. Instead, the motion estimation unit 4404 may signal the motion information of the current video block by referring to the motion information of another video block. For example, the motion estimation unit 4404 may determine that the motion information of the current video block is similar enough to the motion information of an adjacent video block.

[0346] In one example, the motion estimation unit 4404 may indicate a value in the syntax structure associated with the current video block to the video decoder 4500, and the value indicates that the current video block has the same motion information as another video block.

[0347] In another example, the motion estimation unit 4404 may indicate another video block and a motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 4500 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0348] As discussed above, the video encoder 4400 may signal motion vectors in a predictive manner. Two examples of predictive signaling techniques that may be implemented by the video encoder 4400 include advanced motion vector prediction (AMVP) and Merge mode signaling.

[0349] The intra prediction unit 4406 may perform intra prediction on the current video block. When the intra prediction unit 4406 performs intra prediction on the current video block, the intra prediction unit 4406 may generate prediction data for the current video block based on the decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a predicted video block and various syntax elements.

[0350] The residual generation unit 4407 may generate residual data for the current video block by subtracting the (multiple) predicted video blocks of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0351] In other examples, such as in the skip mode, there may be no residual data for the current video block, and the residual generation unit 4407 may not perform the subtraction operation.

[0352] The transform processing unit 4408 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0353] After the transform processing unit 4408 generates the transform coefficient video block associated with the current video block, the quantization unit 4409 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0354] The inverse quantization unit 4410 and the inverse transform unit 4411 may apply inverse quantization and inverse transform to the transform coefficient video block respectively to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 4412 may add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 4402 to generate a reconstructed video block associated with the current block for storage in the cache 4413.

[0355] After the reconstruction unit 4412 reconstructs the video block, a loop filtering operation may be performed to reduce the blocking artifacts in the video block.

[0356] The entropy coding unit 4414 may receive data from other functional components of the video encoder 4400. When the entropy coding unit 4414 receives data, the entropy coding unit 4414 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0357] Figure 9 A block diagram showing an example of a video decoder 4500 is shown, and the video decoder 4500 may be Figure 7The video decoder 4324 in the illustrated system 4300. The video decoder 4500 may be configured to perform any or all of the techniques of the present disclosure. In the illustrated example, the video decoder 4500 includes a plurality of functional components. The techniques described in the present disclosure may be shared among the various components of the video decoder 4500. In some examples, a processor may be configured to perform any or all of the techniques described in the present disclosure.

[0358] In the illustrated example, the video decoder 4500 includes an entropy decoding unit 4501, a motion compensation unit 4502, an intra prediction unit 4503, an inverse quantization unit 4504, an inverse transform unit 4505, a reconstruction unit 4506, and a cache 4507. In some examples, the video decoder 4500 may perform a decoding process generally opposite to the encoding process described for the video encoder 4400.

[0359] The entropy decoding unit 4501 may retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 4501 may decode the entropy-encoded video data, and based on the entropy-decoded video data, the motion compensation unit 4502 may determine motion information including motion vectors, motion vector precision, reference picture list indices, and other motion information. The motion compensation unit 4502 may determine this information, for example, by performing AMVP and Merge modes.

[0360] The motion compensation unit 4502 may generate a motion-compensated block and may perform interpolation based on an interpolation filter. An identifier of the interpolation filter to be used with sub-pixel precision may be included in the syntax element.

[0361] The motion compensation unit 4502 may use the interpolation filter used by the video encoder 4400 during the encoding of a video block to compute the interpolation of sub-integer pixels for a reference block. The motion compensation unit 4502 may determine the interpolation filter used by the video encoder 4400 based on the received syntax information, and the motion compensation unit 4502 may use the interpolation filter to generate a prediction block.

[0362] The motion compensation unit 4502 may use some syntax information to determine the size of the blocks used to encode the (multiple) frames and / or (multiple) slices of the encoded video sequence, the partitioning information describing how each macroblock of a picture of the encoded video sequence is partitioned, the mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.

[0363] The intra prediction unit 4503 may form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 4504 inverse quantizes the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 4501, i.e., dequantizes them. The inverse transform unit 4505 applies an inverse transform.

[0364] The reconstruction unit 4506 may add the residual block to the corresponding prediction block generated by the motion compensation unit 4502 or the intra prediction unit 4503 to form a decoded block. As desired, a deblocking filter may also be used to filter the decoded block to eliminate block effect artifacts. Then, the decoded video block is stored in the buffer 4507, which provides reference blocks for subsequent motion compensation / intra prediction and also produces the decoded video for presentation on a display device.

[0365] Figure 10 is a schematic diagram of an example of the encoder 4600. The encoder 4600 is suitable for implementing the technologies of VVC. The encoder 4600 includes three loop filters, namely, the deblocking filter (DF) 4602, the sample adaptive offset (SAO) 4604, and the adaptive loop filter (ALF) 4606. Different from the DF 4602 that uses predefined filters, the SAO 4604 and the ALF 4606 utilize the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding offsets and by applying a finite impulse response (FIR) filter respectively, where the transcoded side information signals the offsets and the filter coefficients. The ALF 4606 is located at the last processing stage of each picture and can be regarded as a tool for trying to capture and repair the artifacts created by the previous stages.

[0366] The encoder 4600 also includes an intra prediction component 4608 configured to receive an input video and a motion estimation / compensation (ME / MC) component 4610. The intra prediction component 4608 is configured to perform intra prediction, and the ME / MC component 4610 is configured to perform inter prediction using reference pictures obtained from the reference picture cache 4612. Residual blocks from the inter prediction or intra prediction are fed to a transform (T) component 4614 and a quantization (Q) component 4616 to generate quantized residual transform coefficients, and the quantized residual transform coefficients are fed to an entropy coding component 4618. The entropy coding component 4618 performs entropy coding on the prediction results and the quantized transform coefficients and transmits them to a video decoder (not shown). The quantized components output from the quantization component 4616 can be fed to an inverse quantization (IQ) component 4620, an inverse transform component 4622, and a reconstruction (REC) component 4624. The REC component 4624 is capable of outputting an image to the DF 4602, the SAO 4604, and the ALF 4606 to filter the images before they are stored in the reference picture cache 4612.

[0367] Figure 11 is an image or video method implemented by an encoding / decoding device (e.g., an encoder or a decoder) according to an embodiment of the present disclosure. In block 1102, the encoding / decoding device changes the quantization step size of region adaptive hierarchical transform (RAHT) coefficients in a point cloud video unit. In block 1104, the encoding / decoding device performs a conversion between media data and a media data file based on the quantization step size.

[0368] Next, a list of some example preferred solutions is provided.

[0369] 1. A method for processing media data, comprising: determining quantization based on region adaptive hierarchical transform (RAHT) weights; and performing a conversion between visual media data and a bitstream based on the quantization.

[0370] 2. The method according to solution 1, wherein the quantization step size depends on the weights of the RAHT coefficients.

[0371] 3. The method according to solution 1 or 2, wherein different quantization step sizes are used for RAHT coefficients with different weights, and the quantization step sizes are signaled via a quantization parameter (QP) offset.

[0372] 4. The method according to any one of solutions 1-3, wherein the weights are divided into different groups, and different QP offsets are transmitted for different groups or subsets of groups.

[0373] 5. The method according to any one of solutions 1-4, wherein the quantization step size is determined based on a predefined function, and the function includes where Δ is the quantization step size, K is a constant, and w is the weight of the RAHT coefficient.

[0374] 6. The method according to any one of Solutions 1-5, wherein the step size is restricted between a minimum step size value and a maximum step size value, wherein the value of K is transmitted via a signal, and wherein the function is only for a specific range or value of w, or a combination thereof.

[0375] 7. The method according to any one of Solutions 1-6, wherein weight-based quantization is used instead of octree layer-based adaptive quantization.

[0376] 8. The method according to any one of Solutions 1-7, wherein weight-based quantization is the quantization mode, and wherein additional information is transmitted via a signal to indicate the use of the quantization type.

[0377] 9. The method according to any one of Solutions 1-8, wherein the quantization is determined based on both the layer of the RAHT coefficient and the weight of the RAHT coefficient.

[0378] 10. The method according to any one of Solutions 1-9, wherein the QP offset is determined based on the layer and is further modified based on the weight of the RAHT coefficient, or the quantization step size varies based on a function jointly determined by the layer of the RAHT coefficient and the weight of the RAHT coefficient.

[0379] 11. The method according to any one of Solutions 1-10, wherein different quantization algorithms are applied to different attribute channels.

[0380] 12. The method according to any one of Solutions 1-11, wherein the luminance quantization is based on layer-based adaptive quantization, and the chrominance quantization is based on quantization depending on the weight of the RAHT coefficient, or different functions are adopted based on the weight of the RAHT coefficient for different attribute channels, or the partitioning into different subgroups is different for different channels, and the corresponding transmitted offsets are different.

[0381] 13. The method according to any one of Solutions 1-12, wherein the quantization algorithm is selectively enabled for a set of voxels.

[0382] 14. The method according to any one of Solutions 1-13, wherein different quantization is used for different transform coefficients of the 2×2×2 transform residual block after upsampled prediction.

[0383] 15. The method according to any one of Solutions 1-14, wherein a 2×2×2 QP offset matrix is used to derive the quantization step size, and the 2×2×2 QP offset matrix is specified at the sequence level, picture level, slice level or tile level.

[0384] 16. The method according to any one of Solutions 1-15, wherein the 2×2×2 QP offset matrix depends on the octree layer, and the use of the 2×2×2 QP offset matrix is restricted to the specified layer in the octree, or different three-dimensional (3D) QP offset matrices are used for different layers in the octree.

[0385] 17. The method according to any one of Solutions 1-16, wherein the 2×2×2 QP offset matrix depends on the RAHT weight, and the use of the 2×2×2 QP offset matrix is restricted to the specified subbands of the RAHT, where a subband represents a group of coefficients with the same RAHT weight, or different 3D QP matrices are used for different subbands.

[0386] 18. The method according to any one of Solutions 1-17, wherein the 2×2×2 QP offset matrix depends on the attribute channel, and the 2×2×2 QP offset matrix is only used for a subset of the attribute channels, or different QP offset matrices are used for different attribute channels.

[0387] 19. The method according to any one of Solutions 1-18, wherein the 2×2×2 QP offset matrix depends on the picture region, and the 2×2×2 QP offset matrix is only used for some regions, or different QP offset matrices are used for different regions.

[0388] 20. The method according to any one of Solutions 1-19, wherein only a subset of the coefficients is modified, and coarser quantization is applied to the high-frequency coefficients without modifying the low-frequency coefficients, or the use of partial modification of the coefficients depends on the attribute channel, RAHT weight or octree layer.

[0389] 21. The method according to any one of Solutions 1-20, wherein the use of the 2×2×2 QP offset matrix or a partial matrix depends on the octree layer or the attribute channel.

[0390] 22. The method according to any one of Solutions 1-21, wherein selectively disabling for a group of voxels: using different quantization for different transform coefficients.

[0391] 23. The method according to any one of Solutions 1-22, wherein the use of quantization based on the RAHT weight is signaled in the sequence parameter set (SPS), geometric parameter set (GPS) or adaptive parameter set (APS).

[0392] 24. The method according to any one of Solutions 1 - 23, wherein the use of quantization based on RAHT weights is signaled for a video unit, and the video unit is a slice, a strip, or a strip header.

[0393] 25. The method according to any one of Solutions 1 - 24, wherein a QP offset is signaled for different sub - bands, the QP offset is progressively decoded, a flag is signaled to enable adaptive quantization for different sub - bands, the flag is signaled based on the attribute channel, layer, region, slice, or voxel group to enable the adaptive quantization, the flag is signaled to specify an adaptive quantization function for different frames, sequences, strips, slices, attribute channels, or regions based on a list or table of different functions based on RAHT weights, or a combination thereof.

[0394] 26. The method according to any one of Solutions 1 - 25, wherein a QP matrix or a QP offset matrix is signaled for different RAHT coefficients, the QP offset is progressively decoded, a partial QP matrix is signaled, a flag specifies the use of a full matrix or a partial matrix and the flag depends on the octree layer or region, a flag is signaled to signal a subset of 2×2×2 coefficients to which the QP offset is applied, or a combination thereof.

[0395] 27. An apparatus for processing video data, comprising: a processor; and a non - transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of Solutions 1 - 26.

[0396] 28. A non - transitory computer - readable medium, comprising a computer program product for use by a video codec device, the computer program product comprising computer - executable instructions stored on the non - transitory computer - readable medium, such that the computer - executable instructions, when executed by a processor, cause the video codec device to perform the method according to any one of Solutions 1 - 26.

[0397] 29. A non - transitory computer - readable recording medium storing a bitstream of a video generated by a method performed by a video processing apparatus, wherein the method comprises: determining quantization based on Region Adaptive Hierarchical Transform (RAHT) weights; and generating a bitstream based on the determination.

[0398] 30. A method of storing a bitstream of a video, comprising: determining quantization based on Region Adaptive Hierarchical Transform (RAHT) weights; generating a bitstream based on the determination; and storing the bitstream in a non - transitory computer - readable recording medium.

[0399] 31. A method, apparatus, or system described in this document.

[0400] In the solutions described herein, an encoder can conform to format rules by generating an encoded representation according to the format rules. In the solutions described herein, a decoder can utilize the known information about the presence and absence of syntax elements according to the format rules to parse the syntax elements in the encoded / decoded representation to generate a decoded video.

[0401] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from the pixel representation of a video to the corresponding bitstream representation, and vice versa. For example, the bitstream representation of the current video block can correspond to the co-located position within the bitstream or bits propagated at different positions, as defined by the syntax. For example, macroblocks can be encoded based on the transformed and encoded / decoded error residual values, and bits in the headers and other fields in the bitstream can also be used. Additionally, during the conversion, the decoder can parse the bitstream based on this determination, utilizing the known information that some fields may be present or absent, as described in the above solutions. Similarly, the encoder can determine whether to include or exclude specific syntax fields, and the encoder can generate the encoded / decoded representation accordingly by including or excluding the syntax fields from the encoded / decoded representation.

[0402] The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a substance composition affecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" includes all apparatuses, devices, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus can also include code that creates an execution environment for the computer program being discussed, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or a combination of one or more of them. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0403] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program need not correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-operating files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or on multiple computers, which may be located at one site or distributed across multiple sites and interconnected by a communication network.

[0404] The processes and logical flows described in this document can be performed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The processes and logical flows can also be performed by special-purpose logic circuitry, and apparatuses can be implemented as special-purpose logic circuitry, e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

[0405] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, e.g., magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data therefrom or transfer data thereto, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including by way of example semiconductor storage devices, such as erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), and flash memory devices; magnetic disks, e.g., internal hard disks or removable hard disks; magneto-optical disks; and compact disc read only memory (CD ROM) and digital versatile disc read only memory (DVD-ROM) discs. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0406] Although this patent document contains many details, these details should not be construed as limitations on any subject matter or the scope of what can be claimed, but rather as descriptions of features specific to particular embodiments of a particular technology. In this patent document, certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable sub-combination. Additionally, although features may operate in certain combinations as described above, and even were initially claimed in such a manner, in some cases, one or more features in the claimed combination may be excised from that combination, and the claimed combination may be directed to a sub-combination or a variant of a sub-combination.

[0407] Similarly, although operations are depicted in the drawings in a particular order, this should not be construed as requiring that such operations be performed in the particular order or sequence shown, or that all of the illustrated operations be performed to achieve the desired result. Additionally, the partitioning of various system components in the embodiments described in this patent document should not be construed as requiring such partitioning in all embodiments.

[0408] Only a few implementations and examples have been described, and other implementations, improvements, and variations can be made based on what is described and shown in this patent document.

[0409] When there is no intermediate component other than a line, trace, or other medium between a first component and a second component, the first component is directly coupled to the second component. When there is an intermediate component between the first component and the second component in addition to a line, trace, or other medium, the first component is indirectly coupled to the second component. The term "coupled" and its variants include both direct coupling and indirect coupling. Unless otherwise specified, the use of the term "about" means a range of ±10% of the subsequent number.

[0410] Although several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of this disclosure. The current examples are considered illustrative rather than restrictive, and are not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated in another system, or certain features may be omitted or not implemented.

[0411] In addition, the techniques, systems, subsystems, and methods described and illustrated as discrete or separate in various embodiments may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as being coupled may be directly connected or may be indirectly coupled or communicate through some interface, device, or intermediate component, whether electrical, mechanical, or otherwise. Examples of other changes, substitutions, and alterations can be determined by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Claims

1. A method for processing media data, comprising: changing a quantization step size of region adaptive hierarchical transform (RAHT) coefficients in a point cloud video unit; and performing a conversion between the media data and a media data file based on the quantization step size.

2. The method according to claim 1, wherein The point cloud video unit is one of a video sequence, a frame, a cube, and a block.

3. The method according to claim 1 or 2, wherein The quantization step size depends on a weight of the RAHT coefficients.

4. The method according to any one of claims 1-3, wherein, Different quantization step sizes are used for RAHT coefficients having different weights.

5. The method according to any one of claims 1-4, wherein One or more indicators are used to indicate the different quantization step sizes.

6. The method according to any one of claims 1-5, wherein The one or more indicators are included in a bitstream.

7. The method according to claim 4, wherein The different weights are divided into different groups, and different quantization offsets are transmitted for the different groups.

8. The method according to claim 4, wherein, The different weights are divided into different groups, and different quantization offsets are transmitted for a subset of the different groups.

9. The method according to any one of claims 1-8, wherein The quantization step size of the RAHT coefficients is determined based on a predefined function.

10. The method according to claim 9, wherein, The predefined function includes where Δ is the quantization step, K is a constant, and w is the weight of the RAHT coefficient.

11. The method according to claim 10, wherein, A value of K is included in the bitstream.

12. The method according to claim 10, wherein, K varies based on a level of the point cloud video unit, and the level of the point cloud video unit is a slice or a picture.

13. The method according to claim 9, wherein The predefined function includes where Δ is the quantization step, K is a constant, and w is the weight of the RAHT coefficient.

14. The method according to any one of claims 9 - 13, wherein, The predefined function is for different weights of the RAHT coefficients within a specific range, or for different weights of the RAHT coefficients having a specific value.

15. The method according to any one of claims 1-14 further comprises: The quantization step size is restricted between a minimum quantization step size and a maximum quantization step size.

16. The method according to any one of claims 1-14, further comprising: Weight-based quantization is used to replace octree layer-based adaptive quantization.

17. The method according to claim 16, wherein, The weight-based quantization is one of multiple different types of quantization, and additional information may be included in the bitstream or derived by a decoder to indicate which quantization of the multiple different types of quantization is to be used.

18. The method according to any one of claims 1-17, wherein, The quantization step size is based on both a layer of the RAHT coefficients and a weight of the RAHT coefficients.

19. The method according to claim 18, further comprising: A quantization parameter (QP) offset is determined based on the layer of the RAHT coefficients, and the QP offset is modified based on the weight of the RAHT coefficients.

20. The method according to claim 18 further comprises: The quantization step size is changed based on a function, where the function includes both the layer of the RAHT coefficients and the weight of the RAHT coefficients.

21. The method according to any one of claims 1-20, further comprising applying different quantization methods to different attribute channels.

22. The method according to claim 21, wherein, Layer-based adaptive quantization is applied to a luminance channel, and quantization depending on a weight of the RAHT coefficients is applied to a chrominance channel.

23. The method according to claim 21, wherein, Different functions based on the weight of the RAHT coefficients are applied to the different attribute channels.

24. The method according to claim 21 further comprises: The different attribute channels are divided into different subgroups for the different attribute channels, where offsets corresponding to the different subgroups are different for the different attribute channels.

25. The method according to any one of claims 1-24 further comprises: The point cloud video unit is segmented into different regions, and different quantization methods are applied to the different regions.

26. The method according to claim 25 further comprises: The quantization step size of the RAHT coefficients is changed based on a function, where the function is based on each of a region, a layer, and a weight of the RAHT coefficients.

27. The method according to claim 25 further comprises: Layer-based quantization is applied to some regions of the point cloud video unit, and weight-based quantization is applied to other regions of the point cloud video unit.

28. The method according to claim 25, further comprising: Determine a region-based quantization parameter (QP) offset and modify the region-based QP offset based on the weights of the RAHT coefficients.

29. The method according to claim 25, further comprising: Divide the different regions into different subgroups for the different regions, where the offsets corresponding to the different subgroups are different for the different regions.

30. The method according to any one of claims 1-29 further comprises: Selectively enable and disable the method for a set of voxels.

31. The method according to claim 30, wherein, The set of voxels is a set of 2×2×2 voxels.

32. The method according to any one of claims 1-31 further comprises: Apply different quantization to different transform coefficients after a 2×2×2 transform residual block has been predicted by upsampling.

33. The method according to claim 32, further comprising: Derive the quantization step using a 2×2×2 quantization parameter (QP) offset matrix.

34. The method according to claim 33, wherein, The 2×2×2 QP offset matrix is specified at the sequence level, slice level, or picture level of the bitstream.

35. The method according to claim 33 or 34, wherein The 2×2×2 QP offset matrix is based on an octree.

36. The method according to claim 35, further comprising: Restrict the use of the 2×2×2 QP offset matrix to partial layers in the octree.

37. The method according to claim 35, further comprising: Use different three-dimensional (3D) quantization parameter (QP) matrices for different layers in the octree.

38. The method according to claim 33 or 34, wherein, The 2×2×2 QP offset matrix is based on the weights of the RAHT coefficients.

39. The method according to claim 38, further comprising: Restrict the use of the 2×2×2 QP offset matrix to partial subbands of the RAHT coefficients, where each of the subbands represents a group of coefficients with the same weight of the RAHT coefficients.

40. The method according to claim 38, further comprising: Use different three-dimensional (3D) quantization parameter (QP) matrices for different subbands of the RAHT coefficients.

41. The method according to claim 33 or 34, wherein, The 2×2×2 QP offset matrix is based on an attribute channel.

42. The method according to claim 41, further comprising: Use the 2×2×2 QP offset matrix for a subset of the different attribute channels.

43. The method according to claim 41 further comprises: Use the 2×2×2 QP offset matrix for the different attribute channels.

44. The method according to claim 33 or 34, wherein, The 2×2×2 QP offset matrix is based on different regions.

45. The method according to claim 44, further comprising: Use the 2×2×2 QP offset matrix for a subset of the different regions.

46. The method according to claim 44, further comprising: Use the 2×2×2 QP offset matrix for the different regions.

47. The method according to any one of claims 1-46, further comprising: Modify the quantization of a subset of the 2×2×2 transform coefficients instead of the quantization of all the 2×2×2 transform coefficients, where the 2×2×2 transform coefficients include eight transform coefficients.

48. The method according to claim 47, further comprising: Apply coarser quantization to the high-frequency coefficients without modifying the low-frequency coefficients.

49. The method according to claim 47, wherein Modify the quantization of a subset of the 2×2×2 transform coefficients based on one or more of an attribute channel, the weights of the RAHT coefficients, and the layers of the octree.

50. The method according to any one of claims 1-49 further comprises: Divide the RAHT coefficients into different subgroups and apply different quantization parameter (QP) offsets to the different subgroups.

51. The method according to claim 50, wherein The different subgroups are based on the frequency of the RAHT coefficients.

52. The method according to any one of claims 1-51 further comprises: Provide options to use a full 2×2×2 QP offset matrix, a partial 2×2×2 QP offset matrix, or subgroup-based offsets, where the options are based on one or more of the layers of the octree and the attribute channel.

53. The method according to any one of claims 1-52, further comprising: Use a function or a set of functions to associate the quantization parameter (QP) offset with a function of one or more of the layers of the octree and the transform coefficient matrix.

54. The method according to claim 53, wherein, The function or the set of functions is indicated to the decoder.

55. The method according to claim 53, wherein The function or the set of functions is different for different attribute channels.

56. The method according to claim 53, wherein The function or the set of functions is predefined.

57. The method according to any one of claims 1-56, wherein, The quantization parameter (QP) offset depends on one or more of the original point cloud resolution or the display resolution at the client side.

58. The method according to any one of claims 1 - 57, wherein, Quantization for a larger display size is finer than that for a smaller display size.

59. The method according to any one of claims 1-58, wherein, The quantization parameter (QP) offset for chrominance is based on the chrominance subsampling employed.

60. The method according to claim 59, wherein, Quantization for 4:4:4 content is coarser than that for 4:4:4 content.

61. The method according to any one of claims 1-60 further comprises: When the direct current (DC) coefficient has been inherited from a previous layer of the octree, the previously disclosed offset is applied only to the alternating current (AC) coefficients.

62. The method according to any one of claims 1-60 further comprises: The previously disclosed offset is transmitted to the decoder, where, when the direct current (DC) coefficient has been inherited from a previous layer of the octree, the previously disclosed offset is applied only to the alternating current (AC) coefficients.

63. The method according to any one of claims 1 - 62, wherein, One or more syntax elements are included in the bitstream to indicate the use of any of the methods described above.

64. The method according to claim 63, wherein, The one or more syntax elements include flags.

65. The method according to claim 63, wherein, The one or more syntax elements are included in the syntax structure of a parameter set in the bitstream.

66. The method according to claim 63, wherein, The one or more syntax elements are included in the sequence parameter set (SPS) in the bitstream.

67. The method according to claim 63, wherein, The one or more syntax elements are included in the group parameter set (GPS) in the bitstream.

68. The method according to claim 63, wherein, The one or more syntax elements are included in the adaptive parameter set (APS) in the bitstream.

69. The method according to claim 63, wherein, The one or more syntax elements are included in the part of the bitstream corresponding to a slice.

70. The method according to claim 63, wherein, The one or more syntax elements are included in the part of the bitstream corresponding to a strip.

71. The method according to claim 70, wherein, The one strip includes an attribute strip or a geometry strip.

72. The method according to claim 63, wherein, The one or more syntax elements are included in the strip header.

73. The method according to claim 72, wherein, The strip in the strip header includes an attribute strip or a geometry strip.

74. The method according to any one of claims 1-73, wherein, The bitstream includes one or more flags for quantization using the weighted RAHT coefficients, the one or more flags including: A flag indicating the QP offset for different subbands, where the QP offset is binarized using fixed-length coding, exponential Golomb coding, or truncated unary coding; A flag indicating that the QP offset is coded progressively, where progressive coding is defined as the difference between the QP offset of the current subband and the QP offset of the previous subband; A flag indicating whether the quantization method is enabled or disabled for different subbands; A flag indicating whether adaptive quantization is enabled or disabled, where the flag is set based on one or more of an attribute channel, a layer, a region of the point cloud video unit, a slice, or a set of voxels; A flag indicating a function of different frames, sequences, strips, slices, attribute channels, or regions of the point cloud video unit, where a list or table of different functions providing weights based on the RAHT coefficients is provided; and A flag indicating different function parameters for different frames, sequences, strips, slices, attribute channels, or regions of the point cloud video unit, where the one or more flags are context-coded or bypass-coded.

75. The method according to any one of claims 1 to 73, wherein, The bitstream includes one or more flags related to using different quantization for different transform coefficients in a 2×2×2 transform residual block after upsampled prediction, the one or more flags including: A flag indicating different RAHT coefficients of a residual block corresponding to the 2×2×2 transform for a 2×2×2 QP matrix or a QP offset matrix, wherein the entries of the 2×2×2 QP matrix or the QP offset matrix are binarized using fixed-length coding, exponential Golomb coding, or truncated unary coding; A flag indicating that the QP offset is coded progressively, where progressive coding is defined as the difference between the QP offset of the current RAHT coefficient and the QP offset of the previous RAHT coefficient; A flag indicating a partial 2×2×2 matrix instead of a full 2×2×2 matrix; A flag indicating the QP offset for each subgroup; A flag indicating that the QP offset is coded progressively with respect to the levels of the octree, where progressive coding with respect to the octree is defined as the difference between the QP offset of the current level and the QP offset of the previous level; A flag indicating that the QP offset is coded progressively with respect to the attribute channel, where progressive coding with respect to the attribute channel is defined as the difference between the QP offset of the current attribute channel and the QP offset of the previous attribute channel; A flag indicating the use of a partial matrix or a full matrix, where the flag is based on one or more of the levels of the octree, the region of the point cloud video unit, and the flag is included at a level of the bitstream corresponding to a sequence, slice, tile, or region of the point cloud video unit; A flag indicating a subset of the 2×2×2 coefficients to which the QP offset is applied; and A flag indicating a function for determining the relationship between the QP offset and the levels of the octree or the transform coefficient matrix; wherein the one or more flags are context-coded or bypass-coded; When the DC coefficient is inherited from a previous level of the octree, the QP offset is signaled only for the AC coefficients; and The one or more flags are set to values that enable or disable the use of the QP offset for different levels, attribute channels, or subbands of the octree.

76. The method according to any one of claims 1-75, wherein, The specified 2×2×2 is replaced by M×N×K, where M, N, and K are all integers.

77. The method according to any one of claims 1-76, wherein, One or more of the QP matrix and the QP offset are included at different levels of the bitstream.

78. The method according to claim 77, wherein One or more of the first QP matrix are included at a first level of the bitstream, and the first QP offset is included at a second level of the bitstream, where the first level is higher than the second level.

79. The method according to claim 77, wherein One or more of the second QP matrix and the second QP offset are predicted based on the first QP matrix and the first QP offset of the point cloud video unit at the second level of the bitstream.

80. The method according to claim 77, wherein, One or more of the first QP matrix and the first QP offset are covered by the second QP matrix and the second QP offset of the point cloud video unit at the second level of the bitstream.

81. The method according to claim 77, wherein, One or more of the first QP matrix and the first QP offset are used together with the second QP matrix and the second QP offset to derive the final QP matrix and the final QP offset of the point cloud video unit at the second level of the bitstream.

82. The method according to claim 81, wherein, The first QP matrix and the first QP offset are added to the second QP matrix and the second QP offset.

83. The method according to any one of claims 1-82, wherein, The syntax element in any of the methods is binarized into a flag, fixed-length coding / decoding, exponential Golomb (EG) coding / decoding, unary coding / decoding, truncated unary coding / decoding, or truncated binary coding / decoding, and the syntax element is signed or unsigned.

84. The method according to any one of claims 1-82, wherein, The syntax element in any of the methods is coded / decoded or bypassed using at least one context model.

85. The method according to any one of claims 1-82, wherein, The syntax element in any of the methods is conditionally included in the bitstream or signaled only when the function corresponding to the syntax element is applicable.

86. The method according to any one of claims 1-82, wherein The syntax element in any of the methods is included at the block level, octree level, sequence level, group of pictures level, picture level, slice level, point cloud video unit level region in the bitstream, or is included in the coding structure of a coding unit (CU), the coding structure of a prediction unit (PU), sequence header, picture header, sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), or slice header.

87. The method according to any one of claims 1-82, wherein, Whether or how to apply any of the methods is included at the block level, octree level, sequence level, group of pictures level, picture level, slice level, point cloud video unit level region in the bitstream, or is included in the coding structure of a coding unit (CU), the coding structure of a prediction unit (PU), sequence header, picture header, sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS), or slice header.

88. The method according to any one of claims 1 to 87, wherein, The transformation includes encoding the media data into a bitstream.

89. The method according to any one of claims 1-87, wherein, The transformation includes decoding the media data from the bitstream.

90. A device for processing media data, comprising: A processor; And a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1-89.

91. A non-transitory computer-readable medium includes a computer program product for use by a video codec device, wherein, The computer program product includes computer-executable instructions stored on the non-transitory computer-readable medium, and the computer-executable instructions, when executed by a processor, cause the video coding / decoding device to perform the method according to any one of claims 1-89.

92. A non-transitory computer-readable recording medium stores a bitstream of a video generated by a method executed by a video processing device, wherein, The method includes the method according to any one of claims 1-89.

93. A method for storing a bitstream of video, including the method according to any one of claims 1-89.

94. A method, apparatus, or system described in the present disclosure.

Images