Method and apparatus for quantification and inverse quantification design in video encoding and decoding

By determining quantification parameters and performing bit shifts for coding units in video encoders and decoders, the method addresses the challenge of efficient video compression in evolving standards, enhancing encoding efficiency and maintaining quality.

JP7594063B2Active Publication Date: 2024-12-03BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2023136408
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-09-23
Filing Date
2023-08-24
Publication Date
2024-12-03
Estimated Expiration
2040-09-23

AI Technical Summary

Technical Problem

Existing video encoding and decoding techniques face challenges in compressing video data to lower bit rates without degrading video quality, particularly in the context of evolving standards like Versatile Video Coding (VVC) that require more efficient quantification and inverse quantification designs.

Method used

The method involves determining quantification parameters for residual data of a coding unit (CU), scaling with a scale factor, and performing bit shifts to derive coefficients and quantification levels, using processors and memory to execute these operations in video encoders and decoders.

Benefits of technology

This approach enhances video encoding efficiency by optimizing quantification and inverse quantification processes, thereby improving compression without significant quality loss, aligning with advanced standards like VVC.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007594063000030
    Figure 0007594063000030
  • Figure 0007594063000031
    Figure 0007594063000031
  • Figure 0007594063000032
    Figure 0007594063000032
Patent Text Reader

Abstract

To provide video coding / decoding and compression methods and devices for quantization and de-quantization design in video coding / decoding.SOLUTION: A method includes: determining a quantization parameter for residual data of a coding unit (CU); deriving a scale value by scaling the quantization parameter by a scale factor; determining a plurality of coefficients associated with the CU; determining a plurality of parameters associated with the CU; obtaining a plurality of bit shifts by bit-shifting the plurality of parameters; and obtaining a quantized level based on the scale value, the plurality of coefficients and the plurality of bit shifts.SELECTED DRAWING: Figure 11
Need to check novelty before this filing date? Find Prior Art

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 62 / 904,668, filed September 22, 2019, the entire disclosure of which is incorporated herein by reference. [Technical field]

[0002] This application relates generally to video encoding / decoding and compression, and more particularly, but not exclusively, to methods and apparatus for quantification and inverse quantification design in video encoding / decoding. [Background technology]

[0003] A variety of video encoding and decoding techniques can be used to compress video data. Video encoding and decoding is performed according to one or more video encoding and decoding standards. Video encoding and decoding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding and Decoding (H.265 / HEVC), Advanced Video Coding and Decoding (H.264 / AVC), Moving picture Experts Group (MPEG) encoding and decoding, etc. Video encoding and decoding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) due to redundancy present in a video image or sequence. An important goal of video encoding and decoding techniques is to compress video data into a lower bitrate format while avoiding or minimizing the degradation of video quality. Summary of the Invention

[0004] This disclosure generally describes example techniques relating to quantification and inverse quantification design in video encoding and decoding.

[0005] According to a first aspect of the present disclosure, there is provided a video encoding method applicable to an encoder, including determining a quantification parameter for residual data of a coding unit (CU), deriving a scale value by scaling the quantification parameter with a scale factor, determining a number of coefficients associated with the CU, determining a number of parameters associated with the CU, bit-shifting the number of parameters to obtain a number of bit shifts, and obtaining a quantification level based on the scale value, the number of coefficients, and the number of bit shifts.

[0006] According to a second aspect of the present disclosure, there is provided a video encoding method for application to a decoder, the method including: receiving a video bitstream including a quantification parameter and a quantification level; deriving a scale value by scaling the quantification parameter with a scaling factor; determining a plurality of parameters associated with a coding unit (CU); bit-shifting the plurality of parameters to obtain a plurality of bit shifts; and obtaining a plurality of coefficients associated with the CU based on the quantification level, the scale value and the plurality of bit shifts.

[0007] According to a third aspect of the present disclosure, there is provided a computing device including one or more processors, a non-transient memory coupled to the one or more processors, and a plurality of programs stored in the non-transient memory, which, when executed by the one or more processors, cause the one or more processors to perform operations such as determining a quantification parameter for residual data of a CU, deriving a scale value by scaling the quantification parameter with a scale factor, determining a plurality of coefficients associated with the CU, determining a plurality of parameters associated with the CU, bit-shifting the plurality of parameters to obtain a plurality of bit shifts, and obtaining a quantification level based on the scale value, the plurality of coefficients, and the plurality of bit shifts.

[0008] According to a fourth aspect of the present disclosure, there is provided a computing device including one or more processors, a non-transitory memory connected to the one or more processors, and a plurality of programs stored in the non-transitory memory, the plurality of programs, when executed by the one or more processors, causing the one or more processors to perform operations such as receiving a video bitstream including a quantification parameter and a quantification level, deriving a scale value by scaling the quantification parameter with a scaling factor, determining a plurality of parameters associated with a CU, bit-shifting the plurality of parameters to obtain a plurality of bit shifts, and obtaining a plurality of coefficients associated with the CU based on the quantification level, the scale value, and the plurality of bit shifts.

[0009] According to a fifth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing a plurality of programs executed by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, causing the one or more processors to perform operations such as determining a quantification parameter for residual data of a CU, deriving a scale value by scaling the quantification parameter by a scale factor, determining a plurality of coefficients associated with the CU, determining a plurality of parameters associated with the CU, bit-shifting the plurality of parameters to obtain a plurality of bit shifts, and obtaining a quantification level based on the scale value, the plurality of coefficients, and the plurality of bit shifts.

[0010] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing a plurality of programs executed by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, causing the one or more processors to perform operations such as receiving a video bitstream including a quantification parameter and a quantification level, deriving a scale value by scaling the quantification parameter with a scaling factor, determining a plurality of parameters associated with a CU, bit-shifting the plurality of parameters to obtain a plurality of bit shifts, and obtaining a plurality of coefficients associated with the CU based on the quantification level, the scale value and the plurality of bit shifts. [Brief description of the drawings]

[0011] A more particular description of the examples of the present disclosure will be given with reference to the specific examples illustrated in the accompanying drawings, which illustrate only some examples and, therefore, are not to be considered limiting in scope, and will be described with additional specificity and detail by the use of the accompanying drawings.

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example video encoder according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example video decoder according to an embodiment of this disclosure. [Diagram 3] FIG. 3 illustrates an image divided into Coding Tree Units (CTUs) according to one embodiment of this disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating a multi-type tree partitioning mode according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 illustrates a mechanism for signaling partition division information in a quadtree having a nested multi-type tree coding tree structure according to an embodiment of the present disclosure. [Figure 6] FIG. 6 illustrates a CTU divided into multiple CUs in a quadtree with a nested multi-type tree coding block structure according to an embodiment of the present disclosure. [Figure 7] FIG. 7 illustrates an example of a palette mod encoded and decoded block according to an embodiment of the present disclosure. [Figure 8] 8A and 8B illustrate horizontal and vertical traverse scanning according to one embodiment of the present disclosure. [Figure 9] FIG. 9 illustrates encoding of a palette index according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a block diagram illustrating an example apparatus for video encoding and decoding according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a flowchart illustrating an example process of quantification design in video encoding and decoding according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a flowchart illustrating an example process of inverse quantification design in video encoding and decoding according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth in order to facilitate an understanding of the principles described herein. However, it will be apparent to one skilled in the art that various modifications can be implemented. For example, it will be apparent to one skilled in the art that the principles described herein can be implemented in many types of electronic devices having digital video capabilities.

[0014] Reference herein to "one embodiment," "embodiment," "example," "an embodiment," "an example," or similar expressions means that a particular feature, structure, or characteristic described is included in at least one embodiment or example. A feature, structure, element, or characteristic described in connection with one or more embodiments may also be applicable to other embodiments, unless expressly stated otherwise.

[0015] Throughout this disclosure, all terms such as "first," "second," "third," etc. are used solely to refer to associated elements, e.g., devices, components, components, steps, etc., and do not imply any spatial or temporal order, unless the context clearly dictates otherwise. For example, a "first device" and a "second device" may refer to two separately formed devices, or two parts, components, or operational states of the same device, which may be arbitrarily named.

[0016] The terms "module," "sub-module," "circuit," "sub-circuit," "circuit system," "sub-circuit system," "unit," or "sub-unit" include memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits that may or may not store code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. These components may be physically connected to each other, located next to each other, or not.

[0017] As used herein, the terms "if" or "if" can be understood to mean "when" or "depending on," depending on the context. When these terms appear in the claims, they may not imply that the associated limitation or feature is conditional or optional. For example, a method may include steps: i) if condition X exists, function or operation X' is performed; and ii) if condition Y exists, function or operation Y' is performed. The method must be implemented with both the ability to perform function or operation X' and the ability to perform function or operation Y'. Thus, both functions X' and Y' can be implemented in multiple executions of the method at different times.

[0018] A unit or module may be fully implemented in software, fully implemented in hardware, or implemented by a combination of hardware and software. In a fully software implementation, for example, a unit or module may include functionally related code blocks or software components that are directly or indirectly linked together to perform specific functions.

[0019] 1 is a block diagram illustrating an example video encoder according to an embodiment of this disclosure. In the encoder 100, a video frame is partitioned into multiple video blocks for processing. A prediction is formed for each particular video block based on an inter-prediction approach or an intra-prediction approach. In inter-prediction, one or more predictors are formed based on pixels from a previously reconstructed frame by motion estimation and motion compensation. In intra-prediction, a predictor is formed based on reconstructed pixels in a current frame. A mode decision can select an optimal predictor for predicting a current block.

[0020] The transform circuit 102 receives a prediction residual representing the difference between the current video block and its predictor, and the quantification circuit 104 receives the transform coefficients from the transform circuit 102 for entropy reduction. The quantified coefficients are then provided to the entropy coding circuit 106 to generate a compressed video bitstream. As shown in FIG. 1, prediction related information 110, such as video block partition information, motion vectors, reference picture indexes, and intra prediction modes from the inter prediction circuit and / or intra prediction circuit 112, is also provided via the entropy coding circuit 106 and stored in the compressed video bitstream 114.

[0021] The encoder 100 also requires decoder-related circuitry to reconstruct pixels for prediction purposes. First, inverse quantification 116 and inverse transform circuitry 118 reconstructs a prediction residual. This reconstructed prediction residual is combined with a block predictor 120 to generate unfiltered reconstructed pixels for the current video block.

[0022] To further improve coding efficiency and visual quality, in-loop filters are used. For example, the current versions of AVC, HEVC, and VVC provide a deblocking filter. In HEVC, an additional in-loop filter called SAO (Sample Adaptive Offset) is defined to further improve coding efficiency. In the current version of the VVC standard, yet another in-loop filter called ALF (Adaptive Loop Filter) is actively researched and is likely to be included in the final standard.

[0023] FIG. 2 is a block diagram showing an exemplary block-based video decoder 200 that can be used in combination with many video encoding and decoding standards. The decoder 200 is similar to the reconstruction-related parts of the encoder 100 of FIG. 1. In the decoder 200, an input video bitstream 201 is first decoded by an entropy decoding 202 to derive quantified coefficient levels and prediction-related information. The quantified coefficient levels are then processed by an inverse quantification 204 and an inverse transform 206 to obtain a reconstructed prediction residual. A block predictor mechanism implemented in an intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. The reconstructed prediction residual from the inverse transform 206 and the prediction output generated by the block predictor mechanism are summed by a summing unit 214 to obtain a set of unfiltered reconstructed pixels. If an in-loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to derive a final reconstructed video output 222.

[0024] The video coding / decoding standards mentioned above, such as VVC, JEM, HEVC, MPEG-4, Part 10, are conceptually similar. For example, they all use block-based processing. The Joint Video Experts Team (JVET) defined the first draft of Versatile Video Coding (VVC) and the VVC Test Model 1 (VTM1) coding method at the JVET meeting. It was decided that a quadtree with nested multi-type trees with binary and ternary split coding block structures was included as the first new coding feature of VVC. In a quadtree, a parent node can be split into four child nodes, and each child node can become another parent node that is split into four new child nodes.

[0025] In VVC, an image partition structure divides the input video into blocks called coding tree units (CTUs). The CTUs are divided into coding units (CUs) by a quadtree with a nested multi-type tree structure, where a leaf CU defines a region that shares the same prediction mode (e.g., intra or inter). In this disclosure, the term "unit" defines a region of an image that covers all components. The term "block" is intended to define a region that covers a particular component (e.g., luma), which may differ in spatial location when considering chroma sampling formats such as 4:2:0. Partitioning of the CTU in the image

[0026] An image is divided into a set of coding tree units (CTUs). The concept of a CTU is the same as that of HEVC. In the case of an image with three sample arrays, a CTU consists of an N×N luma sample block and two corresponding chroma sample blocks. Figure 3 shows an image divided into CTUs according to an embodiment of the present disclosure. The maximum allowed size of a luma block in a CTU is specified as 128×128. The maximum size of a luma transform block is 64×64. Compartmentalization of CTU using a tree structure

[0027] In HEVC, a CTU is partitioned into CUs by a quadtree structure, denoted as a coding tree, to adapt to various local characteristics. At the leaf CU level, it is decided whether to code an image region using inter-picture (temporal) or intra-picture (spatial) prediction. Each leaf CU can be further partitioned into one, two, or four PUs according to a prediction unit (PU) partition type. Within one PU, the same prediction process is applied, and related information is sent to the decoder on a PU-by-PU basis. After obtaining residual blocks by applying a prediction process based on the PU partition type, a leaf CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree for this CU. One of the main features of the HEVC structure is that there are multiple partition concepts, including CUs, PUs, and TUs.

[0028] In VVC, a quadtree with nested multi-type trees with binary and ternary split segmentation structures replaces the concept of multiple partition unit types. For example, it removes the division of the concepts of CU, PU, ​​and TU, except for CUs where the size of the maximum transform length is too large, and supports more flexibility in CU partition shapes. The coding tree structure allows CUs to have square or rectangular shapes. The CTUs are first partitioned by a quadtree structure. Then, the leaf nodes of this quadtree can be further partitioned by a multi-type tree structure.

[0029] FIG. 4 is a schematic diagram showing a multi-type tree split mode according to an embodiment of the present disclosure. As shown in FIG. 4, in the multi-type tree structure, there are four split types: vertical binary split (SPLIT_BT_VER), horizontal binary split (SPLIT_BT_HOR), vertical ternary split (SPLIT_TT_VER), and horizontal ternary split (SPLIT_TT_HOR). The leaf node of the multi-type tree is called a coding unit (CU), and the CU is used for prediction and transformation processing without further partitioning, as long as the maximum transform length is not too large. So, in the quadtree with nested multi-type tree coding block structure, CU, PU, ​​and TU can have the same block size. The exception is when the maximum supported transform length is smaller than the width or height of the color component of the CU.

[0030] FIG. 5 illustrates a signaling mechanism for partition split information in a quadtree with nested multi-type tree coding tree structure according to an embodiment of the present disclosure. The CTU is treated as the root of the quadtree and is first split by the quadtree structure. Then, each leaf node of the quadtree is further split by the multi-type tree structure as long as it is large enough to be allowed. In the multi-type tree structure, a first flag (mtt_split_cu_flag) is signaled to indicate whether the node is further split. If the node is further split, a second flag (mtt_split_cu_vertical_flag) is signaled to indicate the split direction, and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is a binary or ternary split. Based on the values ​​of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of the CU is derived as shown in Table 1. [Table 1]

[0031] FIG. 6 illustrates a CTU divided into multiple CUs in a quadtree with nested multi-type tree coding block structure according to an embodiment of the present disclosure. As shown in FIG. 6, the thick block edges represent quadtree partitions, and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive coding tree structure of CUs. The size of the CUs can be as large as the CTU or as small as 4×4 in units of luma samples. For 4:2:0 chroma format, the maximum chroma coding block (CB) size is 64×64, and the minimum chroma CB size is 2×2.

[0032] In VVC, the maximum supported luma transform size is 64 x 64, and the maximum supported chroma transform size is 32 x 32. If the width or height of the CB is larger than the maximum transform width or height, the CB is automatically split horizontally and / or vertically to fit the transform size limit in that direction.

[0033] The following parameters are defined and specified by the Sequence Parameter Set (SPS) syntax element for the quadtree with nested multi-type tree coding tree scheme:

[0034] CTU size: the size of the root node of the quadtree;

[0035] MinQTSize: The minimum allowed quadtree leaf node size;

[0036] MaxBtSize: The maximum allowed binary tree root node size;

[0037] MaxTtSize: the maximum allowed ternary tree root node size;

[0038] MaxMttDepth: quadtree leaves of multi-type tree partitions node is the maximum permitted hierarchical depth from

[0039] MinBTSize: The minimum allowed binary tree leaf node size;

[0040] MinTtSize: The minimum allowed ternary tree leaf node size.

[0041] In one example of a quadtree with nested multi-type tree coding tree structure, the CTU size is set to 128x128 luma samples with two corresponding 64x64 blocks of 4:2:0 chroma samples. MinQTSize is set to 16x16, MaxBtSize is set to 128x128, and MaxTtSize is set to 64x64. Both the width and height of MinBtSize and MinTtSize are set to 4x4, and MaxMttDepth is set to 4. The quadtree partition is first applied to the CTU to generate quadtree leaf nodes. The quadtree leaf nodes can have sizes from 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). If a leaf QT node is 128x128, it cannot be further divided by the binary tree because this size exceeds MaxBtSize and MaxTtSize (i.e., 64x64). Otherwise, the leaf QT node can be further partitioned by the multitype tree. Thus, the quadtree leaf node is also the root node of the multitype tree and has a multitype tree depth of 0, i.e., MttDepth. Once this multitype tree depth reaches MaxMttDepth (i.e., 4), no further splits are considered. Once the multitype tree node has a width equal to MinBtSize and less than or equal to 2*MinTtSize, no further horizontal splits are considered. Similarly, once the multitype tree node has a height equal to MinBtSize and less than or equal to 2*MinTtSize, no further vertical splits are considered.

[0042] 7 illustrates an example of a palette mod coded / decoded block according to an embodiment of the present disclosure. In a hardware VVC decoder, to allow for a 64×64 luma block and a 32×32 chroma pipeline design, TT splitting is prohibited if the width or height of the luma coding block is greater than 64, as shown in FIG. 7. TT splitting is also prohibited if the width or height of the chroma coding block is greater than 32.

[0043] In VVC, the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. This CU partition structure is called a dual tree structure or a dual coding tree structure. A CU partition structure shared by both luma and chroma is called a single tree structure or a single coding tree structure. For P and B slices, the luma and chroma coding tree blocks (CTBs) in one CTU must share the same coding tree structure. However, for I slices, luma and chroma can have separate block tree structures. When the separate block tree mode is applied, the luma CTB is partitioned into CUs by one coding tree structure, and the chroma CTB is partitioned into chroma CUs by another coding structure. This means that a CU in an I slice consists of one coding block of luma component or two coding blocks of chroma components, and a CU in a P or B slice always consists of coding blocks of all three color components, unless the video is monochrome. Core conversion Multi-conversion selection (MTS)

[0044] In addition to the DCT-II used in HEVC, the MTS scheme is used for residual coding of both inter-coded and intra-coded blocks. It uses several transforms selected from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table 2 shows the selected DST / DCT basis functions. [Table 2]

[0045] To maintain the orthogonality of the transform matrices, they are quantized more precisely than in HEVC: after horizontal and vertical transforms, all coefficients have 10 bits to keep intermediate values ​​of the transformed coefficients within the 16-bit range.

[0046] To control the MTS scheme, separate enable flags are specified for intra and inter, respectively, at the SPS level. When MTS is enabled at the SPS, a CU level flag is signaled to indicate whether MTS applies or not. Here, MTS applies only to luma. The MTS CU level flag is signaled when the following conditions are met: first, both width and height are less than or equal to 32, and second, the CBF flag is equal to 1.

[0047] If the MTS CU flag is equal to 0, DCT2 is applied in both directions. However, if the MTS CU flag is equal to 1, two other flags are additionally signaled, indicating the horizontal and vertical transform types, respectively. The transform and signaling mapping table is shown in Table 3. A unified transform selection for ISP and implicit MTS is used by removing the dependency on intra mode and block shape. If the current block is in ISP mode, or if the current block is an intra block and intra and inter explicit MTS are on, only DST7 is used for both horizontal and vertical transform cores. For the precision of the transform matrix, an 8-bit primary transform core is used. Therefore, all transform cores used in HEVC remain the same, including 4-point DCT2 and DST7, 8-point, 16-point and 32-point DCT2. Also, other transform cores, including 64-point DCT2, 4-point DCT8, 8-point, 16-point, 32-point DST7 and DCT8, use 8-bit primary transform cores. [Table 3]

[0048] To reduce the complexity of large sizes of DST7 and DCT8, the high frequency transform coefficients are zeroed for DST7 and DCT8 blocks whose size (width or height, or both width and height) is equal to 32. Only the coefficients in the 16x16 low frequency region are retained.

[0049] The residual of the block can be coded in transform skip mode, as in HEVC. To avoid redundancy in syntax coding, the transform skip flag is signaled if the CU level MTS_CU_flag is not equal to zero. The block size restriction for transform skip is the same as for MTS in JEM4, indicating that transform skip is applicable to the CU if both the width and height of the block are less than or equal to 32. Note that the implicit MTS transform is set to DCT2 if LFNST or MIP is activated for the current CU. Also, the implicit MTS can be enabled even if MTS is enabled for inter-coded blocks. Quantification Parameter Control

[0050] In one example, the maximum QP may be extended from 51 to 63, and the initial quantification parameter (QP) signaling may be modified accordingly. When a nonzero value of slice_qp_delta is coded, the initial value of SliceQpY is modified at the slice segment layer. Specifically, the value of init_qp_minus26 may be modified to range from (-26+QpBdOffsetY) to +37. If the size of the transform block is not a power of 4, the transform coefficients are processed with modifications to the QP or QP levelScale table instead of multiplying by 181 / 256 (or 181 / 128) to compensate for the implicit scaling due to the transform process. For transform skip blocks, the minimum allowed QP is defined as 4, since a QP equal to 4 results in a quantification step size of 1.

[0051] In one example, a fixed lookup table is used to convert the luma quantification parameter QPY to the chroma quantification parameter QPC. In VVC, a more flexible luma-chroma QP mapping is used. Instead of using a fixed table, in SPS, which uses a flexible piecewise linear model and the only constraint on the linear model is that the slope of each piece must not be negative, the luma-chroma QP mapping relationship is signaled. That is, as the luma QP increases, the chroma QP must stay flat or increase, but cannot decrease. A piecewise linear model is defined by 1) the number of pieces in the model; 2) the input (luma) and output (chroma) delta QP for that piece. The input range of this piecewise linear model is [-QpBdOffsetY, 63], and the output range of this piecewise linear model is [-QpBdOffsetC, 63]. The QP mapping relationship can be signaled separately for Cb, Cr, and joint Cb / Cr coding, or jointly for all three types of residual coding.

[0052] Similar to HEVC, CU level QP adaptation is allowed in VVC. Delta QP values ​​for luma and chroma components can be signaled separately. For chroma components, the allowed chroma QP offset values ​​are signaled in the form of offset lists in a picture parameter set (PPS) in a similar manner to HEVC. Lists are defined separately for Cb, Cr, and joint Cb / Cr coding. For each of the Cb, Cr, and joint Cb / Cr lists, up to six offset values ​​are allowed. At the CU level, an index is signaled indicating which offset value in the offset list is used to adjust the chroma QP for that CU. Quantification and dequantification of conversion coefficients

[0053] In the transformation process, a shift operation is required because during the quantification and dequantification processes the resulting coefficients are scaled by a certain factor. The scaling factor is defined as:

number

[0054] In one example, the block dimensions are a power of two, so M=2 m and N=2 n This means that if M is equal to N, or indeed if M·N is a power of 4, then the scaling factor can be applied by a right shift. A block that satisfies this condition is called a "normal block". If M·N is not a power of 4, different scale and shift values ​​are used for compensation. The scale values ​​are defined as in Table 4. A block that satisfies this condition is called a "compensated block". [Table 4]

[0055] For blocks coded in transform skip mode, the scale and shift operations defined for normal blocks are performed.

[0056] The derivation of the scaled conversion factors is shown in Table 5. The definitions of all variables in Table 5 can be found in Version 6 of the VVC draft specification. [Table 5]

[0057] If the QP value is used, the quantification and dequantification process of the transform coefficients can be written as follows: The quantification can be expressed by equation (2).

number

[0058] The inverse quantification can be expressed by equation (3).

number

number

[0059] The variable rectNonTsFlag represents whether the current block is classified as a "normal block" or a "compensation block". If it has a value of false or 0, the current block is classified as a normal block. If it has a value of true or 1, the current block is classified as a compensation block.

[0060] The variable transformShift is

number

[0061] In the transform skip mode, the prediction residual is directly quantified and coded without performing a transform operation. More specifically, the quantification and dequantification process can be described as follows:

[0062] The quantification can be expressed by equation (6).

number

number

[0063] VTM6 supports palette mode for screen content coding in 4:4:4 color format. If palette mode is enabled and the CU size is 64x64 or less, a flag is signaled at the CU level indicating whether palette mode is used for this CU. Palette mode is signaled as a prediction mode other than intra prediction, inter prediction, and intra block copy (IBC) modes.

[0064] When palette mode is used for a CU, the sample values ​​in the CU can be represented by a small set of representative color values. This set is called the "palette". For pixels with values ​​close to the palette colors, a palette index may be signaled to convey those values ​​to the decoder. escape Following the symbol index escape By signaling a value, it is also possible to specify samples whose values ​​are not close to any palette color. escape The values ​​are the quantified component values ​​of the sample, which are shown in FIG.

[0065] escape For quantified and unquantified values, the following equations show the corresponding operations applied at the encoder and decoder, respectively:

[0066] The quantification can be expressed by equation (8).

number

[0067] The non-quantification can be expressed by equation (9).

number

[0068] In palette encoding, a palette predictor containing a list of colors is maintained. This predictor is initialized to 0 (i.e. an empty list) at the beginning of each slice in case of non-wavefronts and at the beginning of each CTU row in case of wavefronts. For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette of this CU. This reuse flag is signaled by a run-length code of zeros. After this, the number of new palette entries and the component values ​​of the new palette entries are signaled. After encoding a CU in palette mode, the palette predictor is updated with the current palette and the entries from the palette predictor that are not reused in the current palette are added to the end to form a new palette predictor, up to the maximum allowed palette size. In the current CU escapeOne for each CU to indicate whether a symbol is present escape The flag is notified. escape If the symbol exists, the palette table is extended by one and the last index is escape are assigned to represent symbols.

[0069] 8A and 8B show horizontal and vertical traverse scanning according to an embodiment of the present disclosure. The palette indexes of samples in a CU form a palette index map. The index map is coded by horizontal and / or vertical traverse scanning as shown in FIG. 8A and 8B. The scanning order is explicitly signaled in the bitstream by palette_transpose_flag.

[0070] FIG. 9 illustrates encoding of palette index according to an embodiment of the present disclosure. Palette index is encoded by two main palette sample modes: INDEX and COPY_ABOVE. This mode is signaled by one flag, except for the first row where only horizontal scanning is used, the first column where only vertical scanning is used, or palette sample positions where the previous mode was COPY_ABOVE. In COPY_ABOVE mode, the palette index of the sample in the row above is copied. In INDEX mode, the palette index is explicitly signaled. In both INDEX and COPY_ABOVE modes, a run value is signaled that specifies the number of pixels to be encoded by the same corresponding mode.

[0071] The coding order for the index map is as follows: first, the number of index values ​​associated with an INDEX run is signaled. Following this, the actual index values ​​for the entire CU are signaled in a truncated binary encoding. Next, the palette mode (INDEX or COPY_ABOVE) and the run length of each run are signaled in an interleaved fashion. Finally, the quantified sum of the entire CU is escapeThe modal colors are grouped and encoded with Exponential Golomb encoding.

[0072] For slices with a bi-luminance / chroma tree, the palette is applied separately to the luma (Y component) and chroma (Cb and Cr components). For single-tree slices, the palette is applied jointly to the Y, Cb and Cr components, i.e. each entry in this palette contains Y, Cb and Cr values. In case of deblocking, block boundaries of palette-coded blocks are not deblocked.

[0073] Three different designs of the quantification scheme are available, which apply respectively to the normal transform, the transform skip, and the palette mode. Each different quantification design is associated with a different shift and scale operation. For blocks to which the normal transform is applied, the shift and scale operations depend on the block shape. For blocks to which the transform skip is applied, the shift and scale operations are independent of the block shape. For blocks coded in palette mode, only a scale operation is performed, which is independent of the block shape. Such non-uniform designs may not be optimal from a standardization point of view.

[0074] In one example, a method is provided for simplifying and further improving the transformation and quantification. Quantification and inverse quantification used in transform skip mode Calculations Applying the quantification and inverse quantification transformations used in the skip and palette modes to the palette mode Calculations It is possible to integrate the above.

[0075] In one example, current quantification and inverse quantification in palette mode Calculations to transform skip mode, the quantification and dequantification used in transform skip mode and palette mode Calculations It is possible to integrate the above.

[0076] In one example, quantification and inverse quantification for the normal transformation modes CalculationsQuantification and dequantification of normal transform modes, transform skip modes, and palette modes by applying to all modes including transform skip modes and palette modes. Calculations It is possible to integrate all of the above. Cooperation of quantification and inverse quantification for transform skip and palette modes.

[0077] JPEG0007594063000016.jpg32167

[0078] JPEG0007594063000017.jpg41167

[0079] JPEG0007594063000018.jpg43167

[0080] JPEG0007594063000019.jpg28167

[0081] JPEG0007594063000020.jpg32164

[0082] JPEG0007594063000021.jpg34167 Coordination of quantification and dequantification for normal transformation mode, transformation skip mode, and palette mode

[0083] In one example, quantification and inverse quantification for the normal transformation modes Calculations Quantification and dequantification of normal transform modes, transform skip modes, and palette modes by applying to all modes including transform skip modes and palette modes. Calculations It is possible to integrate all of the above.

[0084] In one example, the quantification and inverse quantification processes for the normal transform mode are also used for the transform skip mode and palette mode, e.g., equations (2), (3), (4), and (5) are also used for the quantification / inverse quantification processes in the transform skip mode and palette mode.

[0085] JPEG0007594063000022.jpg33165

[0086] JPEG0007594063000023.jpg55167

[0087] JPEG0007594063000024.jpg56168

[0088] JPEG0007594063000025.jpg31166

[0089] JPEG0007594063000026.jpg51166

[0090] JPEG0007594063000027.jpg18164

[0091] In one example, when using the above method to integrate quantification and inverse quantification processes of different sample values, it is possible to signal a separate flag, e.g., trans_quant_bypass_flag, in the bitstream to indicate that a particular image region, e.g., CU, was coded in a lossless coding mode. If such a flag indicates that a particular block was coded in a lossless mode, the corresponding quantification and inverse quantification processes are bypassed for coding of the block.

[0092] 10 is a block diagram illustrating an apparatus for video encoding and decoding according to an embodiment of the present disclosure. The apparatus 1100 may be a terminal such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

[0093] The device 1100 may include one or more of a processing unit 1002, a memory 1004, a power supply unit 1006, a multimedia unit 1008, an audio unit 1010, an input / output (I / O) interface 1012, a sensor unit 1014, and a communication unit 1016, as shown in FIG. 10.

[0094] The processing unit 1002 typically controls the overall operation of the device 1000, such as operations related to display, phone calls, data communication, camera operation, and recording operations. The processing unit 1002 may include one or more processors 1020 for executing instructions for implementing all or part of the steps of the above methods. Furthermore, the processing unit 1002 may include one or more modules that contribute to the interaction between the processing unit 1002 and other components. For example, the processing unit 1002 may include a multimedia module for contributing to the interaction between the multimedia unit 1008 and the processing unit 1002.

[0095] The memory 1004 is configured to store different types of data to support the operation of the device 1100. Examples of such data include instructions for any application or method operating on the device 1100, contact data, phone book data, messages, images, videos, etc. The memory 1004 is implemented by any type of volatile or non-volatile storage device or combination thereof, and the memory 1004 may be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or a compact disk.

[0096] The power supply section 1006 provides power to each component of the device 1000. The power supply section 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1000.

[0097] The multimedia unit 1008 includes a screen that provides an output interface between the device 1000 and a user. In an example, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, the screen may be realized as a touch screen that receives input signals from a user. The touch panel may include one or more touch sensors for sensing touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide operation, but also detect the duration and pressure associated with the touch or slide operation. In an example, the multimedia unit 1008 may include a front camera and / or a rear camera. When the device 1000 is in an operation mode such as an imaging mode or a video mode, the front camera and / or the rear camera can receive external multimedia data.

[0098] The audio unit 1010 is configured to output and / or input an audio signal. For example, the audio unit 1010 includes a microphone (MIC). The microphone is configured to receive an external audio signal when the device 1000 is in an operation mode such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may be further stored in the memory 1004 or may be transmitted via the communication unit 1016. In an example, the audio unit 1010 further includes a speaker for outputting the audio signal.

[0099] The I / O interface 1012 provides an interface between the processing unit 1002 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0100] The sensor unit 1014 includes one or more sensors for providing status assessments on different aspects of the device 1000. For example, the sensor unit 1014 can detect the on / off state of the device 1000 and the relative positions of components. For example, these components are the display and keypad of the device 1000. The sensor unit 1014 can also detect position changes of the device 1000 or components of the device 1000, the presence or absence of a user's touch on the device 1000, the orientation or acceleration / deceleration of the device 1000, and temperature changes of the device 1000. The sensor unit 1014 may include a proximity sensor configured to detect the presence of a nearby object without physical contact. The sensor unit 1014 may further include an optical sensor, such as a CMOS or CCD image sensor used in imaging applications. In an example, the sensor unit 1014 may further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0101] The communication unit 1016 is configured to facilitate wired or wireless communication between the device 1000 and other devices. The device 1000 can access a wireless network based on a communication standard such as WiFi, 4G, or a combination thereof. In one example, the communication unit 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one example, the communication unit 1016 may further include a near field communication (NFC) module for facilitating short-range communication. For example, the NFC module may be realized based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0102] In one example, the apparatus 1000 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements to perform the methods described above.

[0103] The non-transitory computer readable storage medium may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a hybrid drive or a solid state hybrid drive (SSHD), a read only memory (ROM), a compact disk read only memory (CD-ROM), a magnetic tape, a floppy disk, etc.

[0104] 11 is a flow chart illustrating an example process of quantification design in video encoding and decoding, which can be applied at an encoder, according to an embodiment of the present disclosure.

[0105] In step 1101, the processor 1020 determines a quantification parameter for the residual data of a CU.

[0106] In step 1102, the processor 1020 derives a scale value by scaling the quantification parameter with a scaling factor.

[0107] In step 1103, the processor 1020 determines a number of coefficients associated with the CU.

[0108] In one example, these multiple coefficients may be transform coefficients, CU's escape It may include color values, and prediction residual sample values.

[0109] In step 1104, the processor 1020 determines a number of parameters associated with the CU.

[0110] In one example, these multiple parameters include a coding bit depth and a bit shift that is determined based on the width and height of the CU, and a scaling rate that is determined based on the width and height of the CU.

[0111] In step 1105, the processor 1020 bit-shifts the parameters to obtain bit-shifts.

[0112] In step 1106, the processor 1020 obtains a quantification level based on the scale value, the plurality of coefficients thereof, and the plurality of bit shifts thereof.

[0113] In one example, the processor 1020, when determining the plurality of coefficients associated with the CU, further escape Determine the color value. escape The color value may be a value of a pixel in a CU having a color other than the predetermined colors selected from the CU. Then, when the processor 1020 obtains the quantification level based on the scale value, the coefficients, and the bit shifts, the scale value, escape A quantification level is obtained based on the color value and a number of bit shifts thereof.

[0114] In one example, the processor 1020 further determines a prediction residual sample associated with the CU. The processor 1020 determines a plurality of coefficients associated with the CU. and, The processor 1020 further determines a prediction residual sample value corresponding to the prediction residual sample. The processor 1020 obtains a quantification level based on the scale value, the plurality of coefficients, and the plurality of bit shifts. death , and further obtains a quantification level based on the scale value, the prediction residual sample value, and a number of bit shifts.

[0115] 12 is a flowchart illustrating an example process of inverse quantification design in video encoding and decoding, which can be applied at a decoder, according to an embodiment of the present disclosure.

[0116] In step 1201, the processor 1020 receives a video bitstream including quantification parameters and quantification levels.

[0117] In step 1202, the processor 1020 derives a scale value by scaling the quantification parameter with a scaling factor.

[0118] In step 1203, the processor 1020 determines a number of parameters associated with the CU.

[0119] In one example, these multiple parameters include a coding bit depth and a bit shift that is determined based on the width and height of the CU, and a scaling rate that is determined based on the width and height of the CU.

[0120] In step 1204, the processor 1020 obtains a number of bit shifts by bit-shifting the parameters.

[0121] In step 1205, the processor 1020 obtains a number of coefficients associated with the CU based on the quantification level, the scale value, and a number of bit shifts thereof.

[0122] In one example, these coefficients may be reconstructed transform coefficients, escape It may include color values, and reconstructed prediction residual sample values.

[0123] In one example, when obtaining a plurality of coefficients associated with a CU based on a quantification level, a scale value, and a plurality of bit shifts thereof, the processor 1020 further performs a reconfiguration of a CU based on a quantification level, a scale value, and a plurality of bit shifts thereof. escape Determine the color value of this reconstructed escape The color value may be the value of a pixel in a CU that has a color other than the predetermined plurality of colors selected from this CU.

[0124] In an example, the processor 1020 further determines a prediction residual sample associated with the CU. When the processor 1020 obtains a plurality of coefficients associated with the CU based on the quantification level, the scale value, and the plurality of bit shifts, the processor 1020 obtains a reconstructed prediction residual sample value of the CU based on the quantification level, the scale value, and the plurality of bit shifts. The reconstructed prediction residual sample value may correspond to the prediction residual sample.

[0125] In one example, a computing device for video encoding is provided that includes a processor 1020 and a memory 1004 configured to store instructions executable by the processor, where the processor, upon execution of the instructions, is configured to perform a method as shown in FIG.

[0126] In one example, a computing device for video encoding is provided that includes a processor 1020 and a memory 1004 configured to store instructions executable by the processor, where the processor, upon execution of the instructions, is configured to perform a method as shown in FIG.

[0127] In another example, a non-transitory computer-readable storage medium 1004 is provided having instructions stored thereon that, when executed by a processor 1020, cause the processor to perform a method such as that shown in FIG.

[0128] In another example, a non-transitory computer readable storage medium 1004 is provided having instructions stored thereon that, when executed by a processor 1020, cause the processor to perform a method such as that shown in FIG.

[0129] The description of the present disclosure has been presented for convenience of illustration and is not intended to be exhaustive or limiting of the present disclosure Numerous modifications, variations, and alternative implementations will be apparent to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings.

[0130] The embodiments have been chosen and described to explain the principles of the disclosure and to enable those skilled in the art to understand the disclosure for various implementations and to best utilize the various implementations to adapt the underlying principles and various modifications to specific applications contemplated. Therefore, it should be understood that the scope of the disclosure is not limited to the particular examples of the disclosed embodiments, and that modifications and other implementations are also within the scope of the disclosure.

Claims

1. determining a quantification parameter and a quantification level for a first coding block; deriving a scale value by looking up a scale level table based on the quantification parameter; in response to determining that the first coding block was coded in transform skip mode, obtaining residual samples for samples in the first coding block by applying a first inverse quantification operation to obtain reconstruction samples for escape samples in a second coding block coded in palette mode based on the quantification level, the scale value, and a plurality of bit shifts; Including, a parameter in the first inverse quantification operation is not determined by a block size of the first coding block or the second coding block; A method for video encoding.

2. and in response to determining that the third coded block was coded in a normal transform mode, obtaining residual samples for samples in the third coded block by applying a second inverse quantification operation different from the first inverse quantification operation; The method of claim 1 , wherein parameters in the second inverse quantification operation are determined by a block size of the first coding block.

3. 2. The method of claim 1, wherein the scale level table is levelScale[k] = { 40, 45, 51, 57, 64, 72}, where k = 0...

5.

4. generating an escape flag indicating whether an escape sample is present in the second coded block; The method of claim 1 further comprising:

5. The method of claim 1 , wherein the multiple bit shifts include a left shift of QP / 6 bits and a right shift of 6 bits, where QP is the quantification parameter.

6. the first inverse quantification operation being: [0010] 【number】 The method of claim 1.

7. one or more processors; a non-transitory memory coupled to the one or more processors; a plurality of programs stored in the non-transitory memory; Including, The plurality of programs, when executed by the one or more processors, cause the one or more processors to: A computing device configured to carry out the method of any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium storing a number of programs for execution by a computing device having one or more processors, comprising: The plurality of programs, when executed by the one or more processors, cause the one or more processors to: A non-transitory computer readable storage medium adapted to carry out the method according to any one of claims 1 to 6.

9. A computer program comprising instructions which, when executed by a processor, cause the processor to carry out the method according to any one of claims 1 to 6.

10. A method for transmitting a bitstream generated by the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • escape color encoding for palette encoding mode

    JP2018510539A

  • Methods and apparatus of video coding using palette mode

    WO2021030667A1

  • Image or video coding based on palette mode

    WO2021040400A1