Combined use of adaptive color conversion and differential encoding and decoding for video

Abstract

Methods, systems, and devices for implementing adaptive color transform (ACT) and block-based delta pulse codec modulation (BDPCM) during image / video encoding or decoding are described. An example method of video processing includes: for a conversion between a current video unit of a video as a chroma video unit and a bitstream representation of the video, determining based on a rule whether a block-based delta pulse codec modulation (BDPCM) mode is available; and performing the conversion based on the determination, wherein the rule is based on whether the adaptive color transform (ACT) mode is used to encode and decode the current video unit and / or whether the BDPCM mode is used to encode and decode a luma video unit corresponding to the current video unit.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is an application entering the Chinese national phase of international patent application No. PCT / CN2020 / 137941 filed on December 21, 2020, which claims priority to international patent application No. PCT / CN2019 / 126472 filed on December 19, 2019. The entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] The patent document relates to image and video encoding and decoding. Background Art

[0004] Digital video consumes the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention

[0005] This document discloses techniques that can be used by video encoders and decoders to perform Adaptive Color Transform (ACT) and Block-Based Delta Pulse Codec Modulation (BDPCM) during image / video encoding or decoding.

[0006] In an example aspect, a method of video processing is disclosed. The method includes performing a conversion between a video including a current video unit and a bitstream representation of the video according to a rule, wherein the rule specifies that an adaptive color transform (ACT) mode and a block-based delta pulse codec modulation (BDPCM) mode can be used to encode and decode the current video unit in a mutually exclusive manner.

[0007] In another example aspect, a method of video processing is disclosed. The method includes: for a conversion between a current video unit of the video as a chroma video unit and a bitstream representation of the video, determining based on a rule whether a block-based delta pulse codec modulation (BDPCM) mode is available; and performing the conversion based on the determination, wherein the rule is based on whether an adaptive color transform (ACT) mode is used to encode and decode the current video unit and / or whether a BDPCM mode is used to encode and decode a luma video unit corresponding to the current video unit.

[0008] In yet another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a video including a current video unit as a chroma video unit and a bitstream representation of the video, wherein the current video unit is encoded and decoded using an adaptive color transform (ACT) mode and a block-based delta pulse coding modulation (BDPCM) mode, wherein the bitstream representation conforms to a format rule, wherein the format rule specifies that an indication of a BDPCM prediction direction of a chroma component of the current video unit is determined by a BDPCM prediction direction of a corresponding luma video unit encoded and decoded in the BDPCM mode, and wherein signaling notification of the indication is not present in the bitstream representation.

[0009] In yet another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a video including a current video unit and a bitstream representation of the video according to a rule, wherein the rule specifies that during encoding, an adaptive color transform (ACT) mode is applied to the current video unit after a block-based delta pulse codec modulation (BDPCM) mode is applied to the current video unit, or during decoding, an ACT mode is applied to the current video unit before a BDPCM mode is applied to the current video unit.

[0010] In yet another example aspect, a video processing method is disclosed. The method includes: for a conversion between a video including a current video unit and a bitstream representation of the video, determining that an adaptive color transform (ACT) mode is enabled for the conversion, and based on the determination, performing the conversion by performing a clipping operation on a quantization parameter (QP) associated with the current video unit.

[0011] In yet another example aspect, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the above method.

[0012] In yet another example aspect, a video decoder apparatus is disclosed. The video decoder includes a processor configured to implement the above method.

[0013] In yet another example aspect, a non-transitory computer-readable medium having stored thereon code is disclosed. The code is in the form of processor-executable code embodying one of the methods described herein.

[0014] These and other features are described throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An example screen content codec (SCC) decoder flow for in-loop adaptive color transform (ACT) in a video decoder is shown.

[0016] Figure 2An example decoding process using ACT is shown.

[0017] Figure 3 is a block diagram illustrating an example video processing system in which the various techniques disclosed herein may be implemented.

[0018] Figure 4 is a block diagram of an example hardware platform for video processing.

[0019] Figure 5 is a block diagram illustrating an example video codec system in which some embodiments of the present disclosure may be implemented.

[0020] Figure 6 is a block diagram illustrating an example of an encoder in which some embodiments of the present disclosure may be implemented.

[0021] Figure 7 is a block diagram illustrating an example of a decoder in which some embodiments of the present disclosure may be implemented.

[0022] Figures 8 to 12 A flow chart illustrating an example method of video processing is shown. DETAILED DESCRIPTION

[0023] The section headings used in this document are intended to facilitate understanding, rather than to limit the applicability of the techniques and embodiments disclosed in each section to that section. In addition, the use of H.266 terminology in some descriptions is intended only to facilitate understanding, rather than to limit the scope of the disclosed techniques. Therefore, the techniques described herein are also applicable to other video codec protocols and designs.

[0024] 1. Introduction

[0025] This patent document relates to image / video codec technology. Specifically, it relates to adaptive color transformation in image / video codec. It can be applied to standards under development, such as universal video codec. It is also applicable to future video codec standards or video codecs.

[0026] 2. Brief description of video encoding and decoding

[0027] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that utilizes temporal prediction plus transform coding. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and input them into a reference software called the Joint Exploration Model (JEM). In April 2018, VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) established the Joint Video Experts Team (JVET) to work on the VVC standard with the goal of reducing the bit rate by 50% compared to HEVC.

[0028] The latest version of the VVC draft, Generic Video Codec (Draft 7), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 16_Geneva / wg11 / JVET-P2001-v14.zip.

[0029] The latest reference software for VVC is called VTM and can be found at: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-7.0.

[0030] 2.1. Adaptive Color Transform (ACT) in HEVC SCC

[0031] Adaptive Color Transform (ACT) was adopted into the HEVC Screen Content Codec (SCC) Test Model 2 at the 18th JCT-VC meeting (June 30-July 9, 2014, Sapporo, Japan). ACT performs an in-loop color space conversion in the prediction residual domain using color transform matrices based on the YCoCg and YCoCg-R color spaces. ACT can be adaptively turned on or off at the CU level using the flag cu_residual_act_flag. ACT can be combined with Cross Component Prediction (CCP), another inter-component decorrelation method already supported by HEVC. When both are enabled, ACT is performed after the CCP in the decoder, as in Figure 1 shown.

[0032] 2.1.1. Color space conversion in ACT

[0033] The color space conversion in ACT is based on the YCoCg-R transform. Lossy codecs and lossless codecs (cu_transquant_bypass_flag = 0 or 1) use the same inverse transform, but in the case of lossy codecs, the Co and Cg components are additionally shifted left by 1 bit. Specifically, the following color space transforms are used for forward and inverse conversion for lossy and lossless codecs:

[0034] Forward transform for lossy codecs (non-normative):

[0035]

[0036] Inverse transform for lossless codec (non-normative):

[0037] Co=RB

[0038] t=B+(Co>>1)

[0039] Cg=(Gt)

[0040] Y=t+(Cg>>1)

[0041] Inverse conversion (normative):

[0042] if(lossy){

[0043] Co=Co<<1

[0044] Cg=Cg<<1

[0045] }

[0046] t=Y-(Cg>>1)

[0047] G=Cg+t

[0048] B=t-(Co>>1)

[0049] R=Co+b

[0050] The forward color transform is unnormalized, with norms for Y and Cg approximately equal to For Co, it is approximately equal to To compensate for the non-normalized nature of the forward transform, a deltaQP of (-5, -3, -5) is applied to (Y, Co, Cg) respectively. In other words, for a given "normal" QP for a CU, if ACT is on, the quantization parameters of (Y, Co, Cg) are set to (QP-5, QP-3, QP-5), respectively. The adjusted quantization parameters only affect the quantization and inverse quantization of the residual in the CU. For deblocking, the "normal" QP value is still used. Clipping to 0 will be applied to the adjusted QP values ​​to ensure that they do not become negative. Note that this QP adjustment only applies to lossy codecs, as lossless codecs do not perform quantization (cu_transquant_bypass_flag=1). In SCM 4, PPS / slice level signaling of additional QP offset values ​​was introduced. When adaptive color transform is applied, these QP offset values ​​can be used for CU instead of (-5, -3, -5).

[0051] When the input bit depths of the color components are different, an appropriate left shift is applied during ACT to align the sample bit depth to the maximum bit depth, and an appropriate right shift is applied after ACT to restore the original sample bit depth.

[0052] ACT in VVC

[0053] Figure 2 FIG. 4 shows a decoding flow chart of VVC using ACT. Figure 2 As shown, the color space conversion is performed in the residual domain. Specifically, an additional decoding module, namely inverse ACT, is introduced after the inverse transform to convert the residual from the YCgCo domain back to the original domain.

[0054] In VVC, unless the maximum transform size is less than the width or height of a codec unit (CU), a CU leaf node is also used as the unit of transform processing. Therefore, in the proposed implementation, the ACT flag is signaled for a CU to select the color space used to encode and decode its residual. In addition, following the HEVC ACT design, for inter and IBC CUs, ACT is only enabled when there is at least one non-zero coefficient in the CU. For intra CUs, ACT is enabled only when the chroma component selects the same intra prediction mode of the luminance component (i.e., DM mode).

[0055] The core transform for color space conversion remains the same as that for HEVC. In addition, similar to the ACT design of HEVC, in order to compensate for the change in dynamic range of the residual signal before and after color conversion, a QP adjustment of (-5, -5, -3) is applied to the transform residual.

[0056] On the other hand, the forward and inverse color transforms require access to the residuals of all three components. Accordingly, in the proposed implementation, ACT is disabled in the following two scenarios, in which all residuals of the three components are not available.

[0057] 1. Split tree partitioning: When split tree is applied, the luma and chroma samples within a CTU are partitioned by different structures. This results in the CU in the luma tree containing only the luma component, and the CU in the chroma tree containing only two chroma components.

[0058] 2. Intra-frame sub-partition prediction (ISP): ISP sub-partitioning is only applied to luma, while chroma signals are encoded and decoded without partitioning. In the current ISP design, except for the last ISP sub-partitioning, other sub-partitioning only contains luma components.

[0059] The text of the codec unit in the VVC draft is as follows.

[0060]

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067] cu_act_enabled_flag is equal to 1 to specify that the residual of the current codec unit is in YC g C o cu_act_enabled_flag equal to 0 specifies that the residual of the current codec unit is encoded in the original color space. When cu_act_enabled_flag is not present, it is inferred to be equal to 0.

[0068] 2.3. Switching to skip mode in VVC

[0069] As in HEVC, the residual of a block can be encoded and decoded using a transform skip mode that completely skips the transform process of the block. In addition, for transform skip blocks, the minimum allowed quantization parameter (QP) signaled in the SPS is used, which is set equal to 6*(internalBitDepth–inputBitDepth)+4 in VTM7.0.

[0070] 2.4. Block-based delta pulse codec modulation (BDPCM)

[0071] In JVET-M0413, block-based delta pulse code modulation (BDPCM) was proposed to efficiently encode screen content and then adopted into VVC.

[0072] The prediction direction used in BDPCM can be vertical and horizontal prediction modes. Similar to intra prediction, intra prediction is performed on the entire block by copying samples in the prediction direction (horizontal or vertical prediction). The residual is quantized, and the delta between the quantized residual and the quantized value of its predictor (horizontal or vertical) is encoded and decoded. This can be described as follows: For a block of size M (rows) × N (columns), let r i,j , 0≤i≤M-1,0≤j≤N-1 is the prediction residual after performing intra prediction horizontally (copying left neighboring pixel values ​​row by row across the predicted block) or vertically (copying top neighboring lines to every line in the predicted block) using unfiltered samples from the upper or left block boundary samples. Let Q(r i,j ), 0≤i≤M-1,0≤j≤N-1 represents the residual r i,j The quantized version of , where the residual is the difference between the original block and the predicted block value. Block DPCM is then applied to the quantized residual samples, producing a quantized version with elements The modified M×N array When vertical BDPCM signaling is activated:

[0073]

[0074] For horizontal prediction, similar rules apply, and the residual quantization samples are obtained by

[0075]

[0076] Residual quantization samples is sent to the decoder.

[0077] At the decoder side, the above calculation is reversed to produce Q(r i,j ),0≤i≤M-1,0≤j≤N-1.

[0078] For the vertical prediction case,

[0079]

[0080] For the horizontal case,

[0081]

[0082] The inverse quantized residual Q -1 (Q(ri,j )) is added to the intra block prediction value to produce the reconstructed sample value.

[0083] The main advantage of this scheme is that inverse BDPCM can be done on the fly during coefficient parsing, simply by adding the predictor when parsing the coefficients, or it can be performed after parsing.

[0084] In VTM7.0, BDPCM can also be applied to the chrominance module. The chrominance BDPCM has an independent flag and BDPCM direction, which is different from the luma BDPCM mode.

[0085] 2.5. Scaling process of transform coefficients

[0086] The text related to the scaling process of transform coefficients in JVET-P2001-vE is as follows.

[0087] Inputs to this process include:

[0088] – specifies the luma position (xTbY, yTbY) of the top left sample of the current luma transform block relative to the top left luma sample of the current picture,

[0089] – variable nTbW that specifies the transform block width,

[0090] – variable nTbH that specifies the transform block height,

[0091] – The variable predMode that specifies the prediction mode of the codec unit,

[0092] – Variable cIdx that specifies the color component of the current block.

[0093] The output of this process is an (nTbW)x(nTbH) array d of scaled transform coefficients with elements d[x][y].

[0094] The quantization parameter qP is derived as follows:

[0095] – If cIdx is equal to 0, the following applies:

[0096] qP=Qp′ Y (1129)

[0097] – Otherwise, if TuCResMode[xTbY][yTbY] is equal to 2, the following applies:

[0098] qP=Qp′ CbCr (1130)

[0099] – Otherwise, if cIdx is equal to 1, the following applies:

[0100] qP=Qp′Cb (1131)

[0101] – Otherwise (cIdx equals 2), the following applies:

[0102] qP=Qp′ Cr (1132)

[0103] The quantization parameter qP is modified, and the variables rectNonTsFlag and bdShift are derived as follows:

[0104] – If transform_skip_flag[xTbY][yTbY][cIdx] is equal to 0, the following applies:

[0105] qP=qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0) (1133)

[0106] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH))&1)==1)? 1:0(1134)

[0107] bdShift=BitDepth+rectNonTsFlag+ (1135)

[0108] ((Log2(nTbW)+Log2(nTbH)) / 2)-5+pic_dep_quant_enabled_flag

[0109] – Otherwise (transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1), the following applies:

[0110] qP=Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]?5:0) (1136)

[0112] rectNonTsFlag=0 (1137)

[0113] bdShift=10 (1138)

[0114] The variable bdOffset is derived as follows:

[0115] bdOffset=(1<<bdShift)> >1 (1139)

[0116] The list levelScale[][] is specified as levelScale[j][k] = {{40,45,51,57,64,72},{57,64,72,80,90,102}}, where j = 0..1, k = 0..5.

[0117] The (nTbW)x(nTbH) array dz is set equal to the (nTbW)x(nTbH) array TransCoeffLevel[xTbY][yTbY][cIdx].

[0118] To derive the scaled transform coefficients d[x][y], where x = 0..nTbW-1, y = 0..nTbH-1, the following applies:

[0119] – The intermediate scaling factor m[x][y] is derived as follows:

[0120] – m[x][y] is set equal to 16 if one or more of the following conditions are true:

[0121] –sps_scaling_list_enabled_flag is equal to 0.

[0122] –pic_scaling_list_present_flag is equal to 0.

[0123] –transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1.

[0124] –scaling_matrix_for_lfnst_disabled_flag is equal to 1 and lfnst_idx[xTbY][yTbY] is not equal to 0.

[0125] – Otherwise, the following applies:

[0126] – The variable id is based on predMode, cIdx, nTbW and nTbH as described in Table 36 and the variable log2MatrixSize is derived as follows:

[0127] log2MatrixSize=(id<2)? 1:(id<8)? 2:3 (1140)

[0128] – The scaling factor m[x][y] is derived as follows:

[0129] m[x][y]=ScalingMatrixRec[id][i][j]

[0130] where i=(x<<log2MatrixSize)> >Log2(nTbW),

[0131] j=(y<<log2MatrixSize)> >Log2(nTbH) (1141)

[0132] – If id is greater than 13, and both x and y are equal to 0, then m[0][0] is further modified as follows:

[0133] m[0][0]=ScalingMatrixDCRec[id-14] (1142)

[0134] Note – The quantization matrix elements m[x][y] may be reset to zero if any of the following conditions are true:

[0135] –x is greater than 32

[0136] –y is greater than 32

[0137] – The decoded tu is not encoded by the default transform mode (i.e., transform type is not equal to 0), and x is greater than 16

[0138] – The decoded tu is not encoded by the default transform mode (i.e., transform type is not equal to 0), and y is greater than 16

[0139] – The scaling factor ls[x][y] is derived as follows:

[0140] – If pic_dep_quant_enabled_flag is equal to 1, and transform_skip_flag[xTbY][yTbY][cIdx] is equal to 0, the following applies:

[0141] ls[x][y]=(m[x][y]*levelScale[rectNonTsFlag][(qP+1)%6])<<((qP+1) / 6)(1143)

[0142] – Otherwise (pic_dep_quant_enabled_flag is equal to 0 or transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1), the following applies:

[0143] ls[x][y]=(m[x][y]*levelScale[rectNonTsFlag][qP%6])<<(qP / 6)(1144)

[0144] – When BdpcmFlag[xTbY][yYbY][cIdx] is equal to 1, dz[x][y] is modified as follows:

[0145] – If BdpcmDir[xTbY][yYbY][cIdx] is equal to 0 and x is greater than 0, the following applies:

[0146] dz[x][y]=Clip3(CoeffMin,CoeffMax,dz[x-1][y]+dz[x][y])(1145)

[0147] – Otherwise, if BdpcmDir[xTbY][yTbY][cIdx] is equal to 1 and y is greater than 0, the following applies:

[0148] dz[x][y]=Clip3(CoeffMin,CoeffMax,dz[x][y-1]+dz[x][y])(1146)

[0149] – The value dnc[x][y] is derived as follows:

[0150] dnc[x][y]=(dz[x][y]*ls[x][y]+bdOffset)>>bdShift (1147)

[0151] – The scaled transform coefficients d[x][y] are derived as follows:

[0152] d[x][y]=Clip3(CoeffMin,CoeffMax,dnc[x][y]) (1148)

[0153] Table 36 – Specifying the scaling matrix identifier variable id according to predMode, cIdx, nTbW and nTbH

[0154]

[0155] 3. Technical Problems Solved by the Embodiments and Solutions Described in This Article

[0156] 1. In the current design, ACT and luma BDPCM modes can be enabled for a block. However, chroma BDPCM mode is always disabled for blocks coded in ACT mode. Therefore, prediction signals may be derived differently for luma and chroma blocks in the same codec unit, which may be less efficient.

[0157] 2. When ACT is enabled, the quantization parameter (QP) of the block may become negative.

[0158] 3. The current design of ACT does not support lossless encoding and decoding.

[0159] 4. Examples of technical solutions

[0160] The technical solutions described below should be considered as examples to explain the general concept. These technical solutions should not be interpreted in a narrow way. In addition, these technical solutions can be combined in any way.

[0161] In the following description, the term "block" can refer to a video region, such as a coding unit (CU), a prediction unit (PU), a transform unit (TU), which can contain samples of three color components. The term "BDPCM" is not limited to the design of VVC, but it can represent a technique for encoding and decoding the residual using different prediction signal generation methods.

[0162] Example embodiment of interaction between ACT and BDPCM

[0163] 1. Whether chroma BDPCM mode is enabled may depend on the use of ACT and / or luma BDPCM mode.

[0164] a. In one example, when ACT is enabled on a block, the indication of use of chroma BDPCM mode (eg, intra_bdpcm_chroma_flag) may be inferred as the indication of use of luma BDPCM mode (eg, intra_bdpcm_luma_flag).

[0165] i. In one example, the inferred value of chroma BDPCM mode is defined as (ACT and luma BDPCM mode enabled? true:false).

[0166] 1. In one example, when intra_bdpcm_luma_flag is false, intra_bdpcm_chroma_flag may be set equal to false.

[0167] a. Alternatively, when intra_bdpcm_luma_flag is true, intra_bdpcm_chroma_flag may be set equal to true.

[0168] ii. Alternatively, in one example, if the indication of use of luma BDPCM mode and ACT for the block is true, then the indication of use of chroma BDPCM mode may be inferred to be true.

[0169] b. Alternatively, it may be conditionally checked whether to signal the use of ACT for a block, such as using the same BDPCM prediction direction for luma and chroma samples in the block.

[0170] i. Alternatively, and in addition, after using the BDPCM mode, signaling an indication of ACT usage.

[0171] 2. When ACT is enabled on a block, the indication of the prediction direction for the chroma BDPCM mode (eg, intra_bdpcm_chroma_dir_flag) may be inferred as the indication of the used prediction direction for the luma BDPCM mode (eg, intra_bdpcm_luma_dir_flag).

[0172] a. In one example, the inferred value of intra_bdpcm_chroma_dir_flag is defined as (ACT enabled? intra_bdpcm_luma_dir_flag: 0).

[0173] i. In one example, if the indication of the prediction direction for the luma BDPCM mode is horizontal, then the indication of the prediction direction for the chroma BDPCM mode may be inferred to be horizontal.

[0174] ii. Alternatively, in one example, if the indication of the prediction direction for the luma BDPCM mode is vertical, then the indication of the prediction direction for the chroma BDPCM mode may be inferred to be vertical.

[0175] 3.ACT and BDPCM modes can be applied exclusively.

[0176] a. In one example, when ACT mode is enabled on a block, BDPCM mode can be disabled on the block.

[0177] i. Alternatively, or in addition, the indication of the use of the BDPCM mode may be signaled after the signaling of the indication of the use of the ACT mode.

[0178] ii. Alternatively, and in addition, the indication of the use of BDPCM mode may not be signaled and inferred to be false (0).

[0179] b. In one example, when BDPCM mode is enabled on a block, ACT mode may be disabled on the block.

[0180] i. Alternatively, and in addition, the indication of the use of the ACT mode may be signaled after the signaling of the indication of the use of the BDPCM mode.

[0181] ii. Alternatively, and in addition, the indication of the use of ACT mode may not be signaled and inferred to be false (0).

[0182] c. In one example, the BDPCM mode in the above example may represent a luma BDPCM mode and / or a chroma BDPCM mode.

[0183] 4. At the decoder, inverse ACT may be applied before inverse BDPCM.

[0184] a. In one example, ACT can be applied even when luma and chroma BDPCM have different prediction modes.

[0185] b. Alternatively, at the encoder, forward ACT can be applied after BDPCM.

[0186] Example embodiment of QP settings when ACT is enabled

[0187] 5. Propose to trim QP when ACT is enabled.

[0188] a. In one example, the trimming function can be defined as (l, h, x), where l is the lowest possible value of the input x and h is the highest possible value of the input x

[0189] i. In one example, l can be set equal to 0.

[0190] ii. In one example, h may be set equal to 63.

[0191] b. In one example, QP may be the qP given in Section 2.5.

[0192] c. In one example, clipping may be performed after QP adjustment in ACT mode.

[0193] d. In one example, when transform skip is applied, l may be set equal to the minimum allowed QP for transform skip mode.

[0194] General Technology (Items 6-7)

[0195] 6. In the above examples, l, h, m, n and / or k are integers and may depend on

[0196] i. Messages signaled in DPS / SPS / VPS / PPS / APS / Picture header / Slice header / Slice group header / Largest codec unit (LCU) / Codec unit (CU) / LCU row / LCU group / TU / PU block / Video codec unit

[0197] ii. Location of CU / PU / TU / block / video codec unit

[0198] iii. Modes for encoding and decoding blocks containing samples along edges

[0199] iv. The transformation matrix applied to the block containing samples along the edge

[0200] v. Block dimensions / block shapes of the current block and / or its neighboring blocks

[0201] vi. Indication of the color format (such as 4:2:0, 4:4:4, RGB, or YUV)

[0202] vii. Codec tree structure (such as dual tree or single tree)

[0203] viii. Slice / slice group type and / or picture type

[0204] ix. Color components (e.g., may apply only to Cb or Cr)

[0205] x. Time domain layer ID

[0206] xi. Standard Profile / Level / Tier

[0207] xii. Alternatively, a, b, m, n and / or k may be signaled to the decoder.

[0208] 7. The method proposed above can be applied under certain conditions.

[0209] a. In one example, the condition is that the color format is 4:2:0 and / or 4:2:2.

[0210] b. In one example, the indication of the use of the above method may be signaled in the sequence / picture / slice / tile / video area level (such as SPS / PPS / picture header / slice header).

[0211] c. In one example, the use of the above method may depend on

[0212] i. Video content (e.g. screen content or natural content)

[0213] ii. Messages signaled in DPS / SPS / VPS / PPS / APS / Picture header / Slice header / Slice group header / Largest codec unit (LCU) / Codec unit (CU) / LCU row / LCU group / TU / PU block / Video codec unit

[0214] iii. Location of CU / PU / TU / block / video codec unit

[0215] iv. Mode for encoding and decoding blocks containing samples along edges

[0216] v. The transformation matrix applied to the block containing the samples along the edge

[0217] vi. Block dimensions of the current block and / or its neighboring blocks

[0218] vii. Block shape of the current block and / or its neighboring blocks

[0219] viii. Indication of the color format (e.g., 4:2:0, 4:4:4, RGB, or YUV)

[0220] ix. Codec tree structure (such as dual tree or single tree)

[0221] x. Strip / slice group type and / or picture type

[0222] xi. Color component (e.g., may apply only to Cb or Cr)

[0223] xii. Temporal layer ID

[0224] xiii. Standard grade / level / level

[0225] xiv. Alternatively, m and / or n may be signaled to the decoder.

[0226] 5. Examples

[0227] These examples are based on JVET-P2001-vE. Newly added text is highlighted in bold italics. Deleted text is marked in italics.

[0228] 5.1. Example #1

[0229] This embodiment involves the interaction between ACT and BDPCM modes.

[0230]

[0231]

[0232] 5.2. Example #2

[0233] This embodiment involves the interaction between ACT and BDPCM modes.

[0234] intra_bdpcm_chroma_flag equal to 1 specifies that BDPCM is applied to the current chroma codec block at position (x0, y0), i.e., transform is skipped and the intra chroma prediction mode is specified by intra_bdpcm_chroma_dir_flag. intra_bdpcm_chroma_flag equal to 0 specifies that BDPCM is not applied to the current chroma codec block at position (x0, y0).

[0235] It is inferred to be equal to 0 when intra_bdpcm_chroma_flag is not present and cu_act_enabled_flag (and cu_act_enabled_flag) is false.

[0236] When intra_bdpcm_chroma_flag is not present and cu_act_enabled_flag is true, it is inferred to be equal to intra_bdpcm_luma_flag.

[0237] For x = x0..x0+cbWidth–1, y = y0..y0+cbHeight-1 and cIdx = 1..2, the variable BdpcmFlag[x][y][cIdx] is set equal to intra_bdpcm_chroma_flag.

[0238] intra_bdpcm_chroma_dir_flag equal to 0 specifies that the BDPCM prediction direction is horizontal. intra_bdpcm_chroma_dir_flag equal to 1 specifies that the BDPCM prediction direction is vertical. When cu_act_enabled_flag is true, intra_bdpcm_chroma_dir_flag is set equal to intra_bdpcm_luma_dir_flag.

[0239] For x = x0..x0+cbWidth-1, y = y0..y0+cbHeight-1 and cIdx = 1..2, the variable BdpcmDir[x][y][cIdx] is set equal to intra_bdpcm_chroma_dir_flag.

[0240] 5.3. Example #3

[0241] This embodiment relates to QP setting.

[0242] 8.7.3 Scaling of Transform Coefficients

[0243] Inputs to this process include:

[0244] – specifies the luma position (xTbY, yTbY) of the top left sample of the current luma transform block relative to the top left luma sample of the current picture,

[0245] – variable nTbW that specifies the transform block width,

[0246] – variable nTbH that specifies the transform block height,

[0247] – The variable predMode that specifies the prediction mode of the codec unit,

[0248] – Variable cIdx that specifies the color component of the current block.

[0249] The output of this process is an (nTbW)x(nTbH) array d of scaled transform coefficients with elements d[x][y].

[0250] …

[0251] The quantization parameter qP is modified, and the variables rectNonTsFlag and bdShift are derived as follows:

[0252] – If transform_skip_flag[xTbY][yTbY][cIdx] is equal to 0, the following applies:

[0253] qP=qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0) (1133)

[0254] qP=Clip3(0,63,qP)

[0255] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH))&1)==1)? 1:0(1134)

[0256] bdShift=BitDepth+rectNonTsFlag+ (1135)

[0257] ((Log2(nTbW)+Log2(nTbH)) / 2)-5+pic_dep_quant_enabled_flag

[0258] – Otherwise (transform_skip_flag[xTbY][yTbY][cIdx] is equal to 1), the following applies:

[0259] qP=Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]?5:0) (1136)

[0261] qP=Max(QpPrimeTsMin,qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0))

[0262] rectNonTsFlag=0 (1137)

[0263] bdShift=10 (1138)

[0264] …

[0265] Figure 33000 is a block diagram illustrating an example video processing system 3000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 3000. System 3000 may include an input 3002 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. Input 3002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces, such as Ethernet, a passive optical network (PON), etc., and wireless interfaces, such as Wi-Fi or a cellular interface.

[0266] System 3000 may include a codec component 3004, which may implement various encodings or encoding methods described in this document. Codec component 3004 may reduce the average bit rate of the video from input 3002 to the output of codec component 3004 to generate a coded representation of the video. Therefore, codec technology is sometimes referred to as video compression or video code conversion technology. As represented by component 3006, the output of codec component 3004 may be stored or transmitted via a connected communication. Component 3008 may use a bitstream (or coded) representation of the storage or transmission of the video received at input 3002 to generate pixel values ​​or displayable video sent to display interface 3010. The process of generating user-visible video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at encoders, and corresponding decoding tools or operations that are reverse to the encoding result will be performed by decoders.

[0267] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Display Port, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be implemented in various electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0268] Figure 4 4000. The apparatus 4000 may be used to implement one or more methods described herein. The apparatus 4000 may be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 4000 may include one or more processors 4002, one or more memories 4004, and video processing hardware 4006. The processor(s) 4002 may be configured to implement the methods described herein (e.g., in Figures 8 to 124004. Memory(s) 4004 may be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 4006 may be used to implement some of the techniques described in this document in hardware circuits.

[0269] Figure 5 1 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure. Figure 5 As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, which may be referred to as a video encoding device. The destination device 120 may decode the encoded video data generated by the source device 110, and the destination device 120 may be referred to as a video decoding device. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0270] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bit stream. The bit stream may include a bit sequence that forms a coded representation of the video data. The bit stream may include coded pictures and associated data. The coded pictures are coded representations of the pictures. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the I / O interface 116 through the network 130a. The coded video data may also be stored on a storage medium / server 130b for access by the destination device 120.

[0271] Destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0272] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120, or may be external to destination device 120, with destination device 120 configured to interface with an external display device.

[0273] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVM) standard, and other current and / or future standards.

[0274] Figure 6 is a block diagram showing an example of a video encoder 200, which may be Figure 5 Video encoder 114 in system 100 is shown.

[0275] Video encoder 200 is configured to perform any or all of the techniques of the present invention. Figure 6 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared between various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0276] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214.

[0277] In other examples, the video encoder 200 may include more, fewer, or different functional components. In an example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture in which the current video block is located.

[0278] Furthermore, some components such as the motion estimation unit 204 and the motion compensation unit 205 may be highly integrated, but for the purpose of explanation, they are described in detail in the following sections. Figure 6 are represented separately in the example.

[0279] The partition unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0280] The mode selection unit 203 may select one of the coding modes (e.g., intra or inter) based on the error result, and provide the resulting intra or inter coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra and inter prediction (CIIP) modes, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may also select a resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision).

[0281] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block of the current video block based on the motion information of pictures other than the picture associated with the current video block from the buffer 213 and the decoded samples.

[0282] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block, eg, depending on whether the current video block is in an I slice, a P slice, or a B slice.

[0283] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search for a reference video block of the current video block in the reference pictures of list 0 or list 1. The motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1 that includes the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

[0284] In other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block, and the motion estimation unit 204 may search for a reference video block of the current video block in the reference pictures in list 0, and may also search for another reference video block of the current video block in the reference pictures in list 1. The motion estimation unit 204 may then generate a reference index indicating the reference pictures in list 0 and list 1 that include the reference video block and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0285] In some examples, motion estimation unit 204 may output complete motion information for use in the decoding process of a decoder.

[0286] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, motion estimation unit 204 may reference motion information of another video block to signal motion information for the current video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0287] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0288] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0289] As described above, video encoder 200 may predictively signal motion vectors.Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0290] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include the predicted video block and various syntax elements.

[0291] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the predicted video block(s) of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of samples in the current video block.

[0292] In other examples, the current video block may not have residual data for the current video block, such as in skip mode, and the residual generation unit 207 may not perform the subtraction operation.

[0293] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0294] After transform processing unit 208 generates a transform coefficient video block associated with a current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0295] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current block for storage in buffer 213.

[0296] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video block artifacts in the video block.

[0297] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. When the entropy encoding unit 214 receives the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0298] Figure 7 is a block diagram showing an example of a video decoder 300, which may be Figure 5 Video decoder 114 in system 100 is shown.

[0299] Video decoder 300 may be configured to perform any or all of the techniques of the present invention. Figure 7 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0300] exist Figure 7 In the example of FIG. 3 , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 may perform operations generally similar to those for the video encoder 200 ( Figure 6 ) is a decoding process that is the inverse of the encoding process described.

[0301] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy-encoded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information, including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine this information, for example, by performing AMVP and merge mode.

[0302] The motion compensation unit 302 may generate motion compensated blocks, possibly performing interpolation based on an interpolation filter. An identifier of an interpolation filter to be used with sub-pixel precision may be included in the syntax element.

[0303] Motion compensation unit 302 may calculate interpolated values ​​of sub-integer pixels of a reference block using interpolation filters as used by video encoder 20 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 based on received syntax information and use the interpolation filters to generate a prediction block.

[0304] The motion compensation unit 302 may use some syntax information to determine the sizes of blocks used to encode the frame(s) and / or slice(s) of the encoded video sequence, partition information describing how each macroblock of the pictures of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.

[0305] The intra prediction unit 303 may form a prediction block from spatially neighboring blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 303 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0306] The reconstruction unit 306 may add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also produces a decoded video for presentation on a display device.

[0307] Figures 8 to 12 It is shown that in e.g. Figures 3 to 7 An exemplary method for implementing the above technical solution is shown in the embodiment.

[0308] Figure 8A flow chart of an example method 800 of video processing is shown. The method 800 includes, at operation 810, performing conversion between a video including a current video unit and a bitstream representation of the video according to a rule that specifies that an adaptive color transform (ACT) mode and a block-based delta pulse codec modulation (BDPCM) mode may be used to encode and decode the current video unit in a mutually exclusive manner.

[0309] Fig. 9 A flow chart of an example method 900 of video processing is shown. The method 900 includes, at operation 910, for a conversion between a current video unit of the video as a chroma video unit and a bitstream representation of the video, determining whether an adaptive color transform (ACT) mode is available based on a rule that specifies whether a block-based delta pulse codec modulation (BDPCM) mode is used to encode and decode a luma video unit corresponding to the current video unit.

[0310] The method 900 includes, at operation 920 , performing the converting based on the determination.

[0311] Fig.10 A flow chart of an example method 1000 of video processing is shown. The method 1000 includes, at operation 1010, performing a conversion between a video including a current video unit as a chroma video unit and a bitstream representation of the video, the current video unit being encoded and decoded using an adaptive color transform (ACT) mode and a block-based delta pulse coding modulation (BDPCM) mode, and the bitstream representation conforming to a format rule, the format rule specifying that an indication of a BDPCM prediction direction for a chroma component of the current video unit is determined by a BDPCM prediction direction for a corresponding luma video unit encoded and decoded in the BDPCM mode, and that signaling notification specifying the indication is not present in the bitstream representation.

[0312] Fig.11 A flow chart of an example method 1100 of video processing is shown. The method 1100 includes, at operation 1110, performing a conversion between a video including a current video unit and a bitstream representation of the video according to a rule that specifies that during encoding, an adaptive color transform (ACT) mode is applied to the current video unit after a block-based delta pulse codec modulation (BDPCM) mode is applied to the current video unit, or during decoding, an ACT mode is applied to the current video unit before a BDPCM mode is applied to the current video unit.

[0313] Fig.12A flow chart of an example method 1200 of video processing is shown. The method 1200 includes, at operation 1210, for a conversion between a video including a current video unit and a bitstream representation of the video, determining that an adaptive color transform (ACT) mode is enabled for the conversion.

[0314] The method 1200 includes, at operation 1220, performing the conversion by performing a clipping operation on a quantization parameter (QP) associated with the current video unit based on the determination.

[0315] A first list of some embodiment preferred solutions is provided next.

[0316] A1. A method of video processing, comprising performing a conversion between a video including a current video unit and a bitstream representation of the video according to a rule, wherein the rule specifies that an adaptive color transform (ACT) mode and a block-based delta pulse codec modulation (BDPCM) mode can be used to encode and decode the current video unit in a mutually exclusive manner.

[0317] A2. The method of solution A1, wherein the ACT mode is enabled for the current video unit, and wherein the BDPCM mode is disabled for the current video unit.

[0318] A3. The method of solution A2, wherein, for the luma component of the video, the indication of use of the BDPCM mode is signaled subsequent to the indication of use of the ACT mode in the bitstream representation.

[0319] A4. The method of solution A2, wherein, for chroma components of the video, the indication of use of BDPCM mode is excluded from the bitstream representation and wherein the indication is inferred to be false.

[0320] A5. The method of solution A1, wherein BDPCM mode is enabled for the current video unit, and wherein ACT mode is disabled for the current video unit.

[0321] A6. The method of solution A5, wherein the indication of use of ACT mode is signaled after the indication of use of BDPCM mode in the bitstream representation.

[0322] A7. The method of solution A5, wherein the indication of ACT mode usage is excluded from the bitstream representation, and wherein the indication is inferred to be false.

[0323] A8. The method of any of solutions A5 to A7, wherein the BDPCM mode corresponds to a BDPCM mode for a luma component of the video.

[0324] A9. The method of any of solutions A5 to A7, wherein the BDPCM mode corresponds to the BDPCM mode for the chrominance components of the video.

[0325] A10. A method for video processing, comprising: for a conversion between a current video unit of a video as a chrominance video unit and a bitstream representation of the video, determining based on a rule whether a block-based delta pulse codec modulation (BDPCM) mode is available, and performing the conversion based on the determination, wherein the rule is based on whether an adaptive color transform (ACT) mode is used to encode and decode the current video unit and / or whether a BDPCM mode is used to encode and decode a luminance video unit corresponding to the current video unit.

[0326] A11. The method of solution A10, wherein ACT mode is enabled, and wherein usage of the BDPCM mode for chroma components is inferred to be the same as usage of the BDPCM mode for luma components.

[0327] A12. The method of solution A10, wherein the indication of use of the ACT mode included in the bitstream representation is based on the BDPCM prediction directions for the luma component and the chroma components.

[0328] A13. A method of video processing, comprising performing conversion between a video including a current video unit as a chroma video unit and a bitstream representation of the video, wherein the current video unit is encoded and decoded using an adaptive color transform (ACT) mode and a block-based delta pulse codec modulation (BDPCM) mode, wherein the bitstream representation conforms to a format rule, wherein the format rule specifies that an indication of a BDPCM prediction direction of a chroma component of the current video unit is determined by the BDPCM prediction direction of a corresponding luminance video unit encoded and decoded in the BDPCM mode, and wherein signaling notification of the indication is not present in the bitstream representation.

[0329] A14. The method of solution A13, wherein upon determining that ACT mode is enabled for the current video unit, the indication of use of BDPCM prediction direction for chroma components of the current video unit is inferred to be zero.

[0330] A15. A method of video processing, comprising performing a conversion between a video including a current video unit and a bitstream representation of the video according to a rule, wherein the rule specifies that during encoding, an adaptive color transform (ACT) mode is applied to the current video unit after a block-based delta pulse codec modulation (BDPCM) mode is applied to the current video unit, or during decoding, an ACT mode is applied to the current video unit before a BDPCM mode is applied to the current video unit.

[0331] A16. The method of solution A15, wherein the first prediction mode of the BDPCM mode for the chrominance components of the video codec unit is different from the second prediction mode of the BDPCM mode for the luma component of the video.

[0332] A17. The method of any one of Solutions A1 to A16, wherein the video unit comprises a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0333] A18. The method of any of solutions A1 to A17, wherein the conversion comprises decoding the video from a bitstream representation.

[0334] A19. The method of any of solutions A1 to A17, wherein the conversion comprises encoding the video into a bitstream representation.

[0335] A20. A video processing device, comprising a processor configured to implement any one or more of the methods described in solutions A1 to A19.

[0336] A21. A computer-readable medium having instructions stored thereon, which, when executed, cause a processor to implement the method described in any one or more of Solutions A1 to A19.

[0337] A22. A computer-readable medium storing a bitstream representation generated according to any of solutions A1 to A19.

[0338] A23. A video processing device for storing a bitstream representation, wherein the video processing device is configured to implement the method described in any one or more of solutions A1 to A19.

[0339] Next a second list of solutions preferred by some embodiments is provided.

[0340] B1. A method of video processing, comprising: for a conversion between a video including a current video unit and a bitstream representation of the video, determining that an adaptive color transform (ACT) mode is enabled for the conversion; and based on the determination, performing the conversion by performing a cropping operation on a quantization parameter (QP) associated with the current video unit.

[0341] B2. The method of solution B1, in which the trimming operation uses a trimming function defined as follows

[0342]

[0343] Where x is the input variable, l is the minimum output value of the trimming function, and h is the maximum output value of the trimming function, and where l, h, and x are integers.

[0344] B3. The method of solution B2, where l=0.

[0345] B4. The method of solution B2, where h=63.

[0346] B5. The method of solution B2, wherein / is equal to the minimum allowed QP of the transform skip mode based on the current video unit encoded using the transform skip mode.

[0347] B6. The method of solution B1, wherein the clipping operation is performed after the QP adjustment operation for ACT mode.

[0348] B7. The method of any of solutions B2 to B5, wherein l and h are based on at least one of the following: (a) a message signaled in a decoder parameter set (DPS), a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), an adaptation parameter set (APS), a picture header, a slice header, a slice group header, a largest codec unit (LCU), a codec unit (CU), an LCU row, a group of LCUs, a transform unit (TU), a prediction unit (PU) block or a video codec unit, (b) the position of a CU, PU, ​​TU, block or a video codec unit, (c) a mode of encoding or decoding of one or more blocks including at least one sample along an edge, (d) one or more transform matrices applied to one or more blocks, (e) a dimension or shape of a current video unit and / or neighboring video units, (f) an indication of a color format, (g) a codec tree structure, (h) a slice type, a slice group type and / or a picture type, (i) color components of the video, (j) a temporal layer identifier (ID), and (k) a profile, level or hierarchy of a standard.

[0349] B8. The method of any one of solutions B1 to B7, wherein performing the cropping operation is also based on whether the color format of the video is 4:2:0 or 4:4:4.

[0350] B9. The method of any one of solutions B1 to B7, wherein performing the cropping operation is further based on an indication of the use of the cropping operation signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, or a slice header.

[0351] B10. A method as in any one of solutions B1 to B7, wherein the clipping operation is also performed based on at least one of: (a) video content, (b) a signaled signal in a decoder parameter set (DPS), a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), an adaptation parameter set (APS), a picture header, a slice header, a slice group header, a maximum codec unit (LCU), a codec unit (CU), an LCU row, a group of LCUs, a transform unit (TU), a prediction unit (PU) block, or a video codec unit. message, (c) the location of a CU, PU, ​​TU, block or video codec unit, (d) the mode of coding of one or more blocks including at least one sample along an edge, (e) one or more transform matrices applied to one or more blocks, (f) the dimensions or shape of the current video unit and / or neighboring video units, (g) an indication of the color format, (h) the codec tree structure, (i) the slice type, slice group type and / or picture type, (j) the color components of the video, (k) the temporal layer identifier (ID), and (l) the profile, level or hierarchy of the standard.

[0352] B11. The method of any one of solutions B1 to B10, wherein the video unit comprises a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0353] B12. The method of any of solutions B1 to B11, wherein converting comprises decoding the video from a bitstream representation.

[0354] B13. The method of any of solutions B1 to B11, wherein the conversion comprises encoding the video into a bitstream representation.

[0355] B14. A video processing device comprising a processor configured to implement the method described in any one or more of solutions B1 to B13.

[0356] B15. A computer-readable medium having instructions stored thereon, which when executed cause a processor to implement the method described in any one or more of solutions B1 to B13.

[0357] B16. A computer-readable medium storing a bitstream representation generated according to any of solutions B1 to B13.

[0358] B17. A video processing device for storing a bitstream representation, wherein the video processing device is configured to implement the method in any one or more of solutions B1 to B13.

[0359] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, during conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa, a video compression algorithm may be applied. As defined by the syntax, the bitstream representation of the current video block may correspond to bits that are co-located or interspersed at different locations within the bitstream. For example, a macroblock may be encoded based on transformed and coded error residual values, and also using bits in the header and other fields in the bitstream.

[0360] The disclosed and other solutions, examples, embodiments, modules and functional operations described herein may be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material combination that implements a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" encompasses all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0361] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one computer or on multiple computers located in one location or distributed across multiple locations and interconnected by a communications network.

[0362] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0363] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, the computer will also include or be operably coupled to one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from the large-capacity storage device or to transfer data to it, or both. However, the computer does not need to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented or incorporated by dedicated logic circuits.

[0364] Although this patent document contains many details, these details should not be interpreted as limitations on the scope of any subject matter or the content claimed, but rather as descriptions of features unique to specific embodiments of specific technologies. Certain features described in the context of independent embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments alone or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of a sub-combination.

[0365] Similarly, although operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0366] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: performing a conversion between a video including a current video unit and a bitstream of said video according to a rule, wherein the rule specifies that when an adaptive color transform mode and a transform skip mode are enabled for the current video unit and the current video unit is a luma component unit, an input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than a minimum allowed quantization parameter of the transform skip mode, a quantization parameter used in a quantization process or an inverse quantization process is set equal to the minimum allowed quantization parameter, wherein the rule specifies that when the adaptive color transform mode is enabled for the current video unit and the transform skip mode is not applied to the current video unit and the current video unit is the luma component unit, the input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than 0, the quantization parameter used in the quantization process or the inverse quantization process is set to be equal to 0, Wherein, in the adaptive color conversion mode, for encoding operation, the visual signal is converted from the first color domain to the second color domain, or for decoding operation, the visual signal is converted from the second color domain to the first color domain.

2. The method according to claim 1, wherein: The rule further provides that when the current video unit is a chroma component unit, in response to enabling the adaptive color transform mode for the current video unit, an indication of use of a differential coding mode for the current video unit is excluded from the bitstream, wherein the indication of the differential coding mode for the current video unit is inferred to be false, indicating that the differential coding mode is disabled for the current video unit, and wherein, in the differential coding mode, a residual of a sample of the current video unit is represented in the bitstream using a difference between a quantized residual and a predicted value of the quantized residual.

3. The method according to claim 2, wherein: The difference is represented using a block-based differential pulse codec modulation representation.

4. The method according to claim 1, wherein: The converting includes decoding the video from the bitstream.

5. The method according to claim 1, wherein: The converting includes encoding the video into the bitstream.

6. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to: performing a conversion between a video including a current video unit and a bitstream of said video according to a rule, wherein the rule specifies that when an adaptive color transform mode and a transform skip mode are enabled for the current video unit and the current video unit is a luma component unit, an input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than a minimum allowed quantization parameter of the transform skip mode, a quantization parameter used in a quantization process or an inverse quantization process is set equal to the minimum allowed quantization parameter, wherein the rule specifies that when the adaptive color transform mode is enabled for the current video unit and the transform skip mode is not applied to the current video unit and the current video unit is the luma component unit, the input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than 0, the quantization parameter used in the quantization process or the inverse quantization process is set to be equal to 0, Wherein, in the adaptive color conversion mode, for encoding operation, the visual signal is converted from the first color domain to the second color domain, or for decoding operation, the visual signal is converted from the second color domain to the first color domain.

7. The device according to claim 6, wherein: The rule further provides that when the current video unit is a chroma component unit, in response to enabling the adaptive color transform mode for the current video unit, an indication of use of a differential coding mode for the current video unit is excluded from the bitstream, wherein the indication of the differential coding mode for the current video unit is inferred to be false, indicating that the differential coding mode is disabled for the current video unit, and wherein, in the differential coding mode, a residual of a sample of the current video unit is represented in the bitstream using a difference between a quantized residual and a predicted value of the quantized residual.

8. The device according to claim 7, wherein: The difference is represented using a block-based differential pulse codec modulation representation.

9. A non-transitory computer-readable storage medium storing instructions that cause a processor to: performing a conversion between a video including a current video unit and a bitstream of said video according to a rule, in, The rule specifies that when an adaptive color transform mode and a transform skip mode are enabled for the current video unit and the current video unit is a luma component unit, an input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than a minimum allowed quantization parameter of the transform skip mode, a quantization parameter used in a quantization process or an inverse quantization process is set equal to the minimum allowed quantization parameter, wherein the rule specifies that when the adaptive color transform mode is enabled for the current video unit and the transform skip mode is not applied to the current video unit and the current video unit is the luma component unit, the input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than 0, the quantization parameter used in the quantization process or the inverse quantization process is set to be equal to 0, Wherein, in the adaptive color conversion mode, for encoding operation, the visual signal is converted from the first color domain to the second color domain, or for decoding operation, the visual signal is converted from the second color domain to the first color domain.

10. The non-transitory computer-readable storage medium of claim 9, wherein: The rule further provides that when the current video unit is a chroma component unit, in response to enabling the adaptive color transform mode for the current video unit, an indication of use of a differential coding mode for the current video unit is excluded from the bitstream, wherein the indication of the differential coding mode for the current video unit is inferred to be false, indicating that the differential coding mode is disabled for the current video unit, and wherein, in the differential coding mode, a residual of a sample of the current video unit is represented in the bitstream using a difference between a quantized residual and a predicted value of the quantized residual.

11. A method for storing a bit stream of a video, comprising: Generating the bitstream for the video including the current video unit according to the rule; as well as storing the bitstream in a non-transitory computer-readable storage medium, wherein the rule specifies that when an adaptive color transform mode and a transform skip mode are enabled for the current video unit and the current video unit is a luma component unit, an input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than a minimum allowed quantization parameter of the transform skip mode, a quantization parameter used in a quantization process or an inverse quantization process is set equal to the minimum allowed quantization parameter, wherein the rule specifies that when the adaptive color transform mode is enabled for the current video unit and the transform skip mode is not applied to the current video unit and the current video unit is the luma component unit, the input quantization parameter is added to an adaptive color transform offset value equal to -5, and when the input quantization parameter added to -5 is less than 0, the quantization parameter used in the quantization process or the inverse quantization process is set to be equal to 0, Wherein, in the adaptive color conversion mode, for encoding operation, the visual signal is converted from the first color domain to the second color domain, or for decoding operation, the visual signal is converted from the second color domain to the first color domain.

12. The method according to claim 11, wherein: The rule further provides that when the current video unit is a chroma component unit, in response to enabling the adaptive color transform mode for the current video unit, an indication of use of a differential coding mode for the current video unit is excluded from the bitstream, wherein the indication of the differential coding mode for the current video unit is inferred to be false, indicating that the differential coding mode is disabled for the current video unit, and wherein, in the differential coding mode, a residual of a sample of the current video unit is represented in the bitstream using a difference between a quantized residual and a predicted value of the quantized residual.

13. A video processing device, comprising a processor configured to implement the method according to any one of claims 4-5, 11-12.

14. A computer readable medium having instructions stored thereon, which when executed cause a processor to implement the method of any one of claims 3-5, 11-12.

Images