Quantization properties of adaptive in-loop color-space conversion for video coding

Abstract

A method of video processing includes, for a conversion between a video component of a video comprising a current video block and a bitstream representation of the video, determining, based on a rule for the current video block, that an adaptive color space transform (ACT) tool is used; and performing the conversion in accordance with the determination, and wherein the rule specifies that a same constant quantization parameter (QP) offset is used for each of one or more color components of the current video block.

Description

[0001] Cross-reference to related applications

[0002] In accordance with the applicable Patent Law and / or the Paris Convention, this application promptly claims priority and benefit to International Patent Application No. PCT / CN2019 / 116280, filed November 7, 2019. For all legal purposes, the entire disclosure of the aforementioned application is incorporated herein by reference as a part of this application disclosure. Technical Field

[0003] This patent document typically addresses video encoding and decoding technologies. Background Technology

[0004] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T proposed H.261 and H.263, while ISO / IEC proposed MPEG-1 and MPEG-4 Vision. These two organizations jointly proposed the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / High-Efficiency Video Codec (HEVC) standards. Starting with H.262, video codec standards have been based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and applied them to reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work toward the goal of a VVC standard that reduces the bit rate by 50% compared to HEVC. Summary of the Invention

[0005] Using publicly available video encoding, transcoding, or decoding techniques, embodiments of video encoders or decoders can handle virtual boundaries of codec tree blocks to provide better compression efficiency and simpler implementation of encoding or decoding tools.

[0006] In one example aspect, a video processing method is disclosed. The method includes: for a conversion between video components of a video including a current video block and a bitstream representation of the video, determining an adaptive color space transformation (ACT) tool based on rules for the current video block; and performing the conversion according to the determination, wherein the rules specify that each of one or more color components of the current video block uses the same constant quantization parameter (QP) offset.

[0007] In another example, a different video processing method is disclosed. This method includes: for a conversion between video blocks and a bitstream representation of the video, determining whether and / or how to apply a deblocking process according to rules; and performing the conversion based on the determination, wherein the rules specify that the deblocking process depends on: i) the enabled state of Adaptive Color Space Transformation (ACT), in which the representation of the visual signal is transformed between two color spaces, or ii) the color space used to represent the samples of the pixels.

[0008] In another example, a different video processing method is disclosed. This method includes: determining, for a conversion between a current video block and the video's codec representation, whether to perform a color transformation due to the use of an Adaptive Color Space Transformation (ACT) codec tool; and performing the transformation based on the determination.

[0009] In another example, a different video processing method is disclosed. This method includes performing a conversion between a video containing one or more video blocks and a bitstream representation of the video, according to rules specifying that, for video blocks encoded using a transform skip mode, the minimum allowed quantization parameter (QP) is independent of the applicability of the adaptive color space transformation (ACT) tool to the video blocks.

[0010] In another example, a different video processing method is disclosed. This method includes performing a conversion between a video containing one or more video blocks and a bitstream representation of the video, according to rules specifying that for video blocks encoded using a transform skip mode, the minimum allowed quantization parameter (QP) depends on the color channels.

[0011] In another example, a different video processing method is disclosed. This method includes: determining exclusion rules applicable to the conversion between video regions of a video and the bitstream representation of the video, wherein the exclusion rules specify that, for video regions, the conversion is not permitted to use the Joint Codec and Chroma Residual Codec (JCCR) tool; and performing the conversion based on the determination.

[0012] In another example, a different video processing method is disclosed. This method includes: a conversion between video units and a bitstream representation of the video; determining, based on quantization parameters used for the conversion of the video units, the suitability of an encoding / decoding tool and / or whether to notify the encoding / decoding tool of information and / or instructions to be used via signaling; and performing the conversion based on the determination.

[0013] In yet another example, a video encoding apparatus configured to perform the above-described methods is disclosed.

[0014] In yet another example, a video decoder configured to perform the methods described above is disclosed.

[0015] In yet another example, a machine-readable medium is disclosed. This medium stores code that, when executed, causes a processor to implement one or more of the methods described above.

[0016] The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, description and claims. Attached Figure Description

[0017] Figure 1 An example of an encoding process utilizing Adaptive Color Space Transformation (ACT) is shown.

[0018] Figure 2 An example of a decoding process with ACT is shown.

[0019] Figure 3 An example of neighboring samples used to derive IC parameters is shown.

[0020] Figure 4 An example flowchart of a decoding process with shaping is shown.

[0021] Figure 5 This is an example of a reference line.

[0022] Figure 6 Examples of splitting 4×8 and 8×4 blocks are shown.

[0023] Figure 7 Examples of splitting all blocks except 4×8, 8×4, and 4×4 are shown.

[0024] Figure 8 This is a diagram of ALWIP for a 4x4 block.

[0025] Figure 9 This is a diagram of ALWIP for an 8x8 block.

[0026] Figure 10 This is a diagram of an 8×4 block ALWIP.

[0027] Figure 11 This is a diagram of ALWIP for a 16x16 block.

[0028] Figure 12 This is a diagram of the sub-block transform modes SBT-V and SBT-H (the gray area represents the TU that can have non-zero coefficients; the white area represents the zero-output TU with all zero coefficients).

[0029] Figure 13 This is a diagram of the subblock transform mode SBT-Q.

[0030] Figure 14A and Figure 14BThis is a block diagram of an exemplary apparatus for video processing.

[0031] Figure 15 This is a block diagram illustrating an example video encoding / decoding system.

[0032] Figure 16 This is a block diagram illustrating an encoder according to some embodiments of the disclosed technology.

[0033] Figure 17 This is a block diagram illustrating a decoder according to some embodiments of the disclosed technology.

[0034] Figures 18A to 18E This is a flowchart of an example method for video processing based on some implementations of the disclosed technology. Detailed Implementation

[0035] The use of section headers in this document is for ease of understanding and does not limit the embodiments disclosed in a section to only that section. Furthermore, although certain embodiments are described with reference to multi-functional video codecs or other specific video codecs, the disclosed techniques are applicable to other video codec techniques. Additionally, while some embodiments describe video encoding and decoding steps in detail, it should be understood that the corresponding decoding steps for unencoding will be performed by the decoder. Furthermore, the term video processing includes video encoding / decoding or compression, video decoding or decompression, and video transcoding, wherein video pixels are represented from one compression format to another or at different compression bitrates.

[0036] 1. Overview

[0037] This document relates to video codec technology. Specifically, it concerns the interaction between adaptive color space transformation and other tools in video codec. It can be applied to existing video codec standards (such as HEVC) or pending standards (Multi-Functional Video Codec). It may also be applicable to future video codec standards or video codecs.

[0038] 2. Introduction to Video Encoding and Decoding

[0039] Video codec standards have primarily evolved from well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4 Vision. These two organizations jointly developed the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec architecture that utilizes time prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work toward the goal of a VVC standard that reduces the bit rate by 50% compared to HEVC.

[0040] 2.1 Adaptive Color Space Transformation (ACT) in HEVC Screen Content Encoding and Decoding Extensions

[0041] In HEVC SCC extensions, several tools have been proposed and adopted to improve SCC efficiency within the HEVC framework. For example, to utilize repeating patterns in SC, an intra-block copying (IBC) scheme is employed. Similar to motion compensation schemes used for inter-frame images, the IBC mode searches for repeating patterns in the reconstructed regions of the current image. Another direction for improving SCC is to reduce the redundancy of color components between the RGB / YCbCr sequences in the 4:4:4 chroma format.

[0042] Cross-component prediction (CCP) is a technique that uses signaling to send the weighted parameter index of each chroma color component of the transform unit. CCP provides a good improvement in encoding and decoding efficiency with a limited increase in complexity, so it was incorporated into the HEVC range extension and is part of HEVC version 2, which specifies the range extension and other extensions.

[0043] To further utilize the inter-frame color component correlation of SCC, an in-loop adaptive color space transform (ACT) is employed for HEVC SCC extension. The basic idea of ​​ACT is to adaptively convert the prediction residuals to a color space with reduced redundancy among the three color components. Before and after this, the signal follows the existing encoding / decoding path in HEVC range extension. To keep complexity as low as possible, only one additional color space is considered (i.e., the RGB to YCgCo-R inverse transform), which can be easily implemented through shift and addition operations.

[0044] 2.1.1 Transformations used in ACT

[0045] YCgCo transform is used for lossy encoding and decoding, while its reversible variant, YCgCo-R, is used for lossless encoding and decoding.

[0046] The forward and inverse transformation processes of YCgCo are as follows: Taking pixels in (R, G, B) color format as an example:

[0047] Forward:

[0048] Reverse:

[0049] Unlike the YCgCo transformation, which can be implemented through matrix multiplication, the invertible color space transformation used in ACT (i.e., YCgCo-R) can only be performed in lifting-based operations, as shown below:

[0050] Forward:

[0051] Reverse:

[0052] 2.1.2 Use of ACT

[0053] For each TU, a signaling flag can be sent to indicate the use of color space transformation. Additionally, for intra-frame encoded CUs, ACT is enabled only when the chroma and luma intra-frame prediction modes are the same, i.e., when the chroma block is encoded / decoded in DM mode.

[0054] Figure 1 A block diagram of the proposed method at the encoder is shown, where the residual signal derived from intra / inter-frame prediction is used as input. The proposed functional blocks (including forward and inverse color space transformations) are located in the encoding / decoding loop and are highlighted. Figure 1As shown, after intra-frame or inter-frame prediction processing (including prediction processing for IBC mode), it is determined whether to perform a forward color space transformation. This introduced color space transformation converts the input signal's color space to YCgCo, where the correlation between the three color components is relatively low. Subsequently, the original coding flow, if available, includes techniques such as CCP, integer transformation (i.e.,... Figure 1 In T), quantization (i.e., Figure 1 The Q) and entropy encoding / decoding processes are further called sequentially. Meanwhile, in... Figure 2 During the reconstruction or decoding process shown, in conventional inverse quantization (i.e., Figure 2 In the IQ), inverse transform (i.e., Figure 2 After the IT and inverse CCP in the code, the inverse color transform is invoked, if applicable, to convert the encoded / decoded residuals back to the original color space. It should be noted that the color space transformation is applied to the residual signal, not the reconstructed signal. Using this method, the decoder only needs to perform the inverse color space transformation, which minimizes the increase in complexity. Furthermore, in ACT, a fixed color space transformation, YCgCo and YCgCo-R, is used regardless of the input color space.

[0055] 2.1.3. ACT in VVC

[0056] A version of ACT was incorporated into VVC draft 7 and VTM-7.0. ACT is based on CU instead of TU. The core transformations used for color space conversion remain the same as those used for HEVC. Specifically, the following forward and inverse YCgCo color transformation matrices, as described below, are applied.

[0057]

[0058] In addition, to compensate for the dynamic range changes of the residual signal before and after color transformation, a QP adjustment of (-5, -5, -3) is applied to the transformation residual.

[0059] On the other hand, as shown in the transformations above, both forward and inverse color transformations require access to the residuals of all three components. Accordingly, in the proposed implementation, ACT is disabled in two cases where all residuals of the three components are unavailable.

[0060] Split tree segmentation: When a split tree is applied, the luminance and chrominance samples within a CTU are segmented by different structures. This results in the CU in the luminance tree containing only the luminance component, while the CU in the chrominance tree contains only the two chrominance components.

[0061] Intra-sub-partition prediction (ISP): ISP sub-segmentation applies only to luma, while the chroma signal is encoded and decoded without being split. In current ISP designs, except for the final ISP sub-segment, other sub-segments only contain the luma component.

[0062] 2.2. Local Illumination Compensation in JEM

[0063] Local illumination compensation (LIC) is based on a linear model of illumination variation, using a scaling factor a and an offset b. Furthermore, for each codec unit (CU) of inter-frame mode encoding and decoding, local illumination compensation (LIC) is adaptively enabled or disabled.

[0064] When LIC is applied to CU, the least squares error method is used to derive parameters a and b by using the nearest samples of the current CU and their corresponding reference samples. More specifically, as... Figure 3 As shown, the neighboring sample points of the CU's sub-sampling (2:1 sub-sampling) and the corresponding sample points in the reference image (identified by the motion information of the current CU or sub-CU) are used.

[0065] 2.2.1 Derivation of the prediction block

[0066] The IC parameters are derived and applied to each prediction direction. For each prediction direction, a first prediction block is generated using the decoded motion information, and then a temporary prediction block is obtained by applying the LIC model. Finally, the two temporary prediction blocks are used to derive the final prediction block.

[0067] When the CU encodes or decodes in Merge mode, it copies the LIC flag from the neighboring block in a manner similar to motion information in Merge mode; otherwise, it sends a signaling message to the CU to notify it of the LIC flag to indicate whether to apply the LIC.

[0068] When LIC is enabled for an image, additional CU-level RD checks are required to determine whether LIC is applied to the CU. When LIC is enabled for the CU, mean-removed sum of absolute difference (MR-SAD) and mean-removed sum of absolute Hadamard-transformed difference (MR-SATD) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively.

[0069] To reduce coding complexity, the following coding scheme was applied in JEM.

[0070] • When there is no significant change in illumination between the current image and its reference images, LIC will be disabled for the entire image. To identify this situation, histograms of the current image and each reference image of the current image are calculated at the encoder. If the histogram difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

[0071] 2.3 Inter-frame prediction methods in VVC

[0072] Several new codec tools exist for improving inter-frame prediction, such as Adaptive Motion Vector Differential Resolution (AMVR) for Signaling Notification MVD, Affine Prediction Mode, Triangular Prediction Mode (TPM), ATMVP, Generalized Bidirectional Prediction (GBI), and Bidirectional Optical Flow (BIO).

[0073] 2.3.1 Code Block Structure in VVC

[0074] In VVC, a quadtree / binary tree / ternary tree (QT / BT / TT) structure is used to split the image into square or rectangular blocks.

[0075] In addition to QT / BT / TT, VVC also employs a split tree (also known as a dual codec tree) for I-frames. Using a split tree, the codec block structure is signaled separately for the luminance and chrominance components.

[0076] 2.4 In-loop reshaping (ILR) in JVET-M0427

[0077] The basic idea of ​​In-Loop Shaping (ILR) is to transform the original (first domain) signal (predicted / reconstructed signal) to the second domain (shaping domain).

[0078] The in-ring luminance shaper is implemented as a pair of lookup tables (LUTs), but only one of the two LUTs needs to be signaled, as the other LUT can be computed from the signaled LUT. Each LUT is a one-dimensional, 10-bit, 1024-entry mapping table (1D-LUT). One LUT is a forward LUT (FwdLUT), which takes the input luminance code value Y... i Mapped to the changed value Y r :Y r =FwdLUT[Y i Another LUT is the inverse LUT (InvLUT), which modifies the code value Y. r Mapped to ( Y represents i (Reconstructed value).

[0079] 2.4.1 PWL Model

[0080] Conceptually, piecewise linear (PWL) is implemented as follows:

[0081] Let x1 and x2 be two input axis points, and y1 and y2 be the corresponding output axis points for a segment. The output value y for any input value x between x1 and x2 can be interpolated using the following equation:

[0082] y=((y2-y1) / (x2-x1))*(x-x1)+y1

[0083] In a fixed-point implementation, this equation can be rewritten as:

[0084] y=((m*x+2 FP_PREC-1 )>>FP_PREC)+c

[0085] Where m is a scalar, c is the offset, and FP_PREC is a constant value used to specify precision.

[0086] Note that in the CE-12 software, the PWL model is used to pre-compute the FwdLUT and InvLUT mapping tables with 1024 entries; however, the PWL model also allows for the real-time computation of equivalent mapping values ​​without pre-compiling the LUTs.

[0087] 2.4.2 Brightness Shaping

[0088] Test 2 of the in-ring luminance shaping (i.e., the proposed CE12-2) provides a lower-complexity pipeline that also eliminates the decoding latency of block-by-block intra-prediction in inter-strip reconstruction. For both inter- and intra-strips, intra-prediction is performed in the shaping domain.

[0089] Regardless of the stripe type, intra-frame prediction is always performed in the integer domain. This arrangement allows intra-frame prediction to begin immediately after the previous TU reconstruction is complete. It also provides uniform, rather than stripe-dependent, processing for intra-frame modes. Figure 4 A block diagram of pattern-based CE12-2 decoding processing is shown.

[0090] CE12-2 also tests a 16-segment piecewise linear (PWL) model with scaled luminance and chrominance residuals, instead of the 32-segment piecewise PWL model of CE12-1.

[0091] Figure 4 A flowchart of the decoding process with shaping is shown.

[0092] Inter-frame strip reconstruction is performed using the in-loop luminance shaper in CE12-2 (the light-shaded blocks indicate the signals in the shaping domain: luminance residual; intra-frame luminance prediction; and intra-frame luminance reconstruction).

[0093] 2.4.3 Luminance-dependent chromaticity residual scaling

[0094] Luminance-dependent chroma residual scaling is a multiplication process implemented using fixed-point integer arithmetic. Chroma residual scaling compensates for the interaction between the luminance and chroma signals. It is used in TU-level applications.

[0095] More specifically, the following applies:

[0096] – For intra-frame, average reconstructed brightness.

[0097] – For inter-frame, average predicted brightness.

[0098] The average is used to identify the index in the PWL model. This index identifies the scaling factor cScaleInv. The chromaticity residual is multiplied by this number.

[0099] It should be noted that the chroma scaling factor is calculated from the predicted luminance value of the forward mapping, not the reconstructed luminance value.

[0100] 2.4.3.1 Signaling for ILR-side information

[0101] Parameters (currently) are sent in the fragment header (similar to ALF). These are said to require 40-100 bits.

[0102] 2.4.3.2. Use of ILR

[0103] On the encoder side, each picture (or group of pictures) is first converted to the integer domain. All encoding processing is then performed in the integer domain. For intra-frame prediction, adjacent blocks are in the integer domain; for inter-frame prediction, the reference block (generated from the original domain of the decoded picture buffer) is first converted to the integer domain. Then, the residual is generated and encoded into the bitstream.

[0104] After the entire image (or group of images) has been encoded / decoded, the samples in the integer domain are converted to the original domain, and then a deblocking filter and other filters are applied.

[0105] Forward shaping of predictive signaling is disabled in the following cases:

[0106] (1) The current block is intra-frame coded.

[0107] (2) The current block is encoded as CPR (Current Picture Reference, also known as Intra-Block Copy, IBC).

[0108] (3) The current block is encoded in combined inter-frame-intra-frame mode (CIIP), and forward shaping is disabled for intra-frame prediction blocks.

[0109] 2.5 Virtual Pipelining Data Units (VPDUs)

[0110] Virtual Pipeline Data Units (VPDUs) are defined as non-overlapping MxM-luminance (L) / NxN-chrominance (C) units in an image. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. Different stages process different VPDUs simultaneously. In most pipeline stages, the size of the VPDU is roughly proportional to the size of the buffer; therefore, keeping the VPDU size small is crucial. In the HEVC hardware decoder, the VPDU size is set to the maximum transform block (TB) size. Increasing the maximum TB size from 32x32-L / 16x16-C (as in HEVC) to 64x64-L / 32x32-C (as in current VVC) can provide encoding / decoding gains, which is expected to increase the VPDU size (64x64-L / 32x32-C) by a factor of four compared to HEVC. However, in addition to quadtree (QT) codec unit (CU) splitting, VVC also employs ternary tree (TT) and binary tree (BT) splitting to achieve additional codec gain. Furthermore, TT and BT splitting can be recursively applied to 128x128-L / 64x64-C codec unit (CTU), which is said to increase the VPDU size (128x128-L / 64x64-C) by 16 times compared to HEVC.

[0111] In the current design of VVC, the dimensions of VPDU are defined as 64x64-L / 32x32-C.

[0112] 2.6 Multiple Reference Lines (MRL)

[0113] Multi-reference line (MRL) intra-prediction uses more reference lines for intra-prediction. Figure 5 In the example depicting four reference lines, the samples for segments A and F are not taken from the reconstructed neighboring samples, but are instead filled with the nearest samples from segments B and E, respectively. Predictions within the HEVC image use the nearest reference line (i.e., reference line 0). In MRL, two additional lines are used (reference line 1 and reference line 3).

[0114] The signaling notifies the index (mrl_idx) of the selected reference line and uses it to generate the intra-predictor. For reference line indices greater than 0, only additional reference line modes from the MPM list are included, and only the MPM index is signaled, with no other modes. The reference line index is signaled before the intra-predictor mode, and in the case of a non-zero reference line index, the planar mode and DC mode are excluded from the intra-predictor modes.

[0115] Figure 5Here are examples of four reference lines adjacent to the prediction block.

[0116] For the first line of a block within a CTU, MRL is disabled to prevent the use of extended reference samples outside the current CTU line. Similarly, PDPC is disabled when additional lines are used.

[0117] 2.7. Intra-frame sub-block segmentation (ISP)

[0118] In JVET-M0102, an ISP was proposed, as shown in Table 1, which splits the block of the in-frame prediction of luminance into 2 or 4 sub-segments vertically or horizontally according to the block size dimension. Figure 6 and Figure 7 Examples of two possibilities are shown. All sub-segments satisfy the condition of having at least 16 samples. For block size, if 4N or N4 (where N>8) are allowed, then 1×N or N×1 sub-segments may exist.

[0119] Table 1: Number of sub-segments depending on block size (maximum transform size is represented by maxTBSize)

[0120]

[0121]

[0122] For each of these sub-segments, a residual signal is generated by entropy decoding of the coefficients sent by the encoder, followed by inverse quantization and inverse transform. Intra-frame prediction is then performed on that sub-segment, and finally, the corresponding reconstructed samples are obtained by adding the residual signal to the predicted signal. Therefore, the reconstructed values ​​of each sub-segment can be used to generate the prediction for the next sub-segment, and this process is repeated, and so on. All sub-segments share the same intra-frame mode.

[0123] Subsequently, intra-frame subsegments are used to represent subsegments other than the first subsegment. If the ISP block is split along the horizontal (vertical) direction, the first subsegment represents the upper (left) subsegment.

[0124] Table 2: Specifications of trTypeHor and trTypeVer depending on predModeIntra

[0125]

[0126] 2.8 Affine Linear Weighted Intra-Prediction (ALWIP, also known as Matrix-Based Intra-Prediction)

[0127] In JVET-N0217, Affine Ray Weighted Intra Prediction (ALWIP, also known as Matrix-Based Intra Prediction (MIP)) was proposed.

[0128] 2.8.1 Generating Reduced Prediction Signals via Matrix-Vector Multiplication

[0129] First, the nearby reference points are downsampled by averaging to generate a reduced reference signal bdry. red Then, the reduced prediction signal pred is calculated by multiplying the matrix and vector and adding the offset. red :

[0130] pred red =A·bdry red +b

[0131] Here, matrix A has W red ·H red Matrix A has 4 rows and 4 columns (if W = H = 4), and in all other cases has 8 columns. b is the size W. red ·H red The vector.

[0132] 2.8.2 Description of the entire ALWIP process

[0133] Figure 8 , Figure 9 , Figure 10 and Figure 11 The diagram illustrates the entire process of averaging, matrix-vector multiplication, and linear interpolation for different shapes. Note that the remaining shapes are treated according to one of the depicted cases.

[0134] 1. Given a 4×4 block, ALWIP takes two averages along each axis of the boundary. The resulting four input samples are then subjected to matrix-vector multiplication. The matrix is ​​obtained from set S0. After adding the offset, this produces 16 final prediction samples. Linear interpolation is not necessary for generating prediction signaling notifications. Therefore, a total of (4.16) / (4.4) = 4 multiplications are performed per sample.

[0135] 2. Given an 8×8 block, ALWIP takes four averages along each axis of the boundary. The resulting eight input samples are fed into a matrix-vector product. The matrix is ​​obtained from set S1. This produces 16 samples at odd positions in the prediction block. Therefore, a total of (8·16) / (8·8) = 2 multiplications are performed for each sample. After adding the offset, these samples are vertically interpolated using a reduced top boundary. Horizontal interpolation is then performed using the original left boundary.

[0136] 3. Given an 8×4 block, ALWIP takes four averages along the horizontal axis of the boundary and four original boundary values ​​at the left boundary. The resulting eight input samples are then subjected to matrix-vector multiplication. The matrix is ​​obtained from set S1. This produces 16 samples at odd levels and each vertical position of the prediction block. Therefore, a total of (8·16) / (8·4) = 4 multiplications are performed for each sample. After adding the offset, these samples are horizontally interpolated by using the original left boundary.

[0137] The transpose case should be handled accordingly.

[0138] 4. Given a 16×16 block, ALWIP takes four averages along each axis of the boundary. The resulting eight input samples are then subjected to matrix-vector multiplication. The matrix is ​​obtained from set S2. This produces 64 samples at odd positions in the prediction block. Therefore, a total of (8·64) / (16·16) = 2 multiplications are performed per sample. After adding the offset, these samples are vertically interpolated by taking the eight averages of the top boundary. Horizontal interpolation is then performed using the original left boundary. In this case, no multiplications are added during the interpolation process. Therefore, a total of two multiplications are required per sample to compute the ALWIP prediction.

[0139] For larger shapes, the process is essentially the same, and it is easy to verify that the number of multiplications per sample point is less than four.

[0140] For a W×8 block where W>8, only horizontal interpolation is necessary because sample points are given at odd horizontal positions and at each vertical position.

[0141] Finally, for W×4 blocks where W>8, A_kbe is made into the resulting matrix by omitting each row corresponding to the odd-numbered entries along the horizontal axis of the downsampled block. Therefore, the output size is 32, and only the horizontal interpolation to be performed is retained.

[0142] The transpose case should be handled accordingly.

[0143] 2.8.3 Derivation of the Adaptive MPM List for Conventional Luminance and Chromatography Intra-Prediction Modes

[0144] The following section describes the MPM-based coding coordination between the proposed ALWIP mode and the conventional intra-prediction mode. The derivation of the luma and chroma MPM lists for the conventional intra-prediction mode uses a fixed table map_alwip_to_angular. idx idx∈{0,1,2}, the ALWIP mode predmode on the given PU. ALWIP Mapped to one of the traditional intra-frame prediction modes

[0145] predmode Angular=map_alwip_to_angular idx(PU) [predmode ALWIP ]

[0146] For the derivation of the brightness MPM list, whenever encountering the use of ALWIP mode predmode ALWIP The adjacent brightness blocks are like those in the regular intra-frame prediction mode (predmode). Angular The same applies to this block. For the chroma MPM list derivation, whenever the current luma block uses LWIP mode, the same mapping converts ALWIP mode to the regular intra-prediction mode.

[0147] 2.9 Quantization Residual Block Differential Pulse Code Modulation (QR-BDPCM)

[0148] In JVET-M0413, a quantized residual block differential pulse code modulation (QR-BDPCM) method is proposed to effectively encode screen content.

[0149] The prediction direction used in QR-BDPCM can be either vertical or horizontal prediction mode. Intra-prediction of the entire block is performed by copying samples in a prediction direction similar to intra-prediction (horizontal or vertical prediction). The residual is quantized, and the delta between the quantized residual and its predictor (horizontal or vertical) quantization value is encoded. This can be described as follows: for a block of size M (rows) × N (columns), let r i,j ,0≤i≤M-1,0≤j≤N-1 is the prediction residual after intra-frame prediction using unfiltered samples from the top or left block boundary, either horizontally (copying the left neighbor pixel values ​​of the entire prediction block row by row) or vertically (copying the top neighbor row to each row in the prediction block). Let Q(r i,j ), 0≤i≤M-1, 0≤j≤N-1 represent residuals r i,j The quantized version is then used, where the residual is the difference between the original block and the predicted block value. The block DPCM is then applied to the quantized residual samples to obtain a result with element-wise... Modified M×N array When signaling is used to notify the vertical BDPCM:

[0150]

[0151] For horizontal forecasting, similar rules apply, and residual quantization samples are obtained through the following operations.

[0152]

[0153] quantize residual samples Send to the decoder.

[0154] On the decoder side, the above calculation is inverted to produce Q(r). i,j ), 0≤i≤M-1, 0≤j≤N-1. For the case of vertical prediction,

[0155]

[0156] Regarding the horizontal situation

[0157]

[0158] Inverse quantization residual Q -1 (Q(r i,j Add to the block-predicted values ​​to generate reconstructed sample values.

[0159] The main advantage of this approach is that inverse DPCM can be performed dynamically during coefficient resolution, simply by adding a predictor when resolving the coefficients, or it can be performed after resolution.

[0160] 2.10. Intra-frame chroma encoding and decoding

[0161] In addition to intra-frame chroma prediction mode, VVC also introduces CCLM and joint chroma residual coding and decoding.

[0162] 2.10.1 Cross-Component Linear Model (CCLM)

[0163] To reduce cross-component redundancy, VTM4 uses a cross-component linear model (CCLM) prediction mode, where chrominance samples are predicted based on reconstructed luminance samples from the same CU using the following linear model:

[0164] pred C (i, j) = α·rec L '(i,j)+β

[0165] Among them, pred C (i,j) represents the predicted chromaticity sample points in the CU, and rec L (i,j) represents the reconstructed luminance sample points downsampled from the same CU. The linear model parameters α and β are derived from the relationship between the luminance and chrominance values ​​of four samples at a specific location. Among these four samples, the two larger values ​​are averaged, and then the two smaller values ​​are averaged. The averaged values ​​are then used to derive the linear model parameters.

[0166] 2.10.2 Joint Chroma Residual Encoding and Decoding

[0167] If the chroma shaper is active, shaping is applied to the received residual in the same way as in separate codec modes (i.e., the joint residual signal is shaped). At the encoder end, when testing this mode, the average of the positive Cb residual and the negative Cr residual is used as the joint residual:

[0168] resJoint=(resCb–resCr) / 2

[0169] A binary bit indicator is signaled in the bitstream to enable this mode. When this mode is enabled, the joint residual signal is encoded and decoded in the bitstream. On the decoder side, the joint residual is used for the Cb component, and the negative version of the residual is used for the Cr component.

[0170] 2.11 Sub-block Transform (SBT) in VVC

[0171] For inter-frame predicted CUs with cu_cbf equal to 1, a cu_sbt_flag can be sent via signaling to indicate whether the entire residual block or a sub-part of the residual block is decoded. In the former case, the inter-frame MTS information is further parsed to determine the transform type of the CU. In the latter case, a portion of the residual block is encoded / decoded via an inferred adaptive transform, while the rest of the residual block is zeroed out. Since almost no encoding / decoding gain is achieved, SBT is not suitable for combined inter-frame and intra-frame modes.

[0172] 2.11.1 Sub-block TU tiling

[0173] When SBT is used in inter-frame CU, the SBT type and SBT position information are further decoded from the bitstream. For example... Figure 12 As shown, there are two SBT types and two SBT positions. For SBT-V (or SBT-H), the TU width (or height) may be equal to half the CU width (or height) or 1 / 4 of the CU width (or height), which is notified by another flag signaling, resulting in a 2:2 split or a 1:3 / 3:1 split. A 2:2 split is like a binary tree (BT) split, while a 1:3 / 3:1 split is like an asymmetric binary tree (ABT) split. If one side of the CU has 8 points in the luminance sample, a 1:3 / 3:1 split along that side is not allowed. Therefore, the CU can have a maximum of 8 SBT modes.

[0174] like Figure 13 As shown, quadtree (QT) splitting is also used to tile a CU into 4 sub-blocks, with one sub-block still having a residual. This SBT type is represented as SBT-Q. This part is not adopted by VVC.

[0175] CUs with width and height not exceeding maxSbtSize are allowed to use SBT-V, SBT-H, and SBT-Q. maxSbtSize is signaled in SPS. For HD and 4K sequences, maxSbtSize is set to 64 by the encoder; for other smaller resolution sequences, maxSbtSize is set to 32.

[0176] 2.11.2 Transformation Types of Sub-blocks

[0177] Position-dependent transformations are applied to the luminance transform blocks in SBT-V and SBT-H (always using DCT-2 chroma TB). The two positions in SBT-H and SBT-V are associated with different core transforms. More specifically, in Figure 12 The code specifies the horizontal and vertical transformations for each SBT location. For example, the horizontal and vertical transformations for SBT-V location 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is greater than 32, the corresponding transformation is set to DCT-2. Therefore, the sub-block transformations collectively specify the TU slice, cbf, and horizontal and vertical transformations of the residual block, which can be considered a syntax shortcut for cases where the main residual of the block is on one side of the block.

[0178] Figure 12 This is a diagram of the sub-block transform modes SBT-V and SBT-H (the gray area represents the TU that can have non-zero coefficients; the white area represents the zero-output TU with all zero coefficients).

[0179] Figure 13 This is a diagram of the subblock transform mode SBT-Q.

[0180] 2.12 Segmentation Tree

[0181] In VTM5, the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. Currently, for P and B slices, the luma CTB and chroma CTB within a CTU must share the same codec tree structure. However, for I slices, luma and chroma can have separate block tree structures. When the separate block tree mode is applied, the luma CTB is split into CUs using one codec tree structure, and the chroma CTB is split into chroma CUs using another codec tree structure. This means that a CU in an I slice may consist of codec blocks for the luma component or codec blocks for both chroma components, while a CU in a P or B slice always consists of codec blocks for all three color components, unless the video is monochrome.

[0182] 3. Examples of the technical problems solved by the solutions described in this document.

[0183] It is necessary to study how to apply ACT to VVC design, especially to address the interaction between ACT and other tools.

[0184] 1. The ILR transforms the luminance components of the entire image / strip / patch from the original domain to a shaped domain and encodes and decodes all content in the shaped domain. However, for the chrominance components, it is encoded and decoded in the original domain. ACT requires obtaining the residual signals of the pixels for the three color components.

[0185] 2. How to handle ACT when dual trees are enabled.

[0186] 3. The current ACT design in VVC7 does not support lossless encoding and decoding.

[0187] 4. ACT's QP adjustment does not take into account the QP offset in the original color space.

[0188] 4. Examples of solutions and implementation methods

[0189] The following list should be considered as examples to illustrate general concepts. These inventions should not be interpreted narrowly. Furthermore, these technologies can be combined in any way.

[0190] In the following discussion, the CU may include information associated with all three color components having a single-tree codec structure. Alternatively, the CU may include information associated only with the luma color component having a monochrome codec. Alternatively, the CU may include information associated only with the luma color component having a dual-tree codec structure (e.g., the Y component in YCbCr format or the G component in GBR format). Alternatively, the CU may include information associated only with the two chroma components having a dual-tree codec structure (e.g., the Cb and Cr components in YCbCr format or the B and R components in GBR format).

[0191] In the following description, "block" may refer to a codec unit (CU) or transform unit (TU) of video data or any rectangular or polygonal region. "Current block" may refer to the codec unit (CU) currently being decoded / encoded or the transform unit (TU) currently being decoded / encoded or any codec rectangular region of video data currently being decoded / encoded. "CU" or "TU" may also be referred to as "codec block" and "transform block".

[0192] In the following discussion, the term "ACT" can refer to any technique that can convert the original / predicted / reconstructed / residual signals of the three color components from one domain to another, without having to be the same as the design in HEVC SCC.

[0193] 1. When a dual-tree segmentation structure is enabled for a video unit (e.g., a strip / piece / brick / picture / area covering one or more CTUs), ACT will be disabled for all blocks in the video unit.

[0194] a.ACT usage instructions can be conditionally signaled based on the use of a dual-tree partitioning structure.

[0195] i. In one example, when applying a dual-tree partitioning structure, the signaling of the ACT usage instruction can be skipped.

[0196] 2. When ILR is enabled for a video unit (e.g., a strip / piece / brick / picture / area covering one or more CTUs), ACT will be disabled for all blocks in the video unit.

[0197] a. The use instructions for ACT may be conditionally signaled based on the use of ILR.

[0198] i. In one example, when applying ILR, the signaling of the ACT usage instruction can be skipped.

[0199] 3. ACT and codec tools X are dedicated to video blocks (e.g., CU / TU).

[0200] a. In one example, X is CCLM. If CCLM is enabled for the chroma component of the video block, ACT will be disabled; otherwise, ACT will be disabled. And vice versa.

[0201] b. In one example, X is Joint Chroma Residual Codec. If Joint Chroma Residual Codec is enabled for the chroma components of a video block, ACT will be disabled; otherwise, ACT will be disabled. The reverse is also true.

[0202] c. In one example, X is a matrix-based intra-prediction method. If a matrix-based intra-prediction method is enabled for the luma component of a video block, ACT will be disabled; otherwise, ACT will be disabled, and vice versa.

[0203] d. In one example, X is QR-BDPCM. If QR-BDPCM is enabled for the luminance component of the video block, ACT will be disabled; otherwise, ACT will be disabled. The reverse is also true.

[0204] e. In one example, X is Subblock Transform (SBT). If SBT is enabled for the luma component of the video block, ACT will be disabled; otherwise, ACT will be disabled, and vice versa.

[0205] f. In one example, X is Multiple Transform Selection (MTS). If MTS is enabled for the luminance component of the video block, ACT will be disabled; otherwise, ACT will be disabled, and vice versa.

[0206] g. In one example, X is the Low-Frequency Inseparable Transform (LFNST). If LFNST is enabled, ACT will be disabled; otherwise, ACT will be disabled. And vice versa.

[0207] h. In one example, X is Pulse Codec Modulation (PCM). If PCM is enabled, ACT will be disabled, and vice versa.

[0208] i. In one example, X is Transform Skip (TS). If TS is enabled, ACT will be disabled; otherwise, ACT will be disabled, and vice versa.

[0209] j. In one example, X is Intra-Frame Sub-Clock Segmentation (ISP). If ISP is enabled, ACT will be disabled, and vice versa.

[0210] k. Alternatively, signaling instructions for the use of ACT can be conditionally sent based on the use of codec tool X.

[0211] i. In one example, when codec tool X is enabled, the signaling of the ACT usage instruction can be skipped.

[0212] l. Alternatively, signaling instructions for the use of tool X can be conditionally sent based on the use of ACT.

[0213] i. In one example, when ACT is enabled, the signaling for the use of codec tool X can be skipped.

[0214] m. Alternatively, the above tools and ACT can be enabled for a single video block.

[0215] 4. Both ACT and dual-tree segmentation structures can be enabled for a single video unit (e.g., image / strip / piece / brick).

[0216] a. Alternatively, the signaling for the use of the dual-tree segmentation structure can be moved from the video unit level to the video block level (e.g., CTU / CTB or VPDU).

[0217] b. Both ACT and dual-tree segmentation structures may be enabled for a single video chunk.

[0218] i. In one example, on the encoder side, ACT can be applied to CTU / CTB before segmenting CTU / CTB.

[0219] ii. In one example, on the decoder side, CTU / CTB can be decoded first, and then the inverse color space transformation can be performed.

[0220] 5. Both ACT and ILR may be enabled for a single video unit (e.g., image / strip / piece / brick).

[0221] a. Alternatively, the signaling used by the ILR can be moved from the video unit level to the video block (e.g., CU / TU) level.

[0222] b.ACT and ILR may be enabled for a video block (e.g., CU / TU).

[0223] i. In one example, on the encoder side, ACT can be applied first, followed by ILR. That is, the predicted signal and the residual signal are first generated in the original domain, ACT is applied to transform the residual signal from the original domain to a different color space domain, and ILR is also used to transform the residual signal to the shaping domain.

[0224] ii. In one example, on the decoder side, ILR can be applied first, followed by inverse color space transformation. That is, ILR is first applied to convert the decoded residual signal from the integer domain to the color space domain; then ACT is applied to convert it from the color space domain back to the original domain.

[0225] 6. ACT and SBT may be enabled for a video block (e.g., CU / TU).

[0226] a. In one example, the predictor error in the transformed color space domain (e.g., the original domain is RGB, and the transformed domain is YCoCg with ACT) is encoded and decoded using two TUs. One has all zero coefficients, and the other has non-zero coefficients.

[0227] i. Alternatively, a TU with non-zero coefficients can be obtained by transformation or transformation jump.

[0228] ii. In one example, signaling notifications could be sent in a manner similar to SBT: how to split it into 2 TUs; and / or what type of transformation could be applied to one of the two TUs.

[0229] 7. For video units (e.g., strips / pieces / blocks / pictures), ACT can be enabled at different levels, such as CU level and TU level.

[0230] a. In one example, for different video blocks in a video unit, the signaling used by ACT can be at different levels, such as CU level and TU level.

[0231] b. It can be determined based on coding characteristics whether to enable / send signaling notifications to the ACT at the CU or TU level, and / or whether to send signaling notifications to the ACT for its use.

[0232] i. In one example, it can be determined whether to enable / send signaling notifications at the CU or TU level based on the dimensions of the current CU. Assume the width and height of the current CU are represented as W and H, respectively.

[0233] 1. For example, it is possible to determine whether to enable / signaling notification ACT at the CU or TU level based on whether the current CU size is larger than the VPDU size.

[0234] a. In one example, if the current CU size is larger than the VPDU size, CU-level signaling notification / use of the ACT can be applied (i.e., all TUs share the same ACT on / off control). Otherwise, TU-level ACT signaling notification / use can be applied.

[0235] b. Alternatively, if the current CU size is larger than the VPDU size, TU-level signaling notification / use of the ACT can be applied (i.e., all TUs share the same ACT on / off control). Otherwise, CU-level ACT signaling notification / use can be applied.

[0236] c. In one example, if the current CU size is larger than the VPDU size, ACT will be disabled without signaling notification.

[0237] 2. In one example, it can be determined whether to enable / signalize an ACT at the CU or TU level based on the current CU having a size greater than the maximum TU size.

[0238] 3. In one example, ACT will be disabled when W>=T1 and H>=T2. For example, T1=T2=32.

[0239] a. Alternatively, ACT will be disabled when W>=T1 or H>=T2. For example, T1=T2=32.

[0240] b. Alternatively, ACT will be disabled when W <= T1 and H <= T2. For example, T1 = T2 = 8.

[0241] c. Alternatively, ACT will be disabled when W <= T1 or H <= T2. For example, T1 = T2 = 8.

[0242] d. Alternatively, ACT will be disabled when W*H >= T. For example, T = 1024.

[0243] e. Alternatively, ACT will be disabled when W*H <= T. For example, T = 64.

[0244] ii. In one example, it can be determined whether to enable / signaling notification ACT at the CU or TU level based on whether the current CU is being encoded and decoded using a sub-block segmentation tool such as ISP.

[0245] In the discussion below, ACT can be used to determine whether a specific color space (such as YCoCg-R) is applied at the sequence / image / strip / piece / block level.

[0246] 8. The QP adjustment of color channels (e.g., Y, Cg, Co) (which may be the difference between the QP of different color components) may be a constant for ACT codec blocks.

[0247] a. In one example, for the YCoCg color space, the QP adjustment of the Y and Cg channels can be equal to a given value K1.

[0248] i. In one example, K1 could be -5.

[0249] ii. Alternatively, the QP adjustment for the Co channel can be equal to another value, K2.

[0250] 1. In one example, K2 can be -1.

[0251] b. In one example, for the YCoCg color space, the QP adjustment for Y can be K1, and the QP adjustment for Cg can be (K1+O).

[0252] i. In one example, K1 could be -5.

[0253] ii. In one example, O can be 6.

[0254] c. In one example, for the YCoCg-R color space, the QP adjustment for Y can be K1, and the QP adjustment for Cg can be (K1+O).

[0255] i. In one example, K1 could be -5.

[0256] ii. In one example, O can be 6.

[0257] d. In one example, for the YCoCg-R color space, the QP adjustment for the Y and / or Cg channels can be K1, and the QP adjustment for Co can be (K1+O).

[0258] i. In one example, K1 could be -5.

[0259] ii. Alternatively, the QP adjustment for the Co channel can be equal to another value, K2.

[0260] 1. In one example, K2 can be -1.

[0261] e. In one example, K1 / O / K2 can be predefined.

[0262] f. In one example, K1 / O / K2 may be indicated at the video unit (e.g., sequence / picture / strip / piece / brick / subpicture) level, such as in VPS and / or DPS and / or SPS and / or PPS and / or picture header and / or strip header.

[0263] i. At least two of K1, K2 and O can be jointly encoded or predictively encoded.

[0264] 9. QP adjustments for the Y, Co, and Cg color channels can be indicated by advanced syntax elements.

[0265] a. In one example, QP adjustments for the Y channel, QP adjustments for the Co channel, and QP adjustments for the Cg channel can be indicated independently using advanced syntax elements.

[0266] b. In one instance, QP adjustments for the Y channel, QP adjustments for the Co channel, and QP adjustments for the Cg channel can be indicated at the video unit (e.g., sequence / picture / strip / piece / brick / subpicture) level, such as in VPS and / or DPS and / or SPS and / or PPS and / or picture headers and / or strip headers.

[0267] 10. The deblocking process may depend on the color space.

[0268] a. In one example, deblocking for one color space can follow deblocking for the 4:2:0 color format, while deblocking for another color space can follow deblocking for the luminance component.

[0269] i. In one example, deblocking for the YCoCg / YCoCg-R color space can be performed after the current deblocking for the 4:2:0 color format.

[0270] ii. In one example, deblocking of all components in the GBR / RGB color space can be performed after the current deblocking of the luminance component.

[0271] b. Whether and / or how the deblocking process is applied may depend on whether ACT is applied.

[0272] 11. When ACT is enabled for a block, the YCoCg-R transform may always be applied.

[0273] a. In one example, when YCoCg-R is applied, the QP adjustment for the (Y, Co, Cg) color channel might be (-5, 3, 1).

[0274] b. In one example, when YCoCg-R is applied, the minimum allowed QP for the transformation skip block can be the same.

[0275] i. In one example, the minimum allowed QP for transforming skip blocks could be QpPrimeTsMin.

[0276] 12. The minimum allowed QP for a transform skip block may be independent of whether ACT is applied.

[0277] a. In one example, the QP adjustment can be added first, such as max(QpPrimeTsMin, derived QP taking into account the QP adjustment), before cutting the QP adjustment of the block of ACT encoding / decoding to a given range.

[0278] b. In one example, the lowest QP of the transform skip block can always be QpPrimeTsMin.

[0279] 13. When ACT is applied, the minimum allowed QP of the transform skip block may depend on the color channel.

[0280] a. In one example, different color channels may have different minimum allowed QPs for the transform skip block.

[0281] b. In one example, the Y and Cg channels may have the same minimum allowed QP for the transform skip block.

[0282] i. Alternatively, the Co channel may have another minimum allowed QP for the transform skip block.

[0283] 14. The above methods can also be applied to other color spaces, not just YCoCg or YCoCg-R color spaces.

[0284] 15. It is recommended to enable only JCCR and Tool X.

[0285] a. Tool X can be a transformation skip mode (e.g., the transform_skip_flag of a block is equal to 1).

[0286] b. Tool X can be a block-based incremental pulse codec modulation (BDPCM) mode (e.g., the block's intra_bdpcm_luma_flag is equal to 1).

[0287] c. Tool X may be ACT.

[0288] d. When tool X is enabled, the indication of JCCR usage will be omitted in the bitstream and inferred to be disabled, and vice versa.

[0289] 16. Whether to enable codec X and / or whether to send auxiliary information of X by signaling and / or the indication used may depend on the quantization parameters associated with the video unit (e.g., strip / picture / subpicture / piece / brick / block).

[0290] a. In one example, if the quantization parameter is greater than (or not less than) the threshold, the codec tool X can be disabled.

[0291] b. In one example, if the quantization parameter is less than (or not greater than) the threshold, the codec tool X can be disabled.

[0292] c. When codec tool X is disabled, auxiliary information and / or signaling instructions for X can be omitted.

[0293] d. In one example, the codec tool X could be JCCR, LMCS, or ACT.

[0294] 5. Other implementation methods

[0295] 5.1. Example #1

[0296] The following changes are based on JVET-P2001-vB.docx to demonstrate the minimum QP (in bold and italics) for changing skip blocks when ACT is used. Deleted text is marked with double brackets (e.g., [[a]] indicates the deletion of the character "a").

[0297] 8.7.3 Scaling process of transformation coefficients

[0298] The input for this process is:

[0299] – Luminance position (xTbY, yTbY), specifies the top-left sample of the current luminance conversion block relative to the top-left luminance sample of the current image.

[0300] – The variable nTbW specifies the width of the transform block.

[0301] – The variable nTbH specifies the height of the transform block.

[0302] – The variable predMode specifies the prediction mode of the codec unit.

[0303] – The variable cIdx specifies the color component of the current block.

[0304] The output of this process is an array d of (nTbW)x(nTbH) with transformation coefficients scaled by d[x][y].

[0305] The quantization parameter qP is derived as follows:

[0306] – If cIdx equals 0, then the following condition applies:

[0307] qP=Qp′ Y (8-936)

[0308] Otherwise, if TuCResMode′[xTbY]][yTbY] equals 2, then the following condition applies:

[0309] qP=Qp′ CbCr (8-937)

[0310] Otherwise, if cIdx equals 1, then the following condition applies:

[0311] qP=Qp′ Cb (8-938)

[0312] Otherwise (cIdx equals 2), the following condition applies:

[0313] qP=Qp′ Cr (8-939)

[0314] The quantization parameter qP is modified, and the variables rectNonTsFlag and bdShift are exported as follows:

[0315] – If transform_skip_flag[xTbY][yTbY][cIdx] equals 0, then the following condition applies:

[0316] qP=qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0) (8-940)

[0317] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH))&1)==1

[0318] (8-941)

[0319] bdShift=BitDepth+(rectNonTsFlag?1:0)+((Log2(nTbW)+Log2(nTbH)) / 2)-5+pic_dep_quant_enabled_flag (8-942)

[0320] Otherwise, if transform_skip_flag[xTbY][yTbY][cIdx] equals 1, then the following condition applies:

[0321]

[0322] rectNonTsFlag = 0 (8-944)

[0323] bdShift = 10 (8-945)

[0324] The variable bdOffset is exported as follows:

[0325] bdOffset = (1 <<bdShift)> >1 (8-946)

[0326] Figure 14AThis is a block diagram of a video processing apparatus 1400. Apparatus 1400 can be used to implement one or more methods described herein. Apparatus 1400 can be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 1400 may include one or more processors 1402, one or more memories 1404, and video processing hardware 1406. The one or more processors 1402 can be configured to implement one or more methods described in this document. The one or more memories 1404 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 1406 can be used to implement some of the techniques described in this document in hardware circuitry.

[0327] Figure 14B This is another example of a block diagram of a video processing system in which the disclosed technologies can be implemented. Figure 14B This is a block diagram illustrating an example video processing system 1400, in which various techniques disclosed herein can be implemented. Various implementations may include some or all of the components of system 1400. System 1400 may include an input 1402 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or in a compressed or encoded format. Input 1402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0328] System 1400 may include codec component 1414, which may implement the various decoding or encoding methods described in this document. Codec component 1414 may reduce the average bit rate of the video from input 1412 to the output of codec component 1414 to generate a codec representation of the video. Therefore, codec techniques are sometimes referred to as video compression or video transcoding techniques. The output of codec component 1414 may be stored or transmitted via connected communication (as shown in component 1416). Component 1418 may use the stored or transmitted bitstream (or encoded) representation of the video received at input 1412 to generate pixel values ​​or displayable video sent to display interface 1420. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “codec” operations or tools, it should be understood that codec tools or operations are used at the encoder and will be performed by the decoder with corresponding decoding tools or operations that are the reverse of the result of codec.

[0329] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described in this document can be embodied in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0330] In some embodiments, the video encoding / decoding method may be used as described above. Figure 14A Or it may be implemented on the hardware platform described in 14B.

[0331] Some embodiments of the disclosed technology involve making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement that tool or mode in the processing of video blocks, but not necessarily modify the resulting bitstream based on the use of that tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from video blocks to a bitstream representation of video will use that video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that it has been modified based on the video processing tool or mode. That is, the conversion from a bitstream representation of video to video blocks will be performed using the video processing tool or mode enabled based on a decision or determination.

[0332] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion from video block to bitstream representation of video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that it has not been modified using a video processing tool or mode enabled based on the decision or determination.

[0333] The disclosed and other solutions, examples, embodiments, modules, and functional operations described herein can be implemented in digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or combinations thereof. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by a data processing apparatus or for controlling the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition affecting machine-readable propagation signals, or a combination thereof. The term "data processing apparatus" includes all means, devices, and machines for processing data, including, for example, programmable processors, computers, or multiprocessors or computer groups. In addition to hardware, the apparatus may also include code that creates an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagation signals are man-made signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiving device.

[0334] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to that program, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed and executed on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[0335] The processing and logic flows described herein can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processing and logic flows can also be executed by special-purpose logic circuitry, and the devices can be implemented as special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0336] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as one or more of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store the instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or receive data from or transfer data to one or more mass storage devices via operative coupling, or both. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0337] Figure 15 This is a block diagram illustrating an example video coding system 100 that can utilize the techniques disclosed herein.

[0338] like Figure 15 As shown, the video encoding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data and may be referred to as a video encoding device. The destination device 120 can decode the encoded video data generated by the source device 110 and may be referred to as a video decoding device.

[0339] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0340] Video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system that generates video data, or a combination of such sources. Video data may include one or more pictures. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming an encoded representation of the video data. The bitstream may include encoded pictures and associated data. An encoded picture is an encoded representation of a picture. Associated data may include a set of sequence parameters, a set of picture parameters, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. Encoded video data may be transmitted directly to destination device 120 via network 130a through I / O interface 116. Encoded video data may also be stored on storage medium / server 130b for access by destination device 120.

[0341] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[0342] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120, or may be external to destination device 120 configured to connect to an external display device.

[0343] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Coding (HEVC) standard, the Multi-Functional Video Coding (VVC) standard, and other current and / or other standards.

[0344] Figure 16 This is a block diagram illustrating an example of a video encoder 200, which can be... Figure 15 The video encoder 114 in the system 100 shown in the figure.

[0345] The video encoder 200 can be configured to perform any or all of the techniques disclosed herein. Figure 16 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor may be configured to perform any or all of the techniques described in this invention.

[0346] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203), a motion estimation unit 204, a motion compensation unit 205, an intra-frame prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding / decoding unit 214.

[0347] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in IBC mode, where at least one reference picture is the picture containing the current video block.

[0348] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, can be highly integrated, but for interpretative purposes... Figure 16 The examples are presented respectively.

[0349] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0350] The mode selection unit 203 can, for example, select one of the intra-frame or inter-frame coding modes based on the error result, and provide the resulting intra-frame or inter-frame codec block to the residual generation unit 207 to generate residual block data and to the reconstruction unit 212 to reconstruct the codec block for use as a reference picture. In some examples, the mode selection unit 203 can select a combined intra-frame and inter-frame prediction (CIIP) mode, in which the prediction is based on inter-frame prediction signaling notification and intra-frame prediction signaling notification. The mode selection unit 203 can also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision) in the case of inter-frame prediction.

[0351] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on motion information and decoded samples from images in buffer 213 other than the image associated with the current video block.

[0352] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations for the current video block, for example, depending on whether the current video block is in an I-band, P-band, or B-band.

[0353] In some examples, motion estimation unit 204 can perform unidirectional prediction for the current video block, and can search for a reference video block for the current video block in the reference images of list 0 or list 1. Then, motion estimation unit 204 can generate a reference index indicating the reference images in list 0 and list 1 containing the reference video block, and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video.

[0354] In other examples, motion estimation unit 204 can perform bidirectional prediction for the current video block. Motion estimation unit 204 can search for a reference video block for the current video block in the reference images in list 0, and can also search for another reference video block for the current video block in the reference images in list 1. Motion estimation unit 204 can then generate a reference index and a motion vector, where the reference index indicates the reference images in lists 0 and 1 that contain the reference video block, and the motion vector indicates the spatial displacement between the reference video block and the current video block. Motion estimation unit 204 can output the reference index and the motion vector of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0355] In some examples, the motion estimation unit 204 can output a complete set of motion information for use in the decoder's decoding process.

[0356] In some examples, motion estimation unit 204 may not output the complete set of motion information for the current video. Instead, motion estimation unit 204 may signal the motion information of the current video block by referencing the motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0357] In one example, motion estimation unit 204 may indicate a value in the syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0358] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) within the syntactic structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0359] As discussed above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling notification techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling Notification.

[0360] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[0361] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., using a negative sign) multiple predicted video blocks from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of the samples in the current video block.

[0362] In other examples, there may be no residual data for the current video block, such as in skip mode, and the residual generation unit 207 may not perform subtraction operations.

[0363] The transform processing unit 208 can generate one or more transform coefficient video blocks of the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[0364] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0365] Inverse quantization unit 210 and inverse transform unit 211 can apply inverse quantization and inverse transform to transform coefficient video blocks, respectively, to reconstruct residual video blocks from transform coefficient video blocks. Reconstruction unit 212 can add the reconstructed residual video blocks to the corresponding samples of one or more predicted video blocks generated by prediction unit 202 to produce reconstructed video blocks associated with the current block, which are then stored in buffer 213.

[0366] After the video block is reconstructed by the reconstruction unit 212, a loop filtering operation can be performed to reduce the video block blocking artifact.

[0367] The entropy encoding / decoding unit 214 can receive data from other functional components of the video encoder 200. When the entropy encoding / decoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0368] Figure 17 This is a block diagram illustrating an example of a video decoder 300, which can be... Figure 15 The video decoder 114 in the illustrated system 100.

[0369] The video decoder 300 can be configured to perform any or all of the techniques disclosed herein. Figure 17 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some instances, the processor can be configured to perform any or all of the techniques described in this invention.

[0370] exist Figure 17 In the example, video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, video decoder 300 can perform decoding processing (pass), which is generally related to video encoder 200 (e.g., Figure 16 The encoding processes described are inverses of each other.

[0371] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., codec blocks of video data). The entropy decoding unit 301 can decode the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 can determine motion information, including motion vectors, motion vector precision, reference image list index, and other motion information. The motion compensation unit 302 can determine such information, for example, by executing AMVP and Merge modes.

[0372] The motion compensation unit 302 can generate blocks of motion compensation, possibly performing interpolation based on an interpolation filter. The identifier of the interpolation filter used at sub-pixel precision can be included in the syntax element.

[0373] The motion compensation unit 302 can use an interpolation filter, such as that used by the video encoder 20, during the encoding of a video block to calculate the interpolated values ​​of a sub-integer number of pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and use the interpolation filter to generate the prediction block.

[0374] The motion compensation unit 302 can use some syntax information to determine: the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a mode indicating how to encode each segment, one or more reference frames (and a list of reference frames) for each inter-frame encoded block, and other information for decoding the encoded video sequence.

[0375] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 303 inverse-quantizes (i.e., de-quantizes) the quantized video block coefficients, which are provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[0376] The reconstruction unit 306 can add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to form a decoded block. If necessary, a deblocking filter can also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation.

[0377] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from the pixel representation of a video to the corresponding bitstream representation, and vice versa. As defined in the syntax, the bitstream representation of the current video block can, for example, correspond to bits co-occurring within the bitstream or extended at different positions within the bitstream. For example, a macroblock can be encoded based on the transformed and encoded / decoded error residuals, and also using bits in the header and other fields in the bitstream.

[0378] The various solutions and embodiments described in this document are further described using a list of terms. The first set of terms shows example embodiments of the techniques discussed in the preceding sections.

[0379] Item 1 in the preceding sections provides further examples of the following clauses.

[0380] 1. A video processing method comprising: determining to disable the use of adaptive color space transformation (ACT) for video units since a dual-tree segmentation structure is used for conversion between video units and bitstream representations of video units; and performing a conversion by disabling ACT for video units based on the determination.

[0381] 2. According to the method of Clause 1, wherein the bit stream represents bits excluded from providing information about the use of the ACT.

[0382] Item 2 in the preceding sections provides further examples of the following clauses.

[0383] 1. A video processing method, comprising: determining to disable adaptive color space transformation (ACT) for conversion since an in-loop shaper is enabled for conversion between video units and bitstream representations of video units; and performing the conversion by disabling ACT for the video units based on the determination.

[0384] 2. The method according to Clause 1, wherein the bit stream representation includes bits used to provide information about the use of the ILR.

[0385] Item 3 in the preceding sections provides further examples of the following clauses.

[0386] 3. A video processing method. It includes: determining, since encoding / decoding tools are used for conversion between video units and bitstream representations of video units, that an adaptive color space transformation tool is disabled for the conversion; and based on this determination, performing the conversion by disabling ACT for the video units.

[0387] 4. The method according to Clause 3, wherein the encoding / decoding tool corresponds to the cross-component linear model tool.

[0388] 5. The method according to Clause 3, wherein the encoding / decoding tool corresponds to joint chroma residual encoding / decoding.

[0389] 6. The method according to Clause 3, wherein the codec tool corresponds to a Multiple Transform Selection (MTS) codec tool.

[0390] Item 4 in the preceding sections provides further examples of the following clauses.

[0391] 7. A video processing method comprising: determining that a dual-tree segmentation structure and an adaptive color space transformation (ACT) codec tool are both used for conversion between video units and bitstream representations of video units; and, based on the determination, performing the conversion by enabling ACT for the video units.

[0392] 8. The method according to Clause 7, wherein the bitstream represents signaling comprising a dual-segment tree structure at the video block level, which is a finer level than the video unit level.

[0393] Item 5 in the preceding sections provides further examples of the following clauses.

[0394] 9. A video processing method comprising: determining that both In-Loop Renormalization (ILR) and Adaptive Color Space Transformation (ACT) codec tools are used for conversion between video units and bitstream representations of video units; and performing the conversion based on the determination using the ILR and ACT codec tools.

[0395] 10. The method according to Clause 9, wherein performing the conversion comprises: first applying the ACT encoding / decoding tool during encoding, and then applying the ILR if ACT is applied.

[0396] 11. The method according to Clause 9, wherein performing the transformation comprises: first applying the ILR, and then applying the inverse color space transformation to the result of the ILR.

[0397] Item 6 in the preceding sections provides further examples of the following clauses.

[0398] 12. A video processing method comprising: using both a deterministic (SBT) and an adaptive color space transformation (ACT) codec tool for conversion between video units and bitstream representations of video units; and performing the conversion using the SBT and ACT codec tools based on the deterministic method.

[0399] 13. The method according to Clause 12, wherein the prediction error during the conversion is transformed from the RGB color space to the YCoCg color space.

[0400] 14. The method according to Clause 13, wherein, after the transformation, the prediction error is encoded and decoded using at least two transform units (TUs).

[0401] Item 7 in the preceding sections provides further examples of the following clauses.

[0402] 15. The method according to any of the above clauses, wherein the video unit comprises strips, pieces, blocks, or pictures.

[0403] 16. The method according to Clause 15, wherein the determination is performed at the sub-video unit level, wherein the sub-unit level corresponds to a codec unit (CU) or a transform unit (TU).

[0404] 17. The method according to Clause 16, wherein the determination at the sub-video unit level is based on the coding characteristics of the sub-unit level.

[0405] 18. The method according to Clause 17, wherein the encoding characteristics include the dimensions of the CU and / or the dimensions of the TU and / or the relationship between the dimensions of the CU and the dimensions of the TU.

[0406] 19. The method according to any of the foregoing clauses, wherein the conversion includes decoding the bitstream representation to generate a video unit.

[0407] 20. The method according to any of the foregoing clauses, wherein the conversion includes encoding video units into a bitstream representation.

[0408] 21. A video encoder apparatus, comprising a processor configured to implement any one or more of the methods described above.

[0409] 22. A video decoder apparatus, comprising a processor configured to implement any one or more of the methods described in the foregoing clauses.

[0410] 23. A computer-readable medium having code for implementing the methods described in any one or more of the foregoing clauses stored thereon.

[0411] The second set of clauses describes certain features and aspects of the technology disclosed in the preceding sections (e.g., items 8 to 15).

[0412] 1. A video processing method (e.g., Figure 18A The method shown (1810) includes: for the conversion between the video components of the video including the current video block and the bitstream representation of the video, determining (1812) an adaptive color space transformation (ACT) tool based on the rules of the current video block; and performing (1814) a conversion according to the determination, wherein the rules specify that each of one or more color components of the current video block uses the same constant quantization parameter (QP) offset.

[0413] 2. The method according to Clause 1, wherein the ACT tool includes transforming the representation of a visual signal between two color spaces.

[0414] 3. According to the method of Clause 1, wherein the rule specifies that for the first color space, the QP offset for the luminance and first chroma channels is equal to K1, where K1 is an integer.

[0415] 4. According to the method of Clause 3, wherein the rule specifies that the QP offset for the second channel is equal to K2, where K2 is an integer.

[0416] 5. According to the method of Clause 3 or 4, where K1 is -5 and K2 is -1.

[0417] 6. The method according to Clause 4 or 5, wherein the first color space is the YCgCo color space, the first chromaticity channel is the Cg channel, and the second chromaticity channel is the Co channel.

[0418] 7. The method according to Clause 1, wherein the rule specifies that for the first color space, the QP offset for the luminance channel is equal to K1, and the QP offset for the first chroma channel is K1+O, where K1 and O are integers.

[0419] 8. According to the method of Clause 7, where K1 is -5 and O is 6.

[0420] 9. The method according to Clause 1, wherein the rule specifies that for the first color space, the QP offset for the luminance channel is equal to K1, and the QP offset for the second channel is equal to K1+O, where K1 and O are integers.

[0421] 10. The method according to any one of clauses 7-9, wherein the rule specifies that the QP offset for the second channel is equal to K2, where K2 is an integer different from K1.

[0422] 11. According to the method of Clause 10, where K2 is 3.

[0423] 12. The method according to any one of clauses 7 to 11, wherein the first color space is YCgCo or YCgCo-R color space, the first chromaticity channel is a Cg channel, and the second chromaticity channel is a Co channel.

[0424] 13. The method according to any one of clauses 7 to 11, wherein the first color space is the YCbCr color space, the first chromaticity channel is the Cr channel, and the second chromaticity channel is the Cb channel.

[0425] 14. The method according to any one of clauses 3 to 13, wherein at least one of K1, K2 or O is predefined.

[0426] 15. The method according to any one of clauses 3 to 13, wherein at least one of K1, K2 or O is indicated in the bitstream representation at the video unit level.

[0427] 16. The method according to any one of clauses 3 to 13, wherein at least two of K1, K2 and O are jointly encoded or predictively encoded.

[0428] 17. The method according to Clause 1, wherein the QP offset for the Y color channel, the QP offset for the Co color channel, and the QP offset for the Cg color channel are independently indicated in the bitstream representation by high-level syntax elements.

[0429] 18. The method according to Clause 1, wherein the QP offset for the Y color channel, the QP offset for the Co color channel, and the QP offset for the Cg color channel are indicated in the bitstream representation at the video unit level.

[0430] 19. The method according to Clause 1, wherein color transformation is applied to the current video block due to the use of the ACT tool.

[0431] 20. The method pursuant to Clause 1 also includes:

[0432] The transformation skip process is determined to be used for the current video block;

[0433] Based on determining that a QP offset is added to the original QP, an updated QP is derived for each color channel; and

[0434] Cut the updated QP to the predetermined range.

[0435] 21. The method according to Clause 20, wherein the same minimum allowed QP is used to export the updated QP of the color channels.

[0436] 22. The method according to Clause 21, wherein the same minimum allowed QP is set to QpPrimeTsMin.

[0437] 23. The method pursuant to Clause 20, wherein a minimum QP is permitted independent of the applicability of the ACT tool.

[0438] 24. In accordance with the method of Clause 23, wherein the minimum allowed QP is always set to QpPrimeTsMin.

[0439] 25. A video processing method (e.g., such as...) Figure 18B The method shown (1820) includes: for the conversion between video blocks and bitstream representations of a video, determining (1822) whether and / or how to apply a deblocking process according to rules; and performing (1824) a conversion based on the determination, wherein the rules specify that the deblocking process depends on: i) the enabled state of Adaptive Color Space Transformation (ACT), in which the representation of the visual signal is transformed between two color spaces, or ii) the color space used to represent the samples of the pixels.

[0440] 26. The method according to Clause 25, wherein the rule further specifies that: deblocking for the first color space follows deblocking for the 4:2:0 color format, and deblocking for the second color space follows deblocking for the luminance component.

[0441] 27. The method according to Clause 26, wherein the first color space corresponds to the YCoCg color space or the YCoCg-R color space, and the second color space corresponds to the GBR or RGB color space.

[0442] 28. A video processing method (e.g., such as...) Figure 18CThe method shown (1830) includes: for the conversion between the current video block of the video and the codec representation of the video, determining (1832) that a color conversion is performed due to the use of an Adaptive Color Space Transformation (ACT) codec tool; and performing (1834) the conversion based on the determination.

[0443] 29. The method according to Clause 28, wherein the color transformation includes applying the YCoCg_R transformation.

[0444] 30. The method according to Clause 28, wherein, due to the use of the ACT codec tool, the quantization parameter (QP) offset for the Y, Co, Cg color channels is (-5, 3, 1).

[0445] 31. According to the method of Clause 28, the rule also specifies that the same minimum allowed QP be used for another video block encoded and decoded using the transform skip mode.

[0446] 32. According to the method of Clause 31, the minimum allowed QP is QpPrimeTsMin.

[0447] 33. A video processing method (e.g., such as...) Figure 18D The method shown (1840) includes: performing (1842) a conversion between a video and a bitstream representation of a video containing one or more video blocks according to a rule, wherein the rule specifies that for video blocks encoded using a transform skip mode, the minimum allowed quantization parameter (QP) is independent of the applicability of the adaptive color space transformation (ACT) tool to the video blocks.

[0448] 34. According to the method of Clause 33, the rule also specifies that the quantization parameter (QP) adjustment for the video block is added before it is cut to a given range.

[0449] 35. According to the method of Clause 33, the rule also specifies that the minimum allowed QP is always QpPrimeTsMin.

[0450] 36. A video processing method comprising: performing a conversion between a video containing one or more video blocks and a bitstream representation of the video according to a rule, wherein the rule specifies that for a video block encoded using a transform skip mode, the minimum allowed quantization parameter (QP) depends on the color channel.

[0451] 37. The method according to Clause 36, wherein the rule specifies that different minimum allowed QPs are used for different color channels.

[0452] 38. The method according to Clause 36, wherein the rule specifies that the same minimum allowed QP is used for the Y channel and the Cg channel.

[0453] 39. The method according to any of the preceding clauses, wherein the rule is applied to any color space including the YCoCg color space, the YCbCr color space, or the YCoCg-R color space.

[0454] 40. A video processing method (e.g., such as...) Figure 18E The method shown in 1850 includes: determining (1852) an exclusion rule applicable to the conversion between a video region of the video and a bitstream representation of the video, wherein the exclusion rule specifies that the conversion is not permitted to use the joint codec tool and the chroma residual (JCCR) tool for the video region; and performing (1854) the conversion based on the determination.

[0455] 41. The method according to Clause 40, wherein the encoding / decoding tool is a transform skip mode, the transform skip mode comprising: encoding the residual of a video block during encoding without applying a non-identifying transform, or determining the decoded video block during decoding without applying an inverse non-identifying transform to the residual encoded / decoded in the bitstream representation.

[0456] 42. The method according to Clause 40, wherein the encoding / decoding tool is a block-based incremental pulse codec modulation (BDPCM) mode, which corresponds to an intra-frame encoding / decoding tool that uses differential pulse codec modulation (DPCM) at the block level.

[0457] 43. The method according to Clause 40, wherein the encoding / decoding tool is an adaptive color space transformation (ACT) tool, in which the representation of the visual signal is transformed between two color spaces.

[0458] 44. The method according to Clause 40, wherein the bitstream representation omits the indication of the use of the JCCR tool, and if the encoding / decoding tool is enabled, it is inferred that the JCCR tool is disabled.

[0459] 45. The method according to Clause 40, wherein, when the JCCR tool is enabled, the bitstream representation omits the instruction for the use of the encoding / decoding tool and is inferred to be disabled.

[0460] 46. ​​A video processing method (e.g., such as...) Figure 18D The method shown (1840) includes: for the conversion between video units and bitstream representations of video, determining (1852) the suitability of the codec tool and / or whether to notify the codec tool of information and / or instructions to be used by signaling based on quantization parameters for the conversion of video units; and performing (1854) the conversion based on the determination.

[0461] 47. The method according to Clause 46, wherein the encoding / decoding tool is disabled when the quantization parameter is greater than or not less than a threshold.

[0462] 48. The method according to Clause 46, wherein the encoding / decoding tool is disabled when the quantization parameter is less than or not greater than a threshold.

[0463] 49. The method according to Clause 46, wherein, in the case that the encoding / decoding tool is disabled, the bitstream representation omits information about the encoding / decoding tool and / or instructions on its use.

[0464] 50. The method according to Clause 46, wherein the encoding / decoding tool corresponds to a Joint Codec for Chromaticity Residuals (JCCR) tool, a Luminance Mapping with Chromaticity Scaling (LMCS) tool, or an Adaptive Color Space Transformation (ACT) tool.

[0465] 51. The method according to any one of clauses 1 to 50, wherein the conversion includes encoding the video into a bitstream representation.

[0466] 52. The method according to any one of clauses 1 to 50, wherein the conversion includes decoding video from a bitstream representation.

[0467] 53. A video processing apparatus comprising: a processor configured to implement any one or more of the methods of clauses 1 to 52.

[0468] 54. A computer-readable medium storing program code, which, when executed, causes a processor to implement one or more of the methods pursuant to clauses 1 to 52.

[0469] 55. A computer-readable medium storing a codec representation or bitstream representation generated according to any one of the methods described above.

[0470] As will be understood from the foregoing, specific embodiments of the currently disclosed technology have been described herein for illustrative purposes, but various modifications may be made without departing from the scope of the invention. Therefore, the currently disclosed technology is not limited except for the appended claims.

[0471] The implementation of the subject matter and functional operations described in this patent document can be implemented in various systems, digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials embodying machine-readable propagation signals, or combinations thereof. The terms "data processing unit" or "data processing apparatus" include all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or combinations thereof.

[0472] A computer program (also called a program, software, software application, script, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on a single computer, or on multiple computers located at a single site or distributed across multiple sites and interconnected via a communication network.

[0473] The processes and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by special-purpose logic circuitry, such as field-programmable gate arrays or application-specific integrated circuits (ASICs), and the apparatus can also be implemented as such special-purpose logic circuitry.

[0474] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any kind of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The fundamental elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, to receive data from or transfer data to, or both. However, a computer does not require such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0475] The specification and accompanying drawings are to be considered exemplary only, where exemplary means example. As used herein, the word "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0476] Although this patent document contains numerous details, these should not be construed as limiting any invention or the scope of the claims, but rather as descriptions of specific features of specific embodiments of a particular invention. Some features described in the context of individual embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually in multiple embodiments or in any suitable sub-combination. Furthermore, although features may be described above as functioning in some combinations, and even initially claimed in this way, in some cases, one or more features from the claimed combination may be removed from that combination, and the claimed combination may be for sub-combinations or variations thereof.

[0477] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or sequence shown, or requiring all the operations shown to obtain the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0478] Only a few implementations and examples are described, and other implementations, enhancements and variations may be made based on what is described and shown in this patent document.

Claims

1. A method for processing video data, comprising: For the conversion between the video including the current video block and the bitstream of the video, the first encoding / decoding tool is determined based on the rules for the current video block; as well as The conversion is performed based on the determination. The first encoding / decoding tool includes a representation of visual signals that transforms between two color spaces; The rule specifies that the first quantization parameter offset for the luminance channel of the current video block is equal to K1, and the second quantization parameter offset for the first chroma channel of the current video block is K1+0, where K1 and 0 are integers. The rule further specifies that all video blocks encoded and decoded using the transform skip mode should use the same minimum allowed quantization parameter; and The minimum allowed quantization parameter is independent of whether the first encoding / decoding tool is applied to a video block encoded / decoded using the transform skip mode.

2. The method according to claim 1, wherein, K1 is -5 and O is 6.

3. The method of claim 1, wherein, The third quantization parameter offset for the second chroma channel of the current video block is equal to 3.

4. The method of claim 1, wherein, The rule also specifies that the quantization parameter is added to either the first quantization parameter offset for the luminance channel of the current video block or the second quantization parameter offset for the first chroma channel of the current video block before being clipped to a given range.

5. The method of claim 1, wherein, The current video block is in any color space, including YCoCg, YCbCr, or YCoCg-R.

6. The method of claim 1, wherein, When the current video block is in the YCgCo or YCgCo-R color space, the first chroma channel is the Cg channel.

7. The method of claim 1, wherein, The conversion includes encoding the video into the bitstream.

8. The method according to claim 1, wherein, The conversion includes decoding the video from the bitstream.

9. An apparatus for processing video data, comprising a processor and a non-transitory memory thereon having instructions, wherein the instructions, when executed by the processor, cause the processor to: For the conversion between the video including the current video block and the bitstream of the video, the first encoding / decoding tool is determined based on the rules for the current video block; and The conversion is performed based on the determination. wherein The first encoding / decoding tool includes transforming the representation of visual signals between two color spaces; The rule specifies that the first quantization parameter offset for the luminance channel of the current video block is equal to K1, and the second quantization parameter offset for the first chroma channel of the current video block is K1+0, where K1 and 0 are integers. The rule further specifies that all video blocks encoded and decoded using the transform skip mode should use the same minimum allowed quantization parameter; and The minimum allowed quantization parameter is independent of whether the first encoding / decoding tool is applied to a video block encoded / decoded using the transform skip mode.

10. The apparatus of claim 9, wherein, K1 is -5 and O is 6; and The third quantization parameter offset for the second chroma channel of the current video block is equal to 3.

11. The apparatus according to claim 9, wherein, The rule also specifies that the quantization parameter is added to either the first quantization parameter offset for the luminance channel of the current video block or the second quantization parameter offset for the first chroma channel of the current video block before being clipped to a given range.

12. The apparatus of claim 9, wherein, The current video block is in any color space, including YCoCg, YCbCr, or YCoCg-R color spaces; and Wherein, when the current video block is in the YCgCo or YCgCo-R color space, the first chroma channel is the Cg channel.

13. A non-transitory computer-readable storage medium for storing instructions, said instructions causing a processor to: For the conversion between the video including the current video block and the bitstream of the video, the first encoding / decoding tool is determined based on the rules for the current video block; and The conversion is performed based on the determination. wherein, The first encoding / decoding tool includes transforming the representation of visual signals between two color spaces; The rule specifies that the first quantization parameter offset for the luminance channel of the current video block is equal to K1, and the second quantization parameter offset for the first chroma channel of the current video block is K1+0, where K1 and 0 are integers. The rule further specifies that all video blocks encoded and decoded using the transform skip mode should use the same minimum allowed quantization parameter; and The minimum allowed quantization parameter is independent of whether the first encoding / decoding tool is applied to a video block encoded / decoded using the transform skip mode.

14. The non-transitory computer-readable storage medium of claim 13, wherein, K1 is -5 and O is 6; and The third quantization parameter offset for the second chroma channel of the current video block is equal to 3.

15. The non-transitory computer-readable storage medium according to claim 13, wherein The rule also specifies that the quantization parameter is added to the first quantization parameter offset for the luminance channel of the current video block or the second quantization parameter offset for the first chroma channel of the current video block before being limited to a given range. Wherein, the current video block is in any color space including YCoCg, YCbCr, or YCoCg-R color spaces; and Wherein, when the current video block is in the YCgCo or YCgCo-R color space, the first chroma channel is the Cg channel.

16. A non-transitory computer-readable recording medium having stored thereon a computer program and a bitstream, wherein, When the computer program is executed by a processor, it implements a method to generate the bit stream, the method comprising: For a video that includes the current video block, the first encoding / decoding tool is determined based on the rules for the current video block; and The bit stream is generated based on the determination. The first encoding / decoding tool includes a representation of visual signals that transforms between two color spaces; The rule specifies that the first quantization parameter offset for the luminance channel of the current video block is equal to K1, and the second quantization parameter offset for the first chroma channel of the current video block is K1+0, where K1 and 0 are integers. The rule further specifies that all video blocks encoded and decoded using the transform skip mode should use the same minimum allowed quantization parameter; and The minimum allowed quantization parameter is independent of whether the first encoding / decoding tool is applied to a video block encoded / decoded using the transform skip mode. 17.The non-transitory computer-readable recording medium of claim 16, wherein, K1 is -5 and O is 6; and The third quantization parameter offset for the second chroma channel of the current video block is equal to 3.

18. The non-transitory computer-readable recording medium according to claim 16, wherein, The rule also specifies that the quantization parameter is added to the first quantization parameter offset for the luminance channel of the current video block or the second quantization parameter offset for the first chroma channel of the current video block before being limited to a given range. Wherein, the current video block is in any color space including YCoCg, YCbCr, or YCoCg-R color spaces; and Wherein, when the current video block is in the YCgCo or YCgCo-R color space, the first chroma channel is the Cg channel.

19. A method for storing a bitstream of video, comprising: For the current video block, the first encoding / decoding tool is determined based on the rules for the current video block; The bit stream is generated based on the determination; as well as The bitstream is stored in a non-transitory computer-readable storage medium. The first encoding / decoding tool includes a representation of visual signals that transforms between two color spaces; The rule specifies that the first quantization parameter offset for the luminance channel of the current video block is equal to K1, and the second quantization parameter offset for the first chroma channel of the current video block is K1+0, where K1 and 0 are integers. The rule further specifies that all video blocks encoded and decoded using the transform skip mode should use the same minimum allowed quantization parameter; and The minimum allowed quantization parameter is independent of whether the first encoding / decoding tool is applied to a video block encoded / decoded using the transform skip mode.

Images