Position-based pattern derivation in downscaling quadratic transformation of video

Abstract

A method for video processing is described. The method includes: for conversion between a current video block of a video unit of a video and a coded representation of the video, determining a default intra-frame prediction mode for a video unit coded and decoded using a certain intra-frame prediction mode, so that a prediction block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction; and performing conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is filed in order to timely claim the priority and benefits of International Patent Application No. PCT / CN2019 / 099158 filed on August 3, 2019, in accordance with applicable patent laws and / or the Paris Convention. For all legal purposes, the entire disclosure of the above application is incorporated by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to video processing technology, devices and systems. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth requirements for digital video usage will continue to grow. Summary of the invention

[0005] The apparatus, system and method relate to digital video coding and, in particular, to context modeling of residual coding in video coding. The described method can be applied to existing video coding standards (e.g., High Performance Video Codec (HEVC)) as well as future video coding standards or both.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a current video block of a video unit of a video and a codec representation of the video, determining a default intra-frame prediction mode for a video unit coded and decoded using a certain intra-frame prediction mode, so that a prediction block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction; and performing conversion based on the determination.

[0007] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: using a rule to determine a predetermined position where a luma block of a video covers a chroma block of the video; and performing a conversion between the video and a codec representation of the video based on the determination, wherein the chroma block is represented in the codec representation using an intra-frame prediction mode.

[0008] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: using a rule to derive an intra-frame prediction mode of a chrominance block of a video based on a codec mode of a luminance block corresponding to the chrominance block; and performing conversion between the luminance block and a codec representation of the video based on the derived intra-frame prediction mode, wherein the rule specifies using a default intra-frame prediction mode in case the codec mode of the luminance block is a certain intra-frame prediction mode, wherein a prediction block of the luminance block is generated without extrapolating neighboring pixels of the luminance block along a direction.

[0009] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a chroma block of a video, making a first determination whether to apply a non-normal chroma intra-frame prediction mode to the chroma block of the video; for a luma block corresponding to the chroma block, making a second determination to apply the luma intra-frame prediction mode to the luma block; making a third determination to apply a transform set or a transform matrix to the chroma block based on the luma intra-frame prediction mode; and performing conversion between the video and a coded representation of the video according to the third determination, and wherein the non-normal chroma intra-frame prediction mode includes encoding and decoding the chroma block without using neighborhood pixel values ​​extrapolated along the chroma prediction direction.

[0010] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a chroma block of a video, making a first determination that a luma block corresponding to the chroma block is encoded and decoded using a non-normal luma intra-frame prediction mode; based on the first determination, making a second determination of a transform set or transform matrix for the chroma block according to a rule; and performing conversion between the video and a coded representation of the video according to the second determination, and wherein the rule specifies one or more default modes or default transform sets associated with the chroma block due to the luma block being encoded and decoded using a non-normal luma intra-frame prediction mode to determine the transform set or transform matrix in case the chroma block is encoded and decoded using a non-normal chroma intra-frame prediction mode, wherein the non-normal luma intra-frame prediction mode includes encoding and decoding the luma block without using neighborhood pixel values ​​extrapolated along a chroma prediction direction.

[0011] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a current video block of a video and a codec representation of the video, determining the applicability of applying a secondary transform tool to the current video block of a color component based on at least one of: 1) a codec mode of a corresponding block of another color component, or 2) a codec mode of the current video block; and performing the conversion based on the determination, and wherein the secondary transform tool is used: during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block before quantization, or during decoding, an inverse primary transform is applied to a dequantized output of the current video block before applying an inverse primary transform.

[0012] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for a chrominance block of a video, making a first determination that a luma block at a predefined position covering the chrominance block is encoded using a non-normal luma intra-frame prediction mode; based on the first determination, making a second determination to apply the predefined intra-frame prediction mode to the chrominance block because the chrominance block is encoded using the non-normal luma intra-frame prediction mode; and performing a conversion of the video to a codec representation of the video based on the second determination, wherein the non-normal luma intra-frame prediction mode includes encoding the luma block without using neighborhood pixel values ​​extrapolated along a luma prediction direction.

[0013] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a chrominance block of a video, making a first determination that a luma block at a predefined position covering the chrominance block is encoded using a normal luma intra prediction mode; based on the first determination, making a second determination to derive a chrominance intra prediction mode based on the normal luma intra prediction mode of the luma block; and performing conversion of the video to a codec representation of the video based on the second determination, wherein the normal luma intra prediction mode includes encoding the luma block using neighborhood pixel values ​​extrapolated along a luma prediction direction.

[0014] In yet another representative aspect, the above method is implemented in the form of processor executable code and stored in a computer readable medium.

[0015] In yet another representative aspect, an apparatus configured or operable to perform the above method is disclosed. The apparatus may include a processor programmed to implement the method.

[0016] In yet another representative aspect, a video decoder device may implement the methods described herein.

[0017] The above and other aspects and features of the disclosed technology will be described in more detail in the drawings, the specification and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A block diagram of an example encoder is shown.

[0019] Figure 2 Examples of 67 intra prediction modes are shown.

[0020] Figure 3 An example of ALWIP for a 4x4 block is shown.

[0021] Figure 4 An example of ALWIP for an 8x8 block is shown.

[0022] Figure 5 An example of ALWIP for an 8x4 block is shown.

[0023] Figure 6 An example of ALWIP for a 16x16 block is shown.

[0024] Figure 7 An example of four reference rows for a neighborhood prediction block is shown.

[0025] Figure 8 Examples of partitioning into 4×8 blocks and 8×4 blocks are shown.

[0026] Fig. 9 Examples of partitioning of all blocks except 4×8, 8×4, 4×4 are shown.

[0027] Fig.10 An example of a quadratic transform in JEM is shown.

[0028] Fig.11 An example of the proposed Reduced Secondary Transform (RST) is shown.

[0029] Fig.12 Examples of forward and reverse downscaling transforms are shown.

[0030] Fig.13 An example of a forward RST 8×8 process using a 16×48 matrix is ​​shown.

[0031] Fig.14 An example of a zeroing region for an 8×8 matrix is ​​shown.

[0032] Fig.15 Examples of sub-block transform modes SBT-V and SBT-H are shown.

[0033] Fig.16 An example of a diagonal upward right scan order for a 4×4 codec group is shown.

[0034] Fig.17 An example of a diagonal top-right scan order of 8×8 blocks with a 4×4 size codec group is shown.

[0035] Fig.18 An example of a template for selecting a probabilistic model is shown.

[0036] Fig.19 An example of two scalar quantizers for correlated quantization is shown.

[0037] Fig. 20 Examples of state transformation and quantizer selection for the proposed correlation quantization process are shown.

[0038] Fig.21 is an example of an 8×8 block with 4 codec groups.

[0039] Figures 22A to 22F A flow chart illustrating an example method of video processing is shown.

[0040] Fig.23A and 23B is a block diagram of an example of a hardware platform that implements the visual media decoding or visual media encoding techniques described in this document.

[0041] Fig.24A and 24B An example of a chroma block and its corresponding luminance block under dual-tree partitioning is shown.

[0042] Fig.25 An example of the 'CR' position of the derived DM from the corresponding luma block is shown.

[0043] Figures 26A to 26C A flow chart illustrating an example method of video processing is shown.

[0044] Figures 27A to 27E A flow chart illustrating an example method of video processing is shown. DETAILED DESCRIPTION

[0045] Embodiments of the disclosed technology can be applied to existing video codec standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used in this document to improve the readability of the specification and do not in any way limit the discussion and embodiments (and / or implementations) to only the corresponding sections.

[0046] 2. Video Codec Introduction

[0047] Due to the growing demand for higher resolution video, video coding methods and techniques are prevalent in contemporary technology. Video coding typically includes electronic circuits or software that compresses or decompresses digital video, and continues to improve to provide higher coding efficiency. Video coding converts uncompressed video into a compressed format or vice versa. There is a complex relationship between video quality, the amount of data (determined by the bit rate) used to represent the video, the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). The compression format usually conforms to standard video compression specifications, such as the High Performance Video Codec (HEVC) standard (also known as H.265 or MPEG-H Part 2), a general video codec standard to be formulated, or other current and / or future video codec standards.

[0048] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced the H.261 and H.263 standards, ISO / IEC produced the MPEG-1 and MPEG-4 Visual standards, and the two organizations jointly produced the H.262 / MPEG-2 Video standard, the H.264 / MPEG-4 Advanced Video Coding (AVC) standard, and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec structure that utilizes temporal prediction plus transform coding. In order to explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM) [3][4]. In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard with the goal of reducing the bit rate by 50% compared to HEVC.

[0049] 2.1 Typical video encoding and decoding process

[0050] Figure 1An example of an encoder block diagram of VCC is shown, which contains three loop filter blocks: Deblocking Filter (DF), Sample Adaptive Offset (SAO) and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original and reconstructed samples by adding the offset and applying a finite impulse response (FIR) filter, respectively, where the codec side information signals the offset and filter coefficients. ALF is positioned at the last processing stage of each picture and can be seen as a tool that tries to capture and repair artifacts created by previous stages.

[0051] 2.2 Intra-frame coding and decoding in VVC

[0052] 2.2.1 Intra-mode encoding and decoding using 67 intra-frame prediction modes

[0053] To capture arbitrary edge directions that occur in natural video, the number of directional intra modes is extended from 33 (as used in HEVC) to 65. Additional directional modes are provided in Figure 2 Depicted with dashed arrows, and planar and DC modes remain the same.These denser directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction.

[0054] The traditional angled intra prediction direction is defined from 45 degrees to -135 degrees in the counterclockwise direction, such as Figure 2 In VTM, several traditional angled intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original method and remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e., 67, and the intra mode codec remains unchanged.

[0055] In HEVC, each intra block has a square shape and the length of each side of the block is a power of 2. Therefore, no partition operation is required to generate intra predictors using DC mode. In VVV2, blocks can have a rectangular shape, which in general requires the use of per-block partitioning. To avoid partitioning for DC prediction, only the long side is used to calculate the average for non-square blocks.

[0056] In addition to the 67 intra prediction modes, wide angle intra prediction (WAIP) for non-square blocks and position-dependent intra prediction combination (PDPC) methods are enabled for certain blocks. PDPC is applied to the following intra modes without signaling: planar, DC, horizontal, vertical, bottom left angled mode and its 8 neighboring angled modes, and top right angled mode and its 8 neighboring angled modes.

[0057] 2.2.2 Affine Linear Weighted Intra Prediction (ALWIP or Matrix-based Intra Prediction)

[0058] JVET-N0217 proposed affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction (MIP)).

[0059] 2.2.2.1 Generate Reduced Prediction Signal by Matrix-Vector Multiplication

[0060] The neighborhood reference samples are first downsampled by averaging to generate a reduced reference signal bdry red Then, the reduced prediction signal pred red Computed by taking the matrix-vector product and adding a bias.

[0061] pred red =A.bdry red +b

[0062] Here, A is a red ·H red rows and 4 columns (4 columns if W=H=4 and 8 columns in all other cases). b is a matrix of size W red ·H red Vector.

[0063] 2.2.2.2 Description of the entire ALWIP process

[0064] Figure 3-6 The whole process of averaging, matrix-vector multiplication and linear interpolation is illustrated in different shapes. Note that the remaining shape is considered as one of the depicted cases.

[0065] 1. Given a 4×4 block, ALWIP takes two averages along each axis of the boundary. The resulting four input samples enter the matrix-vector multiplication. The matrix is ​​taken from the set S0. After adding the offset, this produces 16 final prediction samples. Linear interpolation is not necessary to generate the prediction signal. Therefore, the total number of multiplications performed per sample is (4·16) / (4·4)=4.

[0066] 2. Given an 8×8 block, ALWIP takes four averages along each axis of the border. The resulting eight input samples go into a matrix-vector multiplication. The matrix is ​​taken from set S1. This produces 16 samples at odd positions in the prediction block. Therefore, the total number of multiplications performed per sample is (8·16) / (8·8)=2. After adding the offset, these samples are interpolated vertically by using the reduced top border. Horizontal interpolation is then performed by using the original left border.

[0067] 3. Given an 8×4 block, ALWIP takes four averages along the horizontal axis of the boundary and four original boundary values ​​at the left boundary. The resulting eight input samples enter the matrix-vector multiplication. The matrix is ​​taken from set S1. This produces 16 samples at the odd horizontal positions and each vertical position of the prediction block. Therefore, the total number of multiplications performed per sample is (8·16) / (8·4)=4. After adding the offset, these samples are interpolated horizontally by using the original left boundary.

[0068] 4. Given a 16×16 block, ALWIP takes four averages along each axis of the border. The resulting eight input samples enter the matrix-vector multiplication. The matrix is ​​taken from set S2. This produces 64 samples at odd positions in the prediction block. Therefore, the total number of multiplications performed on each sample is (8·64) / (16·16)=2. After adding the offset, these samples are interpolated vertically by using the eight averages of the top border. Horizontal interpolation is then performed by using the original left border. The interpolation process does not add any multiplications in this case. Therefore, a total of two multiplications are required for each sample to calculate the ALWIP prediction.

[0069] For larger shapes, the process is essentially the same and it is easy to verify that the number of multiplications per sample point is less than four.

[0070] For W×8 blocks, where W>8, only horizontal interpolation is necessary because samples are given at odd horizontal positions and every vertical position.

[0071] Finally, for W×4 blocks, where W>8, let A_k be the matrix produced by leaving out each row corresponding to odd entries along the horizontal axis of the downsampled block. Thus, the output size is 32, and again only horizontal interpolation remains to be performed.

[0072] The transposed case is handled accordingly.

[0073] When constructing the MPM list for an intra prediction block, if the neighboring block is encoded and decoded in MIP mode, the MIP mode is mapped to the intra prediction mode by using the following table. Here, assuming that the width and height of the neighboring luminance block are widthNeig and heightNeig respectively, the MipSizeId is derived as follows:

[0074] • If widthNeig and heightNeig are both equal to 4, then MipSizeld is set equal to 0.

[0075] • Otherwise, if widthNeig and heightNeig are both less than or equal to 8, then MipSizeld is set equal to 1.

[0076] ●Otherwise, set MipSizeId equal to 2.

[0077] Table 8-4 – Specification of mapping between MIPs and intra prediction modes

[0078]

[0079]

[0080] 2.2.2.3 Syntax and Semantics

[0081] 7.3.6.5 Codec unit syntax

[0082]

[0083]

[0084]

[0085] 2.2.3 Multiple Reference Lines (MRL)

[0086] Multiple Reference Line (MRL) intra prediction uses more reference lines for intra prediction. Figure 7 In , an example of four reference lines is depicted, where the samples of segments A and F cannot be extracted from the reconstruction neighborhood samples but are filled with the closest samples to segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used.

[0087] The index of the selected reference row (mrl_idx) is signaled and used to generate the intra predictor. For reference row indices greater than 0, only additional reference row modes are included in the MPM list and only a single MPM index in the case of no remaining modes. The reference row index is signaled before the intra prediction mode, and planar and DC modes are excluded from the intra prediction mode in case a non-zero reference row index is signaled.

[0088] MRL is disabled for the first row of blocks within a CTU to prevent the use of extended reference samples outside the current CTU row. Likewise, PDPC is disabled when additional lines are used.

[0089] 2.2.4 Intra-frame sub-block segmentation (ISP)

[0090] In JVET-M0102, ISP is proposed, which divides the luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions according to the block size dimension, as shown in Table 1. Figure 8 and Fig. 9Two possible examples are shown. All sub-partitions fulfill the condition of having at least 16 samples. For block size, there can be 4xN or Nx4 (where N>8), and if allowed, there can also be 1xN or Nx1 sub-partitions.

[0091] Table 1: Number of sub-partitions depending on the block size (maximum transform size represented by maxTBSize)

[0092]

[0093] For each of these sub-partitions, a residual signal is generated by entropy decoding the coefficients sent by the encoder and then dequantizing and inverse transforming them. The sub-partition is then intra-predicted and the corresponding reconstructed samples are finally obtained by adding the residual signal to the prediction signal. Thus, the reconstructed values ​​of each sub-partition will be available to generate the prediction of the next sub-partition, which will repeat the process and so on. All sub-partitions share the same intra mode.

[0094] Table 2: Specification of trTypeHor and trTypeVer depending on predModeIntra

[0095]

[0096] 2.2.4.1 Syntax and Semantics

[0097] 7.3.7.5 Codec unit syntax

[0098]

[0099]

[0100]

[0101] intra_subpartitions_mode_flag[x0][y0] equal to 1 specifies that the current intra codec unit is partitioned into NumIntraSubPartitions[x0][y0] rectangular transform block sub-partitions.

[0102] intra_subpartitions_mode_flag[x0][y0] equal to 0 specifies that the current intra codec unit is not partitioned into rectangular transform block sub-partitions.

[0103] When intra_subpartitions_mode_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0104] intra_subpartitions_split_flag[x0][y0] specifies whether the intra subpartition partition type is horizontal or vertical.

[0105] When intra_subpartitions_split_flag[x0][y0] does not exist, it can be inferred as follows:

[0106] – If cbHeight is greater than MaxTbSizeY, infer

[0107] intra_subpartitions_split_flag[x0][y0] is equal to 0.

[0108] – Otherwise (cbWidth is greater than MaxTbSizeY), infer

[0109] intra_subpartitions_split_flag[x0][y0] is equal to 1.

[0110] The variable IntraSubPartitionsSplitType specifies the partition type used for the current luma codec block, as described in Table 7-9. IntraSubPartitionsSplitType is derived as follows:

[0111] – If intra_subpartitions_mode_flag[x0][y0] is equal to 0, then infer

[0112] IntraSubPartitionsSplitType is equal to 0.

[0113] – Otherwise, IntraSubPartitionsSplitType is set equal to

[0114] 1+intra_subpartitions_split_flag[x0][y0].

[0115] Table 7-9 – Namespaces associated with IntraSubPartitionsSplitType

[0116]

[0117] The variable NumIntraSubPartitions specifies the number of transform block subpartitions that the intra luma codec block is divided into. NumIntraSubPartitions is derived as follows:

[0118] – If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, then NumIntraSubPartitions is set equal to 1.

[0119] – Otherwise, NumIntraSubPartitions is set equal to 2 if one of the following conditions is true:

[0120] –cbWidth is equal to 4 and cbHeight is equal to 4,

[0121] –cbWidth is equal to 8 and cbHeight is equal to 4,

[0122] – Otherwise, NumIntraSubPartitions is set equal to 4.

[0123] 2.3 Chroma Intra Mode Codec

[0124] For chroma intra mode codec, a total of 8 or 5 intra modes are allowed for chroma intra mode codec, depending on whether the cross component linear model (CCLM) is enabled. These modes include five traditional intra modes and three cross component linear model modes. The chroma DM mode uses the corresponding luma intra prediction mode. Because the discrete block partitioning structure of luma and chroma components is enabled in the I stripe, a chroma block can correspond to multiple luma blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luma block covering the middle position of the current chroma block is directly inherited.

[0125] Table 8-2 – When sps_cclm_enabled_flag is equal to 0, it depends on

[0126] intra_chroma_pred_mode[xCb][yCb] and

[0127] IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2]

[0128] Specification of IntraPredModeC[xCb][yCb]

[0129]

[0130] Table 8-3 – When sps_cclm_enabled_flag is equal to 1, it depends on

[0131] intra_chroma_pred_mode[xCb][yCb] and

[0132] IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2]

[0133] Specification of IntraPredModeC[xCb][yCb]

[0134]

[0135]

[0136] 2.4 Transform Codec in VVC

[0137] 2.4.1 Multiple Transform Sets (MTS) in VVC

[0138] 2.4.1.1 Explicit Multiple Transform Sets (MTS)

[0139] In VTM4, transforms of large block sizes are enabled, up to 64×64 in size, which are mainly used for higher resolution video, such as 1080p and 4K sequences. For transform blocks where the dimensions (width or height, or both) are equal to 64, the high frequency transform coefficients are zeroed so that only low frequency coefficients are retained. For example, for an M×N transform block, where M is the block width and N is the block height, when M is equal to 64, only the left 32 columns of transform coefficients are retained. Similarly, when N is equal to 64, only the top 32 rows of transform coefficients are retained. When transform skip mode is used for large blocks, the entire block is used without zeroing any values.

[0140] In addition to the DCT-II already used in HEVC, a multiple transform selection (MTS) scheme is used to perform residual coding on both inter-frame and intra-frame coding blocks. This scheme uses multiple transforms selected from DCT8 / DST7. The newly introduced transformation matrices are DST-VII and DCT-VIII. Table 4 below shows the basis functions of the selected DST / DCT.

[0141] Table 4: Basis functions of the transformation matrix used in VVC

[0142]

[0143]

[0144] To maintain the orthogonality of the transform matrix, the transform matrix is ​​quantized more accurately than the transform matrix in HEVC. To keep the intermediate values ​​of the transform coefficients within the 16-bit range, all coefficients have 10 bits after horizontal and vertical transforms.

[0145] To control the MTS scheme, separate enable flags are specified at the SPS level for intra and inter frames respectively. When MTS is enabled with SPS, a CU-level flag is signaled to indicate whether MTS is applied. Here, MTS is applied only to luma. The MTS CU-level flag is signaled when the following conditions are met.

[0146] ○ Both width and height are less than or equal to 32

[0147] ○CBF sign equals one

[0148] If the MTS CU flag is equal to zero, DCT2 is applied in both directions. However, if the MTS CU flag is equal to one, the other two flags are additionally signaled to indicate the transform type in the horizontal and vertical directions, respectively. The transform and signaling mapping table is shown in Table 5. When the transform matrix accuracy is achieved, an 8-bit one-time transform core is used. Therefore, all transform cores used in HEVC remain the same, including 4-point DCT-2 and DST-7, 8-point, 16-point, and 32-point DCT-2. Similarly, other transform cores (including 64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7 and DCT-8) use an 8-bit one-time transform core.

[0149] Table 5: Mapping of decoded values ​​of tu_mts_idx to the corresponding transform matrices in the horizontal and vertical directions.

[0150]

[0151]

[0152] To reduce the complexity of large-size DST-7 and DCT-8, high-frequency transform coefficients are zeroed for DST-7 and DCT-8 blocks with size (width or height, or both) equal to 32. Only coefficients in the 16x16 low-frequency region are retained.

[0153] In addition to the case where different transforms are applied, VVC also supports a mode called Transform Skip (TS), which is similar to the concept of TS in HEVC. TS is considered a special case of MTS.

[0154] 2.4.2 Reduced Quadratic Transform (RST) proposed in JVET-N0193

[0155] 2.4.2.1 Non-separable quadratic transform (NSST) in JEM

[0156] In JEM, secondary transforms are applied between the forward primary transform and quantization (at the encoder) and between dequantization and the inverse primary transform (at the decoder side). Fig.10As shown, the execution of the 4x4 (or 8x8) secondary transform depends on the block size. For example, for each 8x8 block, the 4x4 secondary transform is applied to small blocks (i.e., min(width, height) < 8), and the 8x8 secondary transform is applied to large blocks (i.e., min(width, height) > 4).

[0157] The application of the non-separable transform is described below using the input as an example. To apply the non-separable transform, the 4x4 input block X

[0158]

[0159] is first represented as a vector

[0160]

[0161] The non-separable transform is calculated as where indicates the transform coefficient vector, and T is a 16x16 transform matrix. Using the scan order (horizontal, vertical, or diagonal) of the 4x4 block, the 16x1 coefficient vector is subsequently reorganized into this 4x4 block. Coefficients with smaller indices will be placed in the 4x4 coefficient block with smaller scan indices. There are a total of 35 transform sets, and each transform set uses 3 non-separable transform matrices (kernels). A mapping from the intra prediction mode to the transform set is predefined. For each transform set, the secondary transform index signaled explicitly further specifies the selected non-separable secondary transform (NSST) candidate. This index is signaled once in the bitstream for each intra CU after the transform coefficients.

[0162] 2.4.2 Reduced Secondary Transform (RST) in JVET-N0193

[0163] RST (also known as Low-Frequency Non-Separable Transform (LFNST)) was introduced in JVET-K0099 and a mapping of 4 transform sets (instead of 35 transform sets) was introduced in JVET-L0133. In this JVET-N0193, 16x64 (further reduced to 16x48) and 16x16 matrices are adopted. For ease of annotation, the 16x64 (reduced to 16x48) transform is denoted as RST 8x8 and the 16x16 transform is denoted as RST 4x4. Fig.11 An example of RST is shown.

[0164] 2.4.2.2.1 RST Calculation

[0165] The main idea of the reduced transform (RT) is to map an N-dimensional vector to an R-dimensional vector in a different space, where R / N (R < N) is the reduction factor.

[0166] The RT matrix is ​​an R×N matrix as follows:

[0167]

[0168] Among them, the R row transformation is the R basis of the N-dimensional space. The reverse transformation matrix of RT is the transpose of the forward transformation. Fig.12 The forward and reverse RTs are depicted in .

[0169] In this contribution, RST 8x8 with a reduction factor of 4 (1 / 4 size) is applied. Therefore, instead of 64x64 (which is the traditional 8x8 inseparable transform matrix size), a 16x64 direct matrix is ​​used. In other words, a 64×16 inverse RST matrix is ​​used at the decoder side to generate the core (one-time) transform coefficients in the 8×8 upper left region. The forward RST 8x8 uses a 16×64 (or 8x64 for 8x8 blocks) matrix so that the forward RST 8x8 only produces non-zero coefficients in the upper left 4×4 region within a given 8x8 region. In other words, if RST is applied, the 8×8 region except the upper left 4×4 region has only zero coefficients. For RST 4x4, a 16x16 (or 8x16 for 4x4 blocks) direct matrix multiplication is applied.

[0170] Reverse RST is conditionally applied when both of the following conditions are met:

[0171] ○Block size is greater than or equal to a given threshold (W>=4&&H>=4)

[0172] ○ Transform skip mode flag equals zero

[0173] If both the width (W) and height (H) of the transform coefficient block are greater than 4, RST 8x8 is applied to the upper left 8x8 region of the transform coefficient block. Otherwise, RST 4x4 is applied to the upper left min(8,W)×min(8,H) region of the transform coefficient block.

[0174] If the RST index is equal to 0, the RST is not applied. Otherwise, the RST is applied and the RST index is used to select the core of the RST. The RST selection method and the encoding and decoding of the RST index are explained later.

[0175] In addition, RST applies to intra CUs in both intra and inter slices, and to both luma and chroma. If dual tree is enabled, RST indexes for luma and chroma are signaled separately. For inter slices (dual tree disabled), a single RST index is signaled and used for both luma and chroma.

[0176] 2.4.2.2.2 Limitations of RST

[0177] When ISP mode is selected, RST is disabled and the RST index is not signaled, because even if RST is applied to each feasible partition, the performance improvement is negligible. In addition, disabling RST for the residual of ISP prediction can reduce the coding complexity.

[0178] 2.4.2.2.3 RST Selection

[0179] The RST matrix is ​​selected from four transform sets, each of which includes two transforms. The transform set to be applied is determined from the intra prediction mode as follows:

[0180] (1) If one of the three CCLM modes is indicated, transform set 0 is selected.

[0181] (2) Otherwise, perform transformation set selection according to the following table:

[0182] Transformation Set Selection Table

[0183]

[0184]

[0185] The index for accessing the above table (denoted as IntraPredMode) has a range of [-14, 83], which is a transform mode index for wide angle intra prediction.

[0186] Later, the low frequency non-separable transform (LFNST, also known as RST) set selection for the chroma block encoded and decoded in CCLM mode is modified based on the variable IntraPredMode_CCLM, where IntraPredMode_CCLM has a range of [-14,83]. IntraPredMode_CCLM is determined by the co-located luma intra prediction mode and the dimensions of the current chroma block.

[0187] When dual-tree is enabled, a block (eg, PU) covering the corresponding luma sample of the top left chroma sample in the current chroma block is defined as a co-located luma block. Examples are shown in Figures 24a and 24b, where the co-located position is denoted by TL.

[0188] 2.4.2.2.4 Reduced Dimension RST Matrix

[0189] As a further simplification, 16x48 matrices are applied instead of 16x64 with the same transform set configuration, each of the 16x48 matrices takes 48 input data from three 4x4 blocks in the upper left 8x8 block excluding the lower right 4x4 block (e.g. Fig.13 shown).

[0190] 2.4.2.2.5 RST signaling notification

[0191] Forward RST 8x8 uses a 16×48 matrix, so that forward RST 8x8 only produces non-zero coefficients in the upper left 4×4 area of ​​the first three 4x4 areas. In other words, if RST 8x8 is applied, only the upper left 4×4 area (due to RST 8x8) and the lower right 4×4 area (due to one transform) can have non-zero coefficients. Therefore, when the upper right 4×4 block area and the lower left 4×4 block area ( Fig.14 The RST index is not encoded when any non-zero element is detected in the WRITE_ON region (shown in the figure and referred to as the "zeroed" region), because this implies that RST is not applied. In such a case, the RST index is inferred to be zero.

[0192] 2.4.2.2.6 Zeroing Area within a CG

[0193] Typically, any coefficient in a 4x4 subblock may be non-zero before inverse RST is applied to the 4x4 subblock. However, in some cases it is constrained that some coefficients in a 4x4 subblock must be zero before inverse RST is applied to the subblock.

[0194] Let nonZeroSize be a variable. What is required is that when any coefficient is rearranged into a 1D array, the index of any coefficient is not less than nonZeroSize, and then the reverse RST must be zero.

[0195] When nonZeroSize is equal to 16, there is no zero constraint on the coefficients in the upper left 4×4 sub-block.

[0196] In JVET-N0193, when the current block size is 4×4 or 8×8, nonZeroSize is set equal to 8 (i.e., Fig.14 As shown, coefficients with scan indices in the range [8,15] should be 0). For other block dimensions, nonZeroSize is set equal to 16.

[0197] 2.4.2.2.7 Description of RST in the working draft

[0198] 7.3.2.3 Sequence parameter set RBSP syntax

[0199]

[0200] 7.3.7.11 Residual Codec Syntax

[0201]

[0202]

[0203] 7.3.7.5 Codec unit syntax

[0204]

[0205]

[0206] sps_st_enabled_flag equal to 1 specifies that st_idx may be present in the residual codec syntax of an intra codec unit. sps_st_enabled_flag equal to 0 specifies that st_idx is not present in the residual codec syntax of an intra codec unit.

[0207] st_idx[x0][y0] specifies which secondary transform kernel is applied between the two candidate kernels in the selected transform set. st_idx[x0][y0] equal to 0 specifies that no secondary transform is applied. Array indices x0, y0 specify the position (x0, y0) of the upper left sample of the transform block under consideration relative to the upper left sample of the picture.

[0208] When st_idx[x0][y0] does not exist, st_idx[x0][y0] is inferred to be equal to 0.

[0209] Note that whether to send st_idx depends on the number of non-zero coefficients in all TUs within the CU (e.g., for a single tree, the number of non-zero coefficients in 3 blocks (i.e., Y, Cb, Cr); for a dual tree and for luma coding, the number of non-zero coefficients in the luma block; for a dual tree and for chroma coding, the number of non-zero coefficients in the two chroma blocks). Additionally, the threshold depends on the partitioning structure, (treeType == SINGLE_TREE? 2:1).

[0210] The binary number of st_idx is context-coded. More specifically, the following applies:

[0211] Table 9-9 – Syntax elements and their associated binarization

[0212]

[0213] Table 9-15 – Assignment of ctxInc to syntax elements with binary context codec

[0214]

[0215] 9.5.4.2.8 Derivation of ctxInc for syntax element st_idx

[0216] The inputs to this process are: color component index cIdx; luma or chroma position (x0, y0), depending on whether cIdx specifies the top left sample of the current luma or chroma codec block relative to the top left sample of the current picture; tree type treeType; luma intra prediction mode as specified in Section 8.4.2

[0217] IntraPredModeY[x0][y0]; the syntax element intra_chroma_pred_mode[x0][y0] that specifies the intra prediction mode for the chroma samples as specified in section 7.4.7.5; and a plurality of transform selection indices tu_mts_idx[x0][y0].

[0218] The output of this process is the variable ctxInc.

[0219] The variable intraModeCtx is exported as follows:

[0220] If cIdx is equal to 0, intraModeCtx is derived as follows:

[0221] intraModeCtx=(IntraPredModeY[x0][y0]<=1)? 1:0

[0222] Otherwise (cIdx is greater than 0), intraModeCtx is derived as follows:

[0223] intraModeCtx=(intra_chroma_pred_mode[x0][y0]>=4)? 1:0

[0224] The variable mtsCtx is exported as follows:

[0225] mtsCtx=(tu_mts_idx[x0][y0]==0&&treeType!=SINGLE_TREE)? 1:0

[0226] The variable ctxInc is exported as follows:

[0227] ctxInc=(binIdx<<1)+intraModeCtx+(mtsCtx<<2)

[0228] 2.4.2.2.8 Summary of RST usage

[0229] RST can be enabled only when the number of non-zero coefficients in a block is greater than 2 and 1 for single tree and separable tree, respectively. In addition, when RST is enabled, the positions of non-zero coefficients of the codec group (CG) to which RST is applied need to be restricted as follows.

[0230] Table 1: Use of RST

[0231]

[0232]

[0233] 2.4.3 Sub-block Transformation

[0234] For an inter-predicted CU with cu_cbf equal to 1, cu_sbt_flag may be signaled to indicate whether to decode the entire residual block or a sub-portion of the residual block. In the former case, the inter MTS information is further parsed to determine the transform type of the CU. In the latter case, a portion of the residual block is encoded and decoded with the inferred adaptive transform and the rest of the residual block is zeroed. SBT does not apply to combined inter-intra modes.

[0235] In the sub-block transform, a position-dependent transform is applied to the luma transform blocks in SBT-V and SBT-H (chroma TBs always use DCT-2). The two positions of SBT-V and SBT-H are associated with different core transforms. More specifically, Figure 3 The horizontal and vertical transforms are specified for each SBT position. For example, the horizontal and vertical transforms for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is greater than 32, the corresponding transform is set to DCT-2. Therefore, the sub-block transform jointly specifies the TU tiling, cbf, and horizontal and vertical transforms of the residual block, which can be considered a syntactic shortcut for the case where the main residual of the block is at one side of the block.

[0236] 2.4.3.1 Syntactic Elements

[0237] 7.3.7.5 Codec unit syntax

[0238]

[0239]

[0240]

[0241] cu_sbt_flag equal to 1 specifies that sub-block transform is used for the current codec unit. cu_sbt_flag equal to 0 specifies that sub-block transform is not used for the current codec unit.

[0242] When cu_sbt_flag is not present, its value is inferred to be equal to 0.

[0243] Note—: When sub-block transform is used, the codec unit is divided into two transform units; one transform unit has residual data and the other transform unit does not have residual data.

[0244] cu_sbt_quad_flag equal to 1 specifies that for the current codec unit, the sub-block transform includes a transform unit of 1 / 4 the size of the current codec unit. cu_sbt_quad_flag equal to 0 specifies that for the current codec unit, the sub-block transform includes a transform unit of 1 / 2 the size of the current codec unit.

[0245] When cu_sbt_quad_flag is not present, its value is inferred to be equal to 0.

[0246] cu_sbt_horizontal_flag equal to 1 specifies that the current codec unit is split horizontally into 2 transform units. cu_sbt_horizontal_flag[x0][y0] equal to 0 specifies that the current codec unit is split vertically into 2 transform units.

[0247] When cu_sbt_horizontal_flag is not present, its value is derived as follows:

[0248] – If cu_sbt_quad_flag is equal to 0, then cu_sbt_horizontal_flag is set equal to allowSbtHorQ.

[0249] – Otherwise (cu_sbt_quad_flag is equal to 0), then cu_sbt_horizontal_flag is set equal to allowSbtHorH.

[0250] cu_sbt_pos_flag equal to 1 specifies that tu_cbf_luma, tu_cbf_cb, and tu_cbf_cr of the first transform unit in the current codec unit are not present in the bitstream. cu_sbt_pos_flag equal to 0 specifies that tu_cbf_luma, tu_cbf_cb, and tu_cbf_cr of the second transform unit in the current codec unit are not present in the bitstream.

[0251] The variable SbtNumFourthsTb0 is derived as follows:

[0252] sbtMinNumFourths=cu_sbt_quad_flag? 1:2 (7-117)

[0253] SbtNumFourthsTb0=cu_sbt_pos_flag? (4-sbtMinNumFourths):2(7-118)

[0254] sps_sbt_max_size_64_flag equal to 0 specifies that the maximum CU width and height for allowing sub-block transforms is 32 luma samples. sps_sbt_max_size_64_flag equal to 1 specifies that the maximum CU width and height for allowing sub-block transforms is 64 luma samples.

[0255] MaxSbtSize=sps_sbt_max_size_64_flag? 64:32 (7-33)

[0256] 2.4.4 Quantized Residual Domain Block Differential Pulse Codec Modulation Codec (QR-BDPCM)

[0257] In JVET-N0413, quantized residual domain BDPCM (hereinafter referred to as RBDPCM) is proposed. Intra-frame prediction is performed on the entire block by replicating samples in the prediction direction (horizontal or vertical prediction) similar to intra-frame prediction. The residual is quantized, and the delta between the quantized residual and the quantized value of its predictor (horizontal or vertical) is encoded and decoded.

[0258] For a block of size M (rows) × N (columns), let r i,j ,0≤i≤M-1,0≤j≤N-1 is the prediction residual after performing intra prediction horizontally (copying the left neighbor pixel values ​​row by row on the prediction block) or vertically (copying the top neighbor row to every row in the prediction block) using unfiltered samples from the upper or left block boundary samples. Let Q(r i,j ), 0≤i≤M-1,0≤j≤N-1 represents the residual r i,j The quantized version of , where the residual is the difference between the original block and the predicted block value. Then, block DPCM is applied to the quantized residual samples, resulting in a value with elements The modified M×N array When vertical BDPCM is signaled:

[0259]

[0260] For horizontal prediction, similar rules apply and the residual quantization samples are obtained as

[0261]

[0262] Residual quantization samples Send to the codec.

[0263] On the codec side, the above calculation is performed in reverse to produce Q(r i,j ),0≤i≤M-1,0≤j≤N-1. For vertical coding and decoding,

[0264]

[0265] For the horizontal case,

[0266]

[0267] The inverse quantized residual Q -1 (Q(r i,j )) is added to the intra block prediction value to produce the reconstructed sample value.

[0268] When QR-BDPCM is selected, no transformation is applied.

[0269] 2.5 coefficient entropy encoding and decoding

[0270] 2.5.1 Coefficient encoding and decoding of blocks to which transforms are applied

[0271] In HEVC, the transform coefficients of a codec block are encoded and decoded using non-overlapping coefficient groups (or sub-blocks), and each CG contains the coefficients of a 4x4 block of the codec block. The CGs within the codec block and the transform coefficients within the CGs are encoded and decoded according to a predefined scanning order.

[0272] The CG within the codec block and the transform coefficients within the CG are encoded and decoded according to a predefined scanning order. Both the CG and the coefficients within the CG follow a diagonal upward right scanning order. Fig.16 and Fig.17 Depicted are examples of 4x4 block and 8x8 scanning order, respectively.

[0273] Note that the encoding and decoding order is the reverse scanning order (i.e. from Fig.17 CG3 to CG0 decoding), when decoding a block, the coordinates of the last non-zero coefficient are decoded first.

[0274] The encoding and decoding of the transform coefficient levels of a CG having at least one non-zero transform coefficient can be separated into multiple scan paths. In the first path, the first binary number (represented by bin0, also called significant_coeff_flag, indicating that the magnitude of the coefficient is greater than 0) is encoded and decoded. Next, two scan paths for context encoding and decoding the second / third binary numbers (represented by bin1 and bin2, also called coeff_abs_greater1_flag and coeff_abs_greater2_flag, respectively) can be applied. Finally, if necessary, two more scan paths are called for encoding and decoding sign information and residual values ​​of the coefficient levels (also called coeff_abs_level_remaining). Note that only the bins in the first three scan paths are encoded and decoded in normal mode, and these bins are referred to as normal bins in the following description.

[0275] In VVC3, for each CG, the normal codec binary numbers and the bypass codec binary numbers are separated in the codec order, first transmitting all the normal codec binary numbers of the sub-block, followed by the bypass codec binary numbers. The transform coefficient levels of the sub-block are coded and decoded in five ways at the scan position, as follows:

[0276] ○ Path 1: Process the encoding of significance (sig_flag), greater than 1 flag (gt1_flag), parity (par_level_flag), and greater than 2 flag (gt2_flag) in codec order. If sig_flag is equal to 1, then first encode and decode gt1_flag (which specifies whether the absolute level is greater than 1). If gt1_flag is equal to 1, then additionally encode and decode par_flag (which specifies the parity of the absolute level minus 2).

[0277] o Path 2: Process the encoding and decoding of the remaining absolute level (remaining part) for all scan positions when gt2_flag is equal to 1 or gt1_flag is equal to 1. Non-binary syntax elements are binarized with Golomb-Rice code and the resulting binary numbers are encoded and decoded in bypass mode of the arithmetic codec engine.

[0278] o Pass 3: Use Golomb-Rice code in bypass mode of the arithmetic codec engine to fully encode the absolute level (absLevel) of the coefficient for which sig_flag was not encoded in the first pass (due to reaching the binary limit of the conventional codec).

[0279] ○ Path 4: When sig_coeff_flag is equal to 1, encode and decode the sign (sign_flag) for all scan positions

[0280] It is guaranteed that no more than 32 conventionally coded bins (sig_flag, par_flag, gt1_flag and gt2_flag) are encoded or decoded for a 4x4 subblock. For a 2x2 chroma subblock, the number of conventionally coded bins is limited to 8.

[0281] The Rice parameter (ricePar) for encoding the rest of the non-binary syntax elements (in path 3) is derived similarly to HEVC. At the beginning of each sub-block, ricePar is set equal to 0. After encoding the rest of the syntax elements, the Rice parameter is modified according to the predefined equation. To encode the non-binary syntax element absLevel (in path 4), the sum of the absolute values ​​in the local template sumAbs is determined. The variables ricePar and posZero are determined by a lookup table based on the associated quantization and sumAbs. The intermediate variable codeValue is derived as follows:

[0282] If absLevel[k] is equal to 0, codeValue is set equal to posZero;

[0283] If absLevel[k] is less than or equal to posZero, codeValue is set equal to absLevel[k]-1;

[0284] Otherwise (absLevel[k] is greater than posZero), codeValue is set equal to absLevel[k].

[0285] The value of codeValue is encoded or decoded using the Columbus-Rice code with the Rice parameters ricePar.

[0286] 2.5.1.1 Context Modeling for Coefficient Encoding and Decoding

[0287] The syntax element selection probability model related to the absolute value of the transform coefficient level depends on the value of the absolute level or the partially reconstructed absolute level in the local neighborhood. Fig.18 The template used is illustrated in FIG.

[0288] The chosen probability model depends on the sum of the absolute levels (or partially reconstructed absolute levels) in the local neighborhood and the number of absolute levels greater than 0 in the local neighborhood (given by the number of sig_coeff_flags equal to 1). Context modeling and binarization depend on the following measures of the local neighborhood:

[0289] ○numSig: the number of non-zero levels in the local neighborhood;

[0290] ○sumAbs1: the sum of the absolute levels of the partial reconstructions after the first pass in the local neighborhood (absLevel1);

[0291] ○sumAbs: the sum of the absolute levels of reconstructions in the local neighborhood;

[0292] o Diagonal position (d): The sum of the horizontal and vertical coordinates of the current scan position within the transform block.

[0293] Based on the values ​​of numSig, sumAbs1, and d, the probability model for encoding and decoding sig_flag, par_flag, gt1_flag, and gt2_flag is selected. Based on the values ​​of sumAbs and numSig, the Rice parameters for binarizing abs_remainder are selected.

[0294] 2.5.1.2 Dependent Quantization (DQ)

[0295] In addition, the same HEVC scalar quantization is used with a new concept called correlated scalar quantization. Correlated scalar quantization refers to a method in which the set of allowable reconstruction values ​​of a transform coefficient depends on the value of the transform coefficient level before the current transform coefficient level in the reconstruction order. The main effect of this method is that the allowable reconstruction vectors are more densely grouped in the N-dimensional vector space (N represents the number of transform coefficients in the transform block) compared to the conventional unrelated scalar quantization used in HEVC. This means that for a given average number of allowable reconstruction vectors per N-dimensional unit volume, the average distortion between the input vector and the closest reconstruction vector is reduced. The method of correlated scalar quantization is implemented by: (a) defining two scalar quantizers with different reconstruction levels, and (b) defining a process of switching between the two scalar quantizers.

[0296] Fig.19 The two used scalar quantizers, denoted by Q0 and Q1, are illustrated. The position of the available reconstruction levels is uniquely specified by the quantization step size Δ. The used scalar quantizer (Q0 or Q1) is not explicitly signaled in the bitstream. Instead, the quantizer used for the current transform coefficient is determined by the parity of the transform coefficient level that precedes the current transform coefficient in the encoding / reconstruction order.

[0297] like Fig. 20 As shown in FIG. 1 , the switching between the two scalar quantizers (Q0 and Q1) is realized via a state machine with four states. The state can take four different values: 0, 1, 2, 3. This is uniquely determined by the parity of the transform coefficient level before the current transform coefficient in the encoding / reconstruction order. At the beginning of the inverse quantization of the transform block, the state is set equal to 0. The transform coefficients are reconstructed in the scanning order (i.e., in the same order in which they were entropy decoded). After the reconstruction of the current transform coefficient, the state is as follows: Fig. 20 The update is shown, where k represents the value of the transform coefficient level.

[0298] 2.5.1.3 Syntax and Semantics

[0299] 7.3.7.11 Residual Codec Syntax

[0300]

[0301]

[0302]

[0303]

[0304]

[0305]

[0306]

[0307]

[0308] 2.5.2 Coefficient encoding and decoding of TS codec block and QR-BDPCM codec block

[0309] QR-BDPCM follows the context modeling approach for TS codec blocks.

[0310] The modified transform coefficient levels are used for TS residual coding. Relative to the conventional residual coding case, the TS residual coding includes the following changes:

[0311] (1) No signaling of the last x / y position

[0312] (2) coded_sub_block_flag for each sub-block except the last one, when all the above flags are equal to 0;

[0313] (3) sig_coeff_flag using reduced template context modeling,

[0314] (4) Single context modeling for abs_level_gt1_flag and par_level_flag;

[0315] (5) Context modeling of symbolic tokens, additionally greater than 5, 7, or 9 tokens,

[0316] (6) Derivation of modified Rice parameters for binarization of the remaining part;

[0317] (7) The number of binary numbers for context encoding and decoding of each sample point is limited to 2 binary numbers for each sample point in a block.

[0318] 2.5.2.1 Syntax and Semantics

[0319] 7.3.6.10 Transformation unit syntax

[0320]

[0321]

[0322]

[0323]

[0324]

[0325] For each CG, the number of bins for context encoding and decoding is limited to no more than 2 bins per sample point.

[0326] Table 9-15 – Assignment of ctxInc to syntax elements using binary numbers of context codecs

[0327]

[0328] 2.5 Chroma Direct Mode

[0329] In direct mode (DM), the prediction mode of the co-located luma block is used to derive the chroma intra prediction mode.

[0330] First, export the intra prediction mode lumaIntraPredMode:

[0331] ●If the co-located luma block is encoded or decoded in MIP mode, lumaIntraPredMode is set equal to planar mode.

[0332] ●Otherwise, if the co-located luma block is encoded or decoded in IBC mode or palette mode, lumaIntraPredMode is set equal to DC mode.

[0333] ● Otherwise, lumaIntraPredMode is set equal to the intra prediction mode of the co-located luma block covering the corresponding luma sample in the middle of the chroma block. Fig.25 An example is depicted in .

[0334] Second, the intra chroma prediction mode (denoted as IntraPredModeC) is derived from lumaIntraPredMode, as highlighted in bold and italics in the following table. Note that intra_chroma_pred_mode equal to 4 refers to DM mode.

[0335] Table 8-2 – Specification of IntraPredModeC[xCb][yCb] depending on cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode

[0336]

[0337] Finally, if the color format of the picture is 4:2:2, IntraPredModeC is further modified for DM mode according to the following table.

[0338] Specification of the 4:2:2 mapping process from chroma intra prediction mode X to mode Y when chroma_format_idc is equal to 2

[0339]

[0340]

[0341] The detailed draft is specified as follows.

[0342] 8.4.3 Derivation process of chroma intra prediction mode

[0343] The inputs to this process are:

[0344] –Specifies the luminance position (xCb, yCb) of the upper left sample of the current chroma codec block relative to the upper left luminance sample of the current picture,

[0345] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0346] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0347] In this process, the chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived.

[0348] The corresponding luma intra prediction mode lumaIntraPredMode is derived as follows:

[0349] – If intra_mip_flag[xCb][yCb] is equal to 1, lumaIntraPredMode is set equal to INTRA_PLANAR.

[0350] – Otherwise, if CuPredMode[0][xCb][yCb] is equal to MODE_IBC or MODE_PLT, lumaIntraPredMode is set equal to INTRA_DC.

[0351] – Otherwise, lumaIntraPredMode is set equal to

[0352] IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2].

[0353] The chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived using cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode as specified in Table 8-2.

[0354] Table 8-2 – Specification of IntraPredModeC[xCb][yCb] depending on cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode

[0355]

[0356]

[0357] When chroma_format_idc is equal to 2, chroma intra prediction mode Y is derived using chroma intra prediction mode X in Table 8-2, as specified in Table 8-3, and then chroma intra prediction mode X is set equal to chroma intra prediction mode Y.

[0358] Table 8-3 – Specification of the 4:2:2 mapping process from chroma intra prediction mode X to mode Y when chroma_format_idc is equal to 2

[0359]

[0360] 3 Disadvantages of existing implementations

[0361] The current design has the following issues:

[0362] (1) The four predefined transform sets for chrominance components are the same as the four predefined transform sets for luma components. Additionally, luma and chroma blocks with the same intra prediction mode use the same transform set. However, chroma signals are typically smoother than luma components. Using the same set may be suboptimal.

[0363] (2) RST is only applied to certain CGs instead of all CGs. However, the decision to signal the RST index depends on the number of non-zero coefficients in the entire block. When all coefficients in the CG to which RST is applied are zero, there is no need to signal the RST index. However, the current design may still signal the index, which wastes unnecessary bits.

[0364] (3) The RST index is signaled after residual coding and decoding, because this requires recording the number of non-zero coefficients and whether there are non-zero coefficients in a certain position (for example, numZeroOutSigCoeff, numSigCoeff in Section 2.3.2.2.7). This design makes the parsing process more complicated.

[0365] (4) RST index is context-coded, and context modeling depends on the coded luma / chroma intra prediction mode and MTS index. Such a design introduces parsing delay in the reconstruction of intra prediction mode. And it introduces 8 contexts, which may be a burden for hardware implementation.

[0366] (a) DM and CCLM share the same context index offset, which does not make sense since they are two different chroma intra prediction modes.

[0367] (5) The current design of non-TS residual codec first encodes the coefficient information, followed by the index of RST (i.e., whether to use RST, and if so, which matrix to choose). In such a design, the RST on / off information is not considered in the entropy codec of the residual.

[0368] (6) RST is always applied to the upper left region of the transform block with the primary transform applied. However, for different primary transform bases, the energy is not always concentrated in the upper left region of the transform block.

[0369] (7) For the dual-tree and single-tree coding structures, different methods are used to guide the determination of whether to signal RST-related information.

[0370] (8) When there is more than one TU in a CU (e.g., the CU size is 128×128), whether to parse RST-related information can only be determined after decoding all TUs. For example, for a 128x128 CU, the first PB cannot be processed without waiting for the LFNST index after the last PB. Although this does not necessarily interrupt the entire 64x64-based decoder pipeline (if CABAC can be decoupled), the data buffering is increased by 4 times for a certain number of decoder pipeline stages. This is costly.

[0371] (9) In JVET-00219, the intra prediction mode of the co-located luma block is used to determine the LFNST transform set for the chroma block encoded in CCLM mode. However, the co-located luma block may be encoded in a non-intra prediction mode (i.e., not a traditional intra prediction method, such as using DC / planar / angled prediction direction). For example, in a dual-tree case, the co-located luma block may be encoded in a palette mode, an IBC mode, etc. In this case, the intra prediction mode of the co-located luma block is not defined. Therefore, it is unknown how to derive the secondary transform set for the chroma block.

[0372] (10) The derivation of the chroma intra prediction block defined in subclause 8.4.4 checks whether the luma block corresponding to the top left luma sample is encoded or decoded in MIP / IBC / palette mode. If true, the default mode is given. Otherwise, the intra prediction mode of the middle luma sample is used. This raises two questions:

[0373] a. When the codec block covering the upper left luma sample (e.g. TL in Figures 24(a) and 24(b), the luma sample corresponding to the upper left chroma sample in the current chroma block) is coded in MIP mode, and the codec block covering the middle luma sample (e.g. Fig.25 When the coded block with CR in the frame is coded in the normal intra mode, in this case, the default mode is used and is set to the chroma intra prediction mode that breaks the association between the current chroma block and the luminance block covering the CR. This will result in lower coding performance.

[0374] b. When the codec block covering the upper left luma sample (e.g. TL in Figures 24(a) and 24(b), the luma sample corresponding to the upper left chroma sample in the current chroma block) is coded in intra mode, and the codec block covering the middle luma sample (e.g. Fig.25 When the codec block covering the CR is coded in IBC / palette / MIP mode, in this case, the intra prediction mode from the codec block covering the CR is used to derive the chroma DM mode. However, no such intra prediction mode associated with MIP / IBC / palette mode is defined.

[0375] 4 Example Method for Context Modeling for Residual Coding and Decoding

[0376] Embodiments of the technology disclosed herein overcome the shortcomings of existing implementations, thereby providing video codecs with higher codec efficiency. The method for context modeling of residual codecs based on the disclosed technology can improve both existing and future video codec standards, and is illustrated below as examples described in various implementations. The examples of the disclosed technology provided below explain the overall concept and are not meant to be interpreted as limitations. In the examples, the various features described in these examples can be combined unless clearly indicated to the contrary.

[0377] In the following description, a "block" may refer to a codec unit (CU) or a transform unit (TU) or any rectangular area of ​​video data. A "current block" may refer to a codec unit (CU) currently being decoded / encoded or a transform unit (TU) currently being decoded / encoded or a codec rectangular area of ​​video data being decoded / encoded. A "CU" or a "TU" may also be referred to as a "codec block" and a "transform block".

[0378] In these examples, RST can be any variation designed in JVET-N0193. RST can be any technique that can apply a secondary transform to a block or apply a transform to a transform skip (TS) coded block (e.g., RST proposed in JVET-N0193 is applied to a TS coded block).

[0379] In the following, "normal intra prediction mode" is used to refer to a conventional intra prediction method, in which a prediction signal is generated by extrapolating neighboring pixels from a certain direction, such as DC mode, planar mode, and angled intra prediction mode (e.g., wide angle intra prediction mode may also be included). For blocks that are not encoded and decoded using the normal intra prediction mode, the blocks may be encoded and decoded using at least one of the following encoding and decoding methods, such as IBC, MIP, palette, BDPCM intra prediction mode.

[0380] Additionally, "Zeroed Regions" or "Zeroed CGs" may indicate that these regions / CGs always have zero coefficients due to the reduced transform size used during the secondary transform process. For example, if the secondary transform size is 16x32 and the CG size is 4x4, it will be applied to the first two CGs, but only the first CG may have non-zero coefficients, and the second 4x4 CG is also referred to as the Zeroed CG.

[0381] Select the transformation matrix in RST

[0382] 1. A sub-region to which RST is applied may be a sub-region that is not the upper left portion of a block.

[0383] a. In one example, RST can be applied to the top right or bottom right or bottom left or middle sub-region of the block.

[0384] b. To which sub-region RST is applied may depend on the intra prediction mode and / or the primary transform matrix (eg, DCT-II, DST-VII, identity transform).

[0385] 2. The selection of the transform set and / or transform matrix used in the RST may depend on the color component.

[0386] a. In one example, one set of transform matrices may be used for luma (or G) components and one set for chroma components (or B / R).

[0387] b. In one example, each color component may correspond to one set.

[0388] c. In one example, at least one matrix is ​​different in any of the two or more sets for different color components.

[0389] 3. The selection of the transform set and / or transform matrix used in the RST may depend on the intra prediction method (eg, CCLM, multiple reference row based intra prediction method, matrix based intra prediction method).

[0390] a. In one example, one set of transform matrices may be used for CCLM codec blocks and another set may be used for non-CCLM codec blocks.

[0391] b. In one example, one set of transform matrices may be used for normal intra prediction codec blocks, and another set may be used for blocks with multiple reference lines enabled (ie, which do not use neighboring lines for intra prediction).

[0392] c. In one example, one set of transform matrices may be used for blocks decoded with joint chroma residual coding, and another set may be used for blocks to which joint chroma residual coding is not applied.

[0393] d. In one example, for different intra prediction methods, at least one matrix is ​​different in any of the two or more sets.

[0394] e. Alternatively, RST may be disabled for blocks coded with certain intra prediction directions and / or certain codec tools (eg CCLM, and / or joint chroma residual codec, and / or a certain color component (eg chroma)).

[0395] 4. A default intra prediction mode can be assigned to frames that are not in normal intra prediction mode (such as MIP,

[0396] IBC, palette) encoded and decoded blocks (e.g., CU / PU / CB / PB).

[0397] a. The default mode may depend on the codec method (MIP / IBC / Palette).

[0398] b. The default mode can be promptly signaled or exported.

[0399] c. The default mode can be used to export the colorimetric export mode (DM).

[0400] d. The default mode can be used to predict the intra prediction mode of other blocks. For example, the default mode can be used to derive the most probable mode (MPM) list for encoding and decoding subsequent blocks of the current block.

[0401] e. A default mode assigned to a block of one color component (eg, luma) may be used to derive a transform set or transform index for another color component (eg, chroma).

[0402] f. Alternatively, in addition, a default mode may be stored together with the prediction mode (eg, intra / inter / IBC).

[0403] g. The default mode cannot be assigned to the inter-frame codec block.

[0404] 5. It is proposed to use the information of one luma codec block (which may also be referred to as the corresponding luma block) covering the same predefined position of the chroma codec block in all operations of the chroma intra prediction mode derivation process. For example, the same luma codec block covering the same predefined position of the chroma codec block is used to check the prediction mode (or coding method, such as IBC / MIP / palette) of the luma codec block and to extract the intra prediction mode of the luma codec block.

[0405] a. In one example, the same predefined position is defined as the corresponding luma sample of the middle chroma sample of the current chroma block (e.g., Fig.25 CR) is associated with.

[0406] b. In one example, the same predefined position is defined as being associated with the corresponding luma sample of the top left chroma sample of the current chroma block (eg, TL of FIG. 24b ).

[0407] c. Alternatively, further, if the codec block covering the same predefined position is coded in IBC / palette / MIP mode / any other non-normal intra prediction mode, the default intra prediction mode can be used to derive the chroma intra prediction mode. Otherwise, the decoded intra prediction mode of the codec block covering the same predefined position can be used to derive the chroma intra prediction mode.

[0408] 6. For chroma blocks encoded and decoded in a non-normal intra-frame prediction mode (such as CCLM), when the corresponding luminance block is encoded and decoded in a non-normal intra-frame prediction mode (such as IBC / palette / MIP), the transform set / transform matrix or other encoding and decoding tools used in RST can be derived from one or more default modes or default transform sets.

[0409] a. Whether to use a default transform set / transform matrix or derive the transform set / transform matrix from the intra-frame luma prediction mode of the corresponding luma block may depend on the coding mode of the corresponding luma block.

[0410] i. In one example, if the corresponding luminance block is decoded in normal intra-frame prediction mode and / or BDPCM, a transform set / transform matrix can be derived based on the luminance intra-frame prediction mode associated with the corresponding luminance block (e.g., a co-located luminance block or a luminance block defined in Item 5 of the present invention).

[0411] ii. In one example, if the corresponding luma block is encoded in a non-normal intra prediction mode (eg, IBC / palette and / or BDPCM), a default transform set (eg, where the set index is equal to K (eg, K=0)) may be used.

[0412] b. Whether to enable RST for a block of one color component may depend on the coding method of one or more corresponding blocks in another color component and / or the coding method of the current block.

[0413] i. Whether to enable RST for a chroma block may depend on the codec method of one or more corresponding luminance blocks and / or the codec method of the current chroma block.

[0414] ii. In one example, RST may be enabled for a chroma block if the corresponding luma block is decoded in normal intra prediction mode and / or BDPCM codec.

[0415] iii. In one example, RST may be disabled for a chroma block if the corresponding luma block is encoded in a non-normal intra prediction mode (eg, IBC / palette and / or BDPCM).

[0416] 1) Alternatively, RST may be disabled for chroma blocks coded in a non-normal intra prediction mode, if, in addition, the corresponding luma block is coded in a non-normal intra prediction mode (eg, IBC / palette and / or BDPCM).

[0417] c. When the co-located luminance block is not encoded in normal intra prediction mode, a predefined intra prediction mode can be assigned, which is then used to select the transform set and / or transform matrix used in the RST (e.g., according to the transform set selection table shown in Section 2.4.2.2.3).

[0418] i. In one example, if the co-located luma block is encoded in IBC (Intra Block Copy) mode, a first predefined intra prediction mode (eg DC or planar mode) may be assigned.

[0419] ii. In one example, if the co-located luma block is encoded in palette mode, a second predefined intra prediction mode (eg, DC or planar mode) may be assigned.

[0420] iii. In one example, if the co-located luma block is coded in MIP mode, a third predefined intra prediction mode (eg, planar or DC mode) may be assigned.

[0421] iv. In one example, if the co-located luma block is coded in BDPCM (Block Delta Pulse Codec Modulation) mode, a fourth predefined intra prediction mode (eg planar or DC mode) may be assigned.

[0422] v. In one example, for each of the predefined intra prediction modes, it can be a DC mode or a planar mode or a vertical mode or a horizontal mode or a 45 degree mode or a 135 degree mode.

[0423] vi. Alternatively, if the co-located luminance block is encoded in MIP mode, the MIP mode can be mapped to the intra-frame prediction mode according to the MIP mode and the dimensions of the co-located luminance block (for example, by using the table "Specification for mapping between MIP and intra-frame prediction modes" in section 2.2.2.2).

[0424] vii. A predefined mode may be adaptively selected from several candidates such as DC mode and planar mode.

[0425] 1) A predefined mode can be signaled from the encoder to the decoder.

[0426] 2) Predefined patterns can be exported from the encoder to the decoder.

[0427] a. For example, if the decoder determines that the current picture is a screen content picture, the predefined mode may be the DC mode, otherwise, the predefined mode is the planar mode.

[0428] viii. The predefined modes may be defined in the same manner as used for the chroma DM derivation process (eg, lumaIntraPredMode in subsection 8.4.3).

[0429] d. When the co-located luma block is encoded and decoded in normal intra prediction mode and / or BDPCM mode, the chroma intra prediction mode can be derived in the same formula as the DM mode (such as specified in Section 2.7) depending on the intra prediction mode of the co-located luma block. The derived chroma intra prediction mode is then used to select the transform set and / or transform matrix used in the RST (e.g., according to the transform set selection table shown in Section 2.6).

[0430] e. In the above bullet points, the co-located luminance block may be a codec block covering a specified luminance position, such as TL or Fig.25 CR.

[0431] 7. The transform set and / or transform matrix used in RST can depend on a single transform.

[0432] a. In one example, if the single transform applied to a block is an identity transform (e.g., applying the TS mode to a block), the transform set and / or transform matrix used in RST can be different from other types of single transforms.

[0433] b. In one example, if the horizontal and vertical one-dimensional single transforms applied to a block are the same basis (e.g., both DCT-II), the transform set and / or transform matrix used in RST can be different from single transforms with different bases from different directions (vertical or horizontal).

[0434] Signaling notification of RST side information and residual encoding and decoding

[0435] 8. Whether and / or how to signal the side information of RST (e.g., st_idx) can depend on the last non-zero coefficient in the block (in scan order).

[0436] a. In one example, RST can be enabled and the index of RST can be signaled only when the last non-zero coefficient is in the CG to which RST is applied.

[0437] b. In one example, when the last non-zero coefficient is not in the CG to which RST is applied, RST is disabled and signaling of RST is skipped.

[0438] 9. Whether and / or how to signal the side information of RST (e.g., st_idx) can depend on the coefficients of a certain color component, rather than the coefficients of all available color components in the CU.

[0439] a. In one example, only the luminance information can be used to determine whether and / or how to signal the side information of RST.

[0440] i. Alternatively, in addition, the above method is applied only when the dimensions of the block meet certain conditions.

[0441] 1) The condition is W < T1 or H < T2.

[0442] 2) For example, T1 = T2 = 4. Thus, for a 4x4 CU, the luminance block size is 4x4, and the two chrominance blocks in 4:2:0 format are 2x2. In this case, only the luminance information can be utilized.

[0443] ii. Alternatively, in addition, the above method is applied only when the current split type tree is a single tree.

[0444] b. Whether to use one color component information or all color component information may depend on the block dimension / encoding and decoding information.

[0445] 10. Whether and / or how to signal side information of RST (e.g., st_idx) may depend on the coefficients within a partial area of a block rather than the coefficients within the entire block.

[0446] a. In one example, the partial area may be defined as the CG to which RST is applied.

[0447] b. In one example, the partial area may be defined as the first M or last M (e.g., M = 1 or 2) CGs in the scan order or reverse scan order of the block.

[0448] i. In one example, M may depend on the block dimension.

[0449] ii. In one example, if the block size is 4xN and / or Nx4 (N > 8), then M is set to 2.

[0450] iii. In one example, if the block size is 4x8 and / or 8x4 and / or WxH (W >= 8, H >= 8), then M is set to 1.

[0451] c. In one example, the information of a block with dimension W×H (e.g., the number of non-zero coefficients of the block) may not be allowed to be considered to determine the use of RST and / or signal information related to RST.

[0452] i. For example, if W < T1 or H < T2, then the number of non-zero coefficients of the block is not counted. For example, T1 = T2 = 4.

[0453] d. In one example, the partial area may be defined as the upper left M×N area of the current block with dimension W×H.

[0454] i. In one example, M may be less than W and / or N may be less than H.

[0455] ii. In one example, M and N may be fixed numbers. For example, M = N = 4.

[0456] iii. In one example, M and / or N may depend on W and / or H.

[0457] iv. In one example, M and / or N may depend on the maximum allowable transform size.

[0458] 1) For example, if W is greater than 8 and H is equal to 4, then M = 8 and N = 4.

[0459] 2) For example, if H is greater than 8 and W is equal to 4, then M = 4 and N = 8.

[0460] 3) For example, if either of the above two conditions is not satisfied, M=4 and N=4.

[0461] v. Alternatively, furthermore, these methods can only be applied to certain block dimensions, such as not satisfying the conditions in 7.c.

[0462] e. In one example, the partial region may be the same for all blocks.

[0463] i. Alternatively, the partial area may be changed based on block dimensions and / or codec information.

[0464] f. In one example, a partial region may depend on a given range of scan order indices.

[0465] i. In one example, the partial area may be an area covering coefficients within a specified range, where the scanning order index of the coefficients within the specified range is within [dxS, IdxE] (including the end value) based on the coefficient scanning order of the current block with a dimension of W×H (e.g., reverse decoding order).

[0466] 1) In one example, IdxS is equal to 0.

[0467] 2) In one example, IdxE may be less than W×H-1.

[0468] 3) In one example, IdxE may be a fixed number, for example, IdxE=15.

[0469] 4) In one example, IdxE may depend on W and / or H.

[0470] a. For example, if W is greater than 8 and H is equal to 4, then IdxE=31.

[0471] b. For example, if H is greater than 8 and W is equal to 4, then IdxE=31.

[0472] c. For example, if W equals 8 and H equals 8, then IdxE=7.

[0473] d. For example, if W equals 4 and H equals 4, then IdxE=7.

[0474] e. For example, if either of the above two conditions a) and b) is not satisfied, IdxE=15.

[0475] f. For example, if any of the above four conditions a), b), c) and d) is not met, IdxE=15.

[0476] g. For example, if any of the above two conditions c) and d) is not satisfied, IdxE=15.

[0477] ii. Alternatively, furthermore, these methods can only be applied to certain block dimensions, such as not satisfying the conditions in 7.c.

[0478] g. In one example, the method may depend on the location of the non-zero coefficients within the partial region.

[0479] h. In one example, the method may depend on the energy (such as the sum of squares or the sum of absolute values) of the non-zero coefficients within the partial region.

[0480] i. In one example, the method may depend on the number of non-zero coefficients in a partial area of ​​a block rather than the entire block.

[0481] i. In one example, the method may depend on the number of non-zero coefficients in a partial region of one or more blocks in a CU.

[0482] ii. When the number of non-zero coefficients in a partial area of ​​a block is less than a threshold, the side information of signaling RST can be skipped.

[0483] iii. In one example, the threshold is fixed to N (eg, N=1 or 2).

[0484] iv. In one example, the threshold may depend on slice type / picture type / segmentation tree type (dual tree or single tree) / video content (screen content or camera captured content).

[0485] v. In one example, the threshold may depend on the color format, such as 4:2:0 or 4:4:4, and / or the color component, such as Y or Cb / Cr.

[0486] 11. When there are no non-zero coefficients in the CG to which RST can be applied, RST should be disabled.

[0487] a. In one example, when RST is applied to a block, at least one CG to which the RST is applied must contain at least one non-zero coefficient.

[0488] b. In one example, for 4xN and / or Nx4 (N>8), if RST is applied, the first two 4x4 CGs must contain at least one non-zero coefficient.

[0489] c. In one example, for 4x8 and / or 8x4, if RST is applied, the top left 4x4 must contain at least one non-zero coefficient.

[0490] d. In one example, for WxH (W>=8 and H>=8), if RST is applied, the upper left 4x4 must contain at least one non-zero coefficient.

[0491] e. A conforming bitstream must satisfy one or more of the above conditions.

[0492] 12. RST related syntax elements can be signaled before encoding and decoding the residual (eg transform coefficients / direct quantization).

[0493] a. In one example, the counts of the number of non-zero coefficients in the zeroed region (eg, numZeroOutSigCoeff) and the number of non-zero coefficients in the entire block (eg, numSigCoeff) are removed during the parsing of coefficients.

[0494] b. In one example, RST related syntax elements (eg, st_idx) may be encoded and decoded before residual_coding.

[0495] c. RST related syntax elements can be signaled conditionally (eg based on codec block flag, TS mode usage).

[0496] vi. In one example, RST related syntax elements (eg, st_idx) may be coded after signaling the codec block flag or after signaling the TS / MTS related syntax elements.

[0497] vii. In one example, when TS mode is enabled (eg, decoded transform_skip_flag is equal to 1), signaling RST related syntax elements is skipped.

[0498] d. The syntax related to the residual cannot be used for zeroing CG signaling.

[0499] e. How to encode and decode the residual (e.g., scanning order, binarization, syntax to decode, context modeling) may depend on the RST.

[0500] i. In one example, a raster scan order may be applied instead of a diagonal up-right scan order.

[0501] 1) The grid scanning order is from left to right and from top to bottom, or vice versa.

[0502] 2) Alternatively, a vertical scanning order (from top to bottom and from left to right or vice versa) may be applied instead of a diagonal top-right scanning order.

[0503] 3) Alternatively, furthermore, the context modeling may be modified.

[0504] a. In one example, context modeling may depend on previously decoded information in the template, which is the most recently scanned N neighbors in the scan order, rather than using the right, below, right below neighbors.

[0505] b. In one example, context modeling may depend on previously encoded information in the template according to the scan index (eg, -1, -2, .. assuming the current index is equal to 0).

[0506] ii. In one example, different binarization methods (eg, Rice parameter derivation) may be applied to encode the residuals associated with RST-coded and non-RST-coded blocks.

[0507] iii. In one example, for the RST codec block, signaling notification can be skipped

[0508] Certain grammatical elements.

[0509] 1) For CG applying RST, signaling of CG codec block flag (coded_sub_block_flag) can be skipped.

[0510] a. In one example, when RST 8x8 is applied to the first three CGs in a diagonal scan order, signaling the CG codec block flag is skipped for the second and third CGs (e.g., the upper left 4x4 Cg and the lower left 4x4 CG in the upper left 8x8 region of the block).

[0511] i. Alternatively, in addition, the corresponding CG codec block flag is inferred to be 0, i.e. all coefficients are zero.

[0512] b. In one example, when RST is applied to a block, signaling the CG codec block flag is skipped for the first CG in the scan order (or the last CG in the reverse scan order).

[0513] ii. Alternatively, in addition, the CG codec block flag of the upper left CG in the block is inferred to be 1, i.e. it contains at least one non-zero coefficient.

[0514] c. Fig.21 An example of an 8x8 block is depicted. When RST 8x8 or RST 4x4 is applied to an 8x8 block, coded_sub_block_flag is inferred to be 1 for CG0 and coded_sub_block_flag is inferred to be 0 for CG1 and CG2.

[0515] 2) Signaling of coefficient magnitudes and / or sign flags for certain coordinates may be skipped.

[0516] a. In one example, if the index relative to a CG in the scan order is not less than the maximum allowed index where a non-zero coefficient can exist (e.g., nonZeroSize in segment 0), the signaling of the coefficient can be skipped.

[0517] b. In one example, signaling syntax elements such as sig_coeff_flag, abs_level_gtX_flag, par_level_flag, abs_remainder, coeff_sign_flag, dec_abs_level may be skipped.

[0518] 3) Alternatively, the residual can be continuously signaled (e.g., block flags for CG encoding, coefficient amplitudes and / or sign flags for certain coordinates), but the context modeling can be modified differently for other CGs.

[0519] iv. In one example, the encoding and decoding of the residual in the CG to which RST is applied and other CGs may be different.

[0520] 1) For the above sub-bullets, they can be applied only to the CG to which RST is applied.

[0521] 13. RST related syntax elements may be signaled before other transform indications (such as transform skip and / or MTS index).

[0522] a. In one example, signaling transition skipping may depend on RST information.

[0523] i. In one example, if RST is applied to a block, the transform skip indication is not signaled for that block and is inferred to be 0.

[0524] b. In one example, signaling the MTS index may depend on the RST information.

[0525] i. In one example, if RST is applied to a block, one or more MTS change indications are not signaled and are inferred not to be used for the block.

[0526] 14. It is proposed to use different context modeling methods in arithmetic coding and decoding of different parts within a block.

[0527] a. In one example, a block is processed into two parts, scanning the first M in order

[0528] CG and remaining CG.

[0529] i. In one example, M is set to 1.

[0530] ii. In one example, for 4xN and Nx4 (N>8) blocks, M is set to 2; and for all other cases, M is set to 1.

[0531] b. In one example, a block is processed into two parts, a sub-region where RST is applied and a sub-region where RST is not applied.

[0532] i. If RST 4x4 is applied, the sub-region to which RST is applied is the previous one or two CGs of the current block.

[0533] ii. If RST 4x4 is applied, the sub-region to which RST is applied is the first three CGs of the current block.

[0534] c. In one example, it is proposed to disable the use of previously coded information in context modeling for a first part within a block, but enable the use of previously coded information in context modeling for a second part.

[0535] d. In one example, when decoding the first CG, the use of information of the remaining one or more CGs may not be allowed.

[0536] i. In one example, when the CG codec block flag of the first CG is encoded and decoded, the value of the second CG (e.g., to the right or below) is not considered.

[0537] ii. In one example, when the CG codec block flag of the first CG is encoded and decoded, the values ​​of the second CG and the third CG (e.g., the right or lower CG of WxH (W>=8 and H>=8)) are not considered.

[0538] iii. In one example, when encoding and decoding the current coefficient, if its neighbors in the context template are in different CGs, information from the neighbors is not allowed to be used.

[0539] e. In one example, when decoding coefficients in a region to which RST is applied, use of information of a remaining region to which RST is not applied may not be allowed.

[0540] f. Alternatively, furthermore, the above method may be applied under certain conditions.

[0541] i. Conditions may include whether RST is enabled.

[0542] ii. Conditions can include block dimensions.

[0543] Context Modeling in Arithmetic Coding and Decoding of RST Side Information

[0544] 15. When encoding and decoding RST indexes, context modeling may depend on whether explicit or implicit multiple transform selection (MTS) is enabled.

[0545] a. In one example, when implicit MTS is enabled, different contexts may be selected for blocks encoded in the same intra prediction mode.

[0546] i. In one example, block dimensions such as shape (square or non-square) are used to select context.

[0547] b. In one example, instead of checking the transform index of the explicit MTS codec (e.g.

[0548] tu_mts_idx), a transformation matrix basis may be used instead.

[0549] i. In one example, for a transform matrix basis of DCT-II with both horizontal and vertical one-dimensional transforms, the corresponding context may be different from other types of transform matrices.

[0550] 16. When encoding and decoding RST indexes, context modeling can depend on whether CCLM is enabled

[0551] (e.g., sps_cclm_enabled_flag).

[0552] a. Alternatively, whether to enable or how to select a context for RST index encoding and decoding may depend on whether CCLM is applied to a block.

[0553] b. In one example, context modeling may depend on whether CCLM is enabled for the current block.

[0554] i. In one example, intraModeCtx =

[0555] sps_cclm_enabled_flag? (intra_chroma_pred_mode[x0][y0]is CCLM:intra_chroma_pred_mode[x0][y0]is DM)? 1:0.

[0556] c. Alternatively, whether to enable or how to select the context for RST index coding may depend on whether the current chroma block is coded in DM mode.

[0557] i. In one example, the intraModeCtx=(intra_chroma_pred_mode[x0][y0]==(sps_cclm_enabled_flag?7:4))?1:0.

[0558] 17. When encoding and decoding RST indexes, context modeling may depend on the block dimension / partition depth (eg, quadtree depth and / or BT / TT depth).

[0559] 18. When encoding and decoding RST indexes, context modeling may depend on the color format and / or color components.

[0560] 19. When encoding and decoding RST indexes, context modeling can be independent of intra prediction mode and / or MTS index.

[0561] 20. When encoding and decoding the RST index, the first and / or second binary tree can be context-encoded with a unique context; or bypass-encoded.

[0562] Call the RST procedure under the condition

[0563] 21. Whether to call the reverse RST process may depend on the CG codec block flag.

[0564] a. In one example, if the upper left CG codec block flag is zero, this process does not need to be called.

[0565] i. In one example, if the upper left CG codec block flag is zero and the block size is not equal to 4xN / Nx4 (N>8), this process does not need to be called.

[0566] b. In one example, if the first two CG codec block flags in the scan order are both equal to zero, then the process does not need to be called.

[0567] i. In one example, if the first two CG codec block flags in the scan order are both equal to zero and the block size is equal to 4xN / Nx4 (N>8), this process does not need to be called.

[0568] 22. Whether to call the reverse RST process may depend on the block dimension.

[0569] a. In one example, for certain block dimensions (such as 4x8 / 8x4), RST may be disabled. Alternatively, signaling RST-related syntax elements may also be skipped.

[0570] Unification of dual-tree and single-tree encoding and decoding

[0571] 23. In dual-tree and single-tree coding, the use of RST and / or the signaling of RST-related information may be determined in the same manner.

[0572] a. For example, when the number of counted non-zero coefficients (e.g., numSigCoeff specified in JVET-N0193) is not greater than T1 in the case of dual-tree coding or not greater than T2 in the case of single-tree coding, RST should not be applied and relevant information is not signaled, where T1 is equal to T2.

[0573] b. In one example, both T1 and T2 are set to N, such as N=1 or 2.

[0574] Consider multiple TUs in a CU

[0575] 24. Whether and / or how to apply RST may depend on the block dimensions W×H.

[0576] a. In one example, if W>T1 or H>T2, RST may not be applied.

[0577] b. In one example, if W>T1 and H>T2, RST may not be applied.

[0578] c. In one example, if W*H>=T, then RST may not be applied.

[0579] d. For the above example, the following applies:

[0580] i. In one example, a block is a CU.

[0581] ii. In one example, T1=T2=64.

[0582] iii. In one example, T1 and / or T2 may depend on the maximum allowed transform size. For example, T1 = T2 = maximum allowed transform size.

[0583] iv. In one example, T is set to 4096.

[0584] e. Alternatively, further, if it is determined that RST is not to be applied, the related information may not be signaled.

[0585] 25. When there are N (N>1) TUs in a CU, the coding and decoding information of only one TU among the N TUs is used to determine the use of RST and / or the signaling notification of information related to RST.

[0586] a. In one example, the first TU of a CU in decoding order may be used to make the determination.

[0587] b. In one example, the top left TU of the CU in decoding order may be used to make the determination.

[0588] c. In one example, for the case when there is only one TU in the CU, the determination of the designated TU may be made in the same way.

[0589] 26. The use of RST and / or the signaling of the related information of RST may be performed at the TU level or PU level instead of the CU level.

[0590] a. Alternatively, different TU / PUs within a CU may select different secondary transform matrices or enable / disable control flags.

[0591] b. Alternatively, furthermore, for the dual-tree case and encoding / decoding chroma blocks, different color components may select different secondary transform matrices or enable / disable control flags.

[0592] c. Alternatively, whether to signal RST-related information at the video unit level may depend on the partition tree type (dual tree or single).

[0593] d. Alternatively, whether to signal RST-related information at the video unit level may depend on the relationship between CU / PU / TU and the maximum allowed transform block size, such as larger or smaller.

[0594] 5 Example Implementations of the Technology Disclosed

[0595] In the following exemplary embodiments, changes over JVET-N0193 are highlighted in bold and italics. Deleted text is marked with double brackets (eg, [[a]] indicates deletion of the character "a").

[0596] 5.1 Example #1

[0597] The signaling of the RST index depends on the number of non-zero coefficients in a sub-region of the block rather than in the entire block.

[0598] 7.3.6.11 Residual Codec Syntax

[0599]

[0600]

[0601]

[0602]

[0603]

[0604] Alternatively, the condition could be replaced with:

[0605]

[0606] 5.2 Example #2

[0607] RST can be called without a certain CG codec block flag.

[0608] 8.7.4. Scaling the transformation coefficients

[0609] 8.7.4.1 General

[0610] The inputs to this process are:

[0611] –Specify the luminance position (xTbY, yTbY) of the upper left sample of the current luminance transform block relative to the upper left luminance sample of the current picture,

[0612] – The variable nTbW specifies the width of the current transform block,

[0613] – Variable nTbH, specifies the height of the current transform block,

[0614] – Variable cIdx, specifies the color component of the current block,

[0615] – A (nTbW) x (nTbH) array d[x][y] of scaled transform coefficients, where x = 0..nTbW–1, y = 0..nTbH-1.

[0616] The output of this process is an (nTbW)x(nTbH) array r[x][y] of residual samples, where x=0..nTbW–1 and y=0..nTbH-1.

[0617] The variable bInvokeST is set to 0 and is further modified to 1 if one of the following conditions is true:

[0618] – If coded_sub_block_flag[0][0] is equal to 1 and nTbW x nTbH ! = 32

[0619] – If coded_sub_block_flag[0][0] and coded_sub_block_flag[0][1] are equal to 1 and nTbW is equal to 4 and nTbH is greater than 8.

[0620] – If coded_sub_block_flag[0][0] and coded_sub_block_flag[1][0] are equal to 1 and nTbW is equal to 8 and nTbH is equal to 4.

[0621] If bInvokeST is equal to 1 and st_idx[xTbY][yTbY] is not equal to 0, the following applies:

[0622] 1. The variables nStSize, log2StSize, numStX, numStY, and nonZeroSize are derived as follows:

[0623] – If both nTbW and nTbH are greater than or equal to 8, log2StSize is set to 3 and nStOutSize is set to 48.

[0624] – Otherwise, log2StSize is set to 2 and nStOutSize is set to 16.

[0625] –nStSize is set to (1< <log2StSize)。

[0626] – If nTbH is equal to 4 and nTbW is greater than 8, numStX is set equal to 2.

[0627] – Otherwise, numStX is set equal to 1.

[0628] – If nTbW is equal to 4 and nTbH is greater than 8, numStY is set equal to 2.

[0629] – Otherwise, numStY is set equal to 1.

[0630] – If nTbW and nTbH are both equal to 4 or nTbW and nTbH are both equal to 8, then nonZeroSize is set equal to 8.

[0631] – Otherwise, nonZeroSize is set equal to 16.

[0632] 2. For xSbIdx = 0..numStX–1 and ySbIdx = 0..numStY–1, the following applies:

[0633] – The variable array u[x], where x = 0..nonZeroSize–1, is derived as follows:

[0634] xC=(xSbIdx< <log2StSize)+

[0635] DiagScanOrder[log2StSize][log2StSize][x][0]

[0636] yC=(ySbIdx< <log2StSize)+

[0637] DiagScanOrder[log2StSize][log2StSize][x][1]

[0638] u[x]=d[xC][yC]

[0639] – Transform u[x] (where x = 0..nonZeroSize–1) into the variable array v[x] (where x = 0..nStOutSize–1) by using the following call to the one-dimensional transform process specified in Section 8.7.4.4: The scaling transform coefficient, the transform input length nonZeroSize, the transform output length nStOutSize, the list u[x] (where x = 0..nonZeroSize–1), the index stPredModeIntra of the transform set selection, and the index st_idx[xTbY][yTbY] of the transform selection in the transform set are used as inputs, and the output is the list v[x] (where x = 0..nStOutSize–1). The variable stPredModeIntra is set to predModeIntra specified in Section 8.4.4.2.1.

[0640] – The array d[(xSbIdx<<log2StSize)+x][(ySbIdx<<log2StSize)+y], where x = 0..nStSize–1, y = 0..nStSize–1, is derived as follows:

[0641] – If stPredModeIntra is less than or equal to 34, or equal to INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM, then the following applies:

[0642] d[(xSbIdx<<log2StSize)+x][(ySbIdx<<log2StSize)+y] = (y < 4)? v[x+(y<<log2StSize)] : ((x < 4)? v[32+x+((y–4)<<2)] : d[(xSbIdx<<log2StSize)+x][(ySbIdx<<log2StSize)+y])

[0643] – Otherwise, the following applies:

[0644] d[(xSbIdx<<log2StSize)+x][(ySbIdx<<log2StSize)+y] = (y < 4)? v[y+(x<<log2StSize)] : ((x < 4)? v[32+(y–4)+(x<<2)] : d[(xSbIdx<<log2StSize)+x][(ySbIdx<<log2StSize)+y])

[0645] The variable implicitMtsEnabled is derived as follows:

[0646] – implicitMtsEnabled is set equal to 1 if sps_mts_enabled_flag is equal to 1 and one of the following conditions is true:

[0647] –IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT

[0648] –cu_sbt_flag is equal to 1 and Max(nTbW,nTbH) is less than or equal to 32

[0649] –sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0 and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA.

[0650] – Otherwise, implicitMtsEnabled is set equal to 0.

[0651] The variable trTypeHor that specifies the horizontal transformation kernel and the variable trTypeVer that specifies the vertical transformation kernel are derived as follows:

[0652] – If cIdx is greater than 0, trTypeHor and trTypeVer are set equal to 0.

[0653] – Otherwise, if implicitMtsEnabled is equal to 1, the following applies:

[0654] – If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, trTypeHor and trTypeVer are specified in Table 8-15, depending on intraPredMode.

[0655] – Otherwise, if cu_sbt_flag is equal to 1, trTypeHor and trTypeVer are specified in Table 8-14, depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag.

[0656] – Otherwise (sps_explicit_mts_intra_enabled_flag and

[0657] sps_explicit_mts_inter_enabled_flag is equal to 0), trTypeHor and trTypeVer are derived as follows:

[0658] trTypeHor=(nTbW>=4&&nTbW<=16&&nTbW<=

[0659] nTbH)? 1:0(8-1029)

[0660] trTypeVer=(nTbH>=4&&nTbH<=16&&nTbH<=

[0661] nTbW)? 1:0(8-1030)

[0662] – Otherwise, trTypeHor and trTypeVer are specified in Table 8-13, depending on

[0663] tu_mts_idx[xTbY][yTbY].

[0664] The variables nonZeroW and nonZeroH are derived as follows:

[0665] nonZeroW=Min(nTbW,(trTypeHor>0)?16:32) (8-1031)

[0666] nonZeroH=Min(nTbH,(trTypeVer>0)?16:32) (8-1032)

[0667] The (nTbW)x(nTbH) array of residual samples is derived as follows:

[0668] 1. When nTbH is greater than 1, each (vertical) column is scaled by the transform coefficients d[x][y] (where x = 0..nonZeroW–1,

[0669] y=0..nonZeroH–1) is transformed into e[x][y] (where x=0..nonZeroW–1,

[0670] y=0..nTbH–1): For each column x=0..nonZeroW-1, the height nTbH of the transform block, the non-zero height nonZeroH of the scaled transform coefficients, the list d[x][y] (where y=0..nonZeroH–1) and the transform type variable trType set equal to trTypeVer are taken as input, and the output is the list e[x][y] where y=0..nTbH-1.

[0671] 2. When both nTbH and nTbW are greater than 1, the intermediate sample value g[x][y], where x = 0..nonZeroW–1, y = 0..nTbH–1, is derived as follows:

[0672] g[x][y]=Clip3(CoeffMin,CoeffMax,(e[x][y]+64)>>7)

[0673] (8-1033)

[0674] When nTbW is greater than 1, the resulting array g[x][y] (where x = 0..nonZeroW–1, y = 0...nTbH–1) for each (horizontal) row is transformed into r[x][y] (where x = 0...nTbW–1, y = 0..nTbH–1) using the following one-dimensional transformation process specified in calling section 8.7.4.2: for each row y = 0...nTbH-1, the width nTbW of the transform block, the non-zero width nonZeroW of the resulting array g[x][y], the list g[x][y] (where x = 0..nonZeroW–1) and the transform type variable trType set equal to trTypeHor are taken as input, and the output is the list r[x][y], where y = 0..nTbW-1.

[0675] 5.3 Example #3

[0676] Revised context modeling of RST indexes.

[0677] 5.3.1 Alternative Example #1

[0678] 9.5.4.2.8 Derivation of ctxInc for syntax element st_idx

[0679] The inputs to this process are: the color component index cIdx; the luma or chroma position (x0, y0), depending on whether cIdx specifies the top left sample of the current luma or chroma codec block relative to the top left sample of the current picture; the tree type treeType; the luma intra prediction mode IntraPredModeY[x0][y0] as specified in Section 8.4.2; the syntax element intra_chroma_pred_mode[x0][y0] specifying the intra prediction mode for chroma samples as specified in Section 7.4.7.5; the block width nTbW and height nTbH; and a number of transform select indices tu_mts_idx[x0][y0].

[0680] The output of this process is the variable ctxInc.

[0681] The variable intraModeCtx is exported as follows:

[0682] If cIdx is equal to 0, intraModeCtx is derived as follows:

[0683] intraModeCtx=(IntraPredModeY[x0][y0]<=1)? 1:0

[0684] Otherwise (cIdx is greater than 0), intraModeCtx is derived as follows:

[0685] intraModeCtx=(intra_chroma_pred_mode[x0][y0]>=4)? 1:0

[0686] The variable mtsCtx is exported as follows:

[0687] mtsCtx=((sps_explicit_mts_intra_enabled_flag?tu_mts_idx[x0][y0]==0:nTbW==nTbH)&&treeType!=SINGLE_TREE)? 1:0

[0688] The variable ctxInc is exported as follows:

[0689] ctxInc=(binIdx<<1)+intraModeCtx+(mtsCtx<<2)

[0690] 5.3.2 Alternative Example #2

[0691]

[0692] Table 9-15 – Assignment of ctxInc to syntax elements using binary numbers of context codecs

[0693]

[0694] [[9.5.4.2.8 Derivation process of ctxInc of syntax element st_idx

[0695] The inputs to this process are: the color component index cIdx; the luma or chroma position (x0, y0), depending on whether cIdx specifies the top left sample of the current luma or chroma codec block relative to the top left sample of the current picture; the tree type treeType; the luma intra prediction mode IntraPredModeY[x0][y0] as specified in Section 8.4.2; the syntax element intra_chroma_pred_mode[x0][y0] specifying the intra prediction mode for chroma samples as specified in Section 7.4.7.5; and a number of transform select indices tu_mts_idx[x0][y0].

[0696] The output of this process is the variable ctxInc.

[0697] The variable intraModeCtx is exported as follows:

[0698] If cIdx is equal to 0, intraModeCtx is derived as follows:

[0699] intraModeCtx=(IntraPredModeY[x0][y0]<=1)? 1:0

[0700] Otherwise (cIdx is greater than 0), intraModeCtx is derived as follows:

[0701] intraModeCtx=(intra_chroma_pred_mode[x0][y0]>=4)? 1:0

[0702] The variable mtsCtx is exported as follows:

[0703] mtsCtx=(tu_mts_idx[x0][y0]==0&&treeType!=SINGLE_TREE)? 1:0

[0704] The variable ctxInc is exported as follows:

[0705] ctxInc=(binIdx<<1)+intraModeCtx+(mtsCtx<<2)]]

[0706] 5.4 Example #4

[0707] Corresponds to bullets 7.c and 7.d.

[0708] 7.3.7.11 Residual Codec Syntax

[0709]

[0710]

[0711] In an alternative example, the following may apply:

[0712]

[0713]

[0714] In one example, the following may apply:

[0715] If log2TbWidth>3&&log2TbHeight==2, then SigRangeX is equal to 8. Otherwise, SigRangeX is equal to 4. If log2TbHeight>3&&log2TbWidth==2, then SigRangeY is equal to 8. Otherwise, SigRangeY is equal to 4.

[0716] 5.5 Example #5

[0717] Corresponds to item 19

[0718] 7.3.6.5 Codec unit syntax

[0719]

[0720]

[0721] 5.6 Example #6

[0722] Corresponds to item 20

[0723] 7.3.6.5 Codec unit syntax

[0724]

[0725]

[0726] 5.7 Example #7

[0727] Corresponds to item 21

[0728] 7.3.7.11 Residual Codec Syntax

[0729]

[0730]

[0731] (CbX[x0][y0],CbY[x0][y0]) specifies the upper left position of the codec unit overlay position (x0,y0).

[0732] 5.8 Example #8

[0733] The RST transform set index is derived from the default mode assigned to the non-normal intra prediction mode. New additions are highlighted in bold and italics and deleted parts are marked with double brackets (eg, [[a]] indicates deletion of the character "a").

[0734] 8.7.4.1 General

[0735] The inputs to this process are:

[0736] –Specify the luminance position (xTbY, yTbY) of the upper left sample of the current luminance transform block relative to the upper left luminance sample of the current picture,

[0737] – The variable nTbW specifies the width of the current transform block,

[0738] – Variable nTbH, specifies the height of the current transform block,

[0739] – Variable cIdx, specifies the color component of the current block,

[0740] – A (nTbW) x (nTbH) array d[x][y] of scaled transform coefficients, where x = 0..nTbW–1, y = 0..nTbH-1.

[0741] The output of this process is an (nTbW)x(nTbH) array r[x][y] of residual samples, where x=0..nTbW–1 and y=0..nTbH-1.

[0742] When lfnst_idx[xTbY][yTbY] is not equal to 0 and both nTbW and nTbH are greater than or equal to 4, the following applies:

[0743] –Variables predModeIntra, nLfnstOutSize, log2LfnstSize, nLfnstSize, and

[0744] nonZeroSize is exported as follows:

[0745] predModeIntra=

[0746] (cIdx==0)? IntraPredModeY[xTbY][yTbY]:IntraPredModeC[xTbY][yTbY](8-965)

[0747] nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16 (8-966)

[0748] log2LfnstSize=(nTbW>=8&&nTbH>=8)? 3:2 (8-967)

[0749] nLfnstSize=1< <log2LfnstSize (8-968)

[0750] nonZeroSize=((nTbW==4&&nTbH==4)||

[0751] (nTbW==8&&nTbH==8))? 8:16(8-969)

[0752] – When intra_mip_flag[xTbComp][yTbComp] is equal to 1 and cIdx is equal to 0, predModeIntra is set equal to INTRA_PLANAR.

[0753] – When predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM, predModeIntra is derived as follows: [[Set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2]. ]]

[0754] – If intra_mip_flag[xTbY][yTbY] is equal to 1, predModeIntra is set equal to INTRA_PLANAR.

[0755] – Otherwise, if CuPredMode[0][xTbY][yTbY] is equal to MODE_IBC or MODE_PLT, predModeIntra is set equal to INTRA_DC.

[0756] – Otherwise, predModeIntra is set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2].

[0757] – Invoke the wide angle intra prediction mode mapping process specified in section 8.4.5.2.6 with: predModeIntra, nTbW, nTbH and cIdx as inputs and the modified predModeIntra as output.

[0758] Alternatively, the following may apply:

[0759] – When predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM, predModeIntra is derived as follows: [[Set to equal

[0760] IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2]. ]]

[0761] – If intra_mip_flag[xTbY+nTbW / 2][yTbY+nTbH / 2] is equal to 1, predModeIntra is set equal to INTRA_PLANAR.

[0762] – Otherwise, if CuPredMode[0][xTbY+nTbW / 2][yTbY+nTbH / 2] is equal to MODE_IBC or MODE_PLT, predModeIntra is set equal to INTRA_DC.

[0763] – Otherwise, predModeIntra is set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2].

[0764] 5.9 Example #9

[0765] The RST transform set index is derived from the default mode assigned to the non-normal intra prediction mode and depends on the color format. New additions are highlighted in bold and italics and deleted parts are marked with double brackets (e.g., [[a]] means deleting the character "a").

[0766] 8.7.4.2 General

[0767] The inputs to this process are:

[0768] –Specify the luminance position (xTbY, yTbY) of the upper left sample of the current luminance transform block relative to the upper left luminance sample of the current picture,

[0769] – The variable nTbW specifies the width of the current transform block,

[0770] – Variable nTbH, specifies the height of the current transform block,

[0771] – Variable cIdx, specifies the color component of the current block,

[0772] – an (nTbW) x (nTbH) array d[x][y] of scaled transform coefficients, where x = 0..nTbW–1,

[0773] y=0..nTbH-1.

[0774] The output of this process is an (nTbW)x(nTbH) array r[x][y] of residual samples, where x=0..nTbW–1 and y=0..nTbH-1.

[0775] When lfnst_idx[xTbY][yTbY] is not equal to 0 and both nTbW and nTbH are greater than or equal to 4, the following applies:

[0776] – The variables predModeIntra, nLfnstOutSize, log2LfnstSize, nLfnstSize, and nonZeroSize are derived as follows:

[0777] predModeIntra=

[0778] (cIdx==0)? IntraPredModeY[xTbY][yTbY]:IntraPredModeC[xTbY][yTbY] (8-965)

[0779] nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16 (8-966)

[0780] log2LfnstSize=(nTbW>=8&&nTbH>=8)? 3:2 (8-967)

[0781] nLfnstSize=1< <log2LfnstSize (8-968)

[0782] nonZeroSize=((nTbW==4&&nTbH==4)||

[0783] (nTbW==8&&nTbH==8))? 8:16 (8-969)

[0784] – When intra_mip_flag[xTbComp][yTbComp] is equal to 1 and cIdx is equal to 0, predModeIntra is set equal to INTRA_PLANAR.

[0785] – When predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM, predModeIntra is derived as follows: [[Set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2]. ]]

[0786] – If intra_mip_flag[xTbY][yTbY] is equal to 1, predModeIntra is set equal to INTRA_PLANAR.

[0787] – Otherwise, if CuPredMode[0][xTbY][yTbY] is equal to MODE_IBC or MODE_PLT, predModeIntra is set equal to INTRA_DC.

[0788] – Otherwise, predModeIntra is set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2].

[0789] – When chroma_format_idc is equal to 2, predModeIntra is further modified according to Table 8-3. Chroma intra prediction mode X is set equal to predModeIntra to derive chroma intra prediction mode Y. Then, predModeIntra is set equal to Y.

[0790] – Invoke the wide angle intra prediction mode mapping process specified in section 8.4.5.2.6 with: predModeIntra, nTbW, nTbH and cIdx as inputs and the modified predModeIntra as output.

[0791] Table 8-3 – Specification of the 4:2:2 mapping process from chroma intra prediction mode X to mode Y when chroma_format_idc is equal to 2

[0792]

[0793]

[0794] Alternatively, the following may apply:

[0795] – When predModeIntra is equal to INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM, predModeIntra is derived as follows: [[Set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2]. ]]

[0796] – If intra_mip_flag[xTbY+nTbW / 2][yTbY+nTbH / 2] is equal to 1, predModeIntra is set equal to INTRA_PLANAR.

[0797] – Otherwise, if CuPredMode[0][xTbY+nTbW / 2][yTbY+nTbH / 2] is equal to MODE_IBC or MODE_PLT, predModeIntra is set equal to INTRA_DC.

[0798] – Otherwise, predModeIntra is set equal to IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2].

[0799] – When chroma_format_idc is equal to 2, predModeIntra is further modified according to Table 8-3. Chroma intra prediction mode X is set equal to predModeIntra to derive chroma intra prediction mode Y. Then, predModeIntra is set equal to Y.

[0800] 5.10 Example #10

[0801] In this embodiment, it is checked whether the middle luma sample of the corresponding luma area of ​​the current chroma block is encoded or decoded in MIP, IBC or palette mode; and the setting of DM is also based on the middle luma sample.

[0802] New additions are highlighted in bold and italics and deleted parts are marked with double brackets (eg, [[a]] indicates deletion of the character "a").

[0803] 8.4.3 Derivation process of chroma intra prediction mode

[0804] The inputs to this process are:

[0805] –Specifies the luminance position (xCb, yCb) of the upper left sample of the current chroma codec block relative to the upper left luminance sample of the current picture,

[0806] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0807] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0808] In this process, the chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived.

[0809] The corresponding luma intra prediction mode lumaIntraPredMode is derived as follows:

[0810] – If intra_mip_flag[xCb+cbWidth / 2][yCb+cbHeight / 2] is equal to 1, lumaIntraPredMode is set equal to INTRA_PLANAR.

[0811] – Otherwise, if CuPredMode[0][xCb+cbWidth / 2][yCb+cbHeight / 2] is equal to MODE_IBC or MODE_PLT, lumaIntraPredMode is set equal to INTRA_DC.

[0812] – Otherwise, lumaIntraPredMode is set equal to

[0813] IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2].

[0814] The chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived using cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode as specified in Table 8-2.

[0815] Table 8-2 – Specification of IntraPredModeC[xCb][yCb] depending on cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode

[0816]

[0817] When chroma_format_idc is equal to 2, chroma intra prediction mode Y is derived using chroma intra prediction mode X in Table 8-2, as specified in Table 8-3, and then chroma intra prediction mode X is set equal to chroma intra prediction mode Y.

[0818] 5.11 Example #11

[0819] In this embodiment, it is checked whether the upper left luma sample of the corresponding luma area of ​​the current chroma block is encoded or decoded in MIP, IBC or palette mode; and the setting of DM is also based on the middle luma sample.

[0820] New additions are highlighted in bold and italics and deleted parts are marked with double brackets (eg, [[a]] indicates deletion of the character "a").

[0821] 8.4.3 Derivation process of chroma intra prediction mode

[0822] The inputs to this process are:

[0823] –Specifies the luminance position (xCb, yCb) of the upper left sample of the current chroma codec block relative to the upper left luminance sample of the current picture,

[0824] – The variable cbWidth specifies the width of the current codec block in luminance samples.

[0825] – The variable cbHeight specifies the height of the current codec block in luma samples.

[0826] In this process, the chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived.

[0827] The corresponding luma intra prediction mode lumaIntraPredMode is derived as follows:

[0828] – If intra_mip_flag[xCb][yCb] is equal to 1, lumaIntraPredMode is set equal to INTRA_PLANAR.

[0829] – Otherwise, if CuPredMode[0][xCb][yCb] is equal to MODE_IBC or MODE_PLT, lumaIntraPredMode is set equal to INTRA_DC.

[0830] – Otherwise, lumaIntraPredMode is set equal to

[0831] IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2].

[0832] The chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived using cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode as specified in Table 8-2.

[0833] Table 8-2 – Specification of IntraPredModeC[xCb][yCb] depending on cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode

[0834]

[0835]

[0836] When chroma_format_idc is equal to 2, chroma intra prediction mode Y is derived using chroma intra prediction mode X in Table 8-2, as specified in Table 8-3, and then chroma intra prediction mode X is set equal to chroma intra prediction mode Y.

[0837] The examples described above may be incorporated into the context of the methods described below, such as methods 2200 , 2210 , 2220 , 2230 , 2240 , and 2250 , which may be implemented at a video decoder or a video encoder.

[0838] Fig.22A A flowchart of an exemplary method of video processing is shown. Method 2200 includes, at step 2202, selecting a transform set or a transform matrix based on characteristics of a current video block to apply a reduced secondary transform to the current video block.

[0839] The method 2200 includes, at step 2204, applying a selected transform set or transform matrix to a portion of the current video block as part of a conversion between the current video block and a bitstream representation of a video including the current video block.

[0840] In some embodiments, the portion of the current video block is an upper right sub-region, a lower right sub-region, a lower left sub-region, or a middle sub-region of the current video block.

[0841] In some embodiments, the characteristic of the current video block is an intra prediction mode or a primary transform matrix of the current video block.

[0842] In some embodiments, the characteristic is a color component of the current video block.In an example, a first transform set is selected for a luma component of the current video block, and wherein a second transform set different from the first transform set is selected for one or more chroma components of the current video block.

[0843] In some embodiments, the characteristic is an intra prediction mode or an intra coding method of the current video block. In an example, the intra prediction method includes a prediction method based on multiple reference lines (MRL) or a matrix-based intra prediction method. In another example, when the current video block is a cross-component linear model (CCLM) coded block, a first transform set is selected, and wherein, when the current video block is a non-CCLM coded block, a second transform set different from the first transform set is selected. In another example, when the current video block is coded and decoded in a joint chroma residual coding method, a first transform set is selected, and wherein, when the current video block is not coded and decoded in a joint chroma residual coding method, a second transform set different from the first transform set is selected.

[0844] In some embodiments, the characteristic is a transform of the current video block.

[0845] Fig. 22B A flow chart of an exemplary method of video processing is shown. The method 2210 includes, at step 2212, making a decision based on one or more coefficients associated with a current video block regarding selectively including signaling side information to apply a reduced secondary transform (RST) in a bitstream representation of the current video block.

[0846] The method 2210 includes, at step 2214, performing a conversion between the current video block and a video represented by a bitstream including the current video block based on the determination.

[0847] In some embodiments, the one or more coefficients comprise the last non-zero coefficient in the scan order of the current video block.

[0848] In some embodiments, the one or more coefficients include a plurality of coefficients within a partial region of the current video block. In an example, the partial region includes one or more codec groups to which the RST is applicable. In another example, the partial region includes the first M or last M codec groups in a scan order of the current video block. In yet another example, the partial region includes the first M or last M codec groups in a reverse scan order of the current video block. In yet another example, a determination is further made based on an energy of one or more non-zero coefficients of the plurality of coefficients.

[0849] Fig. 22C A flowchart of an exemplary method of video processing is shown. Method 2220 includes, at step 2222, configuring a bitstream representation of a current video block to apply a reduced secondary transform (RST) to the current video block, wherein syntax elements related to the RST are signaled in the bitstream representation before encoding and decoding residual information.

[0850] The method 2220 includes, at step 2224, performing a conversion between the current video block and a bitstream representation of the current video block based on the configuration.

[0851] In some embodiments, signaling of RST related syntax elements is based on usage of at least one codec block flag or transform selection mode.

[0852] In some embodiments, the bitstream representation excludes codec residual information corresponding to codec groups having all zero coefficients.

[0853] In some embodiments, the coding residual information is based on the application of RST.

[0854] Fig.22DA flowchart of an exemplary method of video processing is shown. Method 2230, at step 2232, configure a bitstream representation of a current video block to apply a reduced secondary transform (RST) to the current video block, wherein a syntax element related to the RST is signaled in the bitstream representation before a transform skip indication or a multiple transform set (MTS) index.

[0855] The method 2230 includes, at step 2234, performing a conversion between the current video block and a bitstream representation of the current video block based on the configuration.

[0856] In some embodiments, the transform skip indication or the MTS index is based on RST related syntax elements.

[0857] Fig.22E A flow chart of an exemplary method of video processing is shown. Method 2240 includes, at step 2242, configuring a context model based on characteristics of a current video block to encode and decode an index of a reduced secondary transform (RST).

[0858] The method 2240 includes, at step 2244, performing a conversion between the current video block and a bitstream representation of a video including the current video block based on the configuration.

[0859] In some embodiments, a feature is to explicitly or implicitly enable a Multiple Transform Selection (MTS) process.

[0860] In some embodiments, the characteristic is enabling a cross-component linear model (CCLM) codec mode in the current video block.

[0861] In some embodiments, the characteristic is the size of the current video block.

[0862] In some embodiments, the characteristic is a partition depth of a partitioning process applied to the current video block.In an example, the partitioning process is a quadtree (QT) partitioning process, a binary tree (BT) partitioning process, or a ternary tree (TT) partitioning process.

[0863] In some embodiments, the characteristic is the color format or color components of the current video block.

[0864] In some embodiments, the characteristics exclude the intra-prediction mode of the current video block and the index of a multiple transform selection (MTS) process.

[0865] Fig.22F A flow chart of an exemplary method of video processing is shown. Method 2250 includes, at step 2252, making a decision based on characteristics of a current video block regarding selectively applying a reverse downscaled secondary transform (RST) process on the current video block.

[0866] The method 2250 includes, at step 2254, performing a conversion between the current video block and a bitstream representation of a video including the current video block based on the determination.

[0867] In some embodiments, the characteristic is a codec block flag of a codec group of the current video block. In an example, a reverse RST process is not applied, and wherein the codec block flag of the top left codec group is zero. In another example, a reverse RST process is not applied, and wherein the codec block flags of the first and second codec groups in the scan order of the current video block are zero.

[0868] In some embodiments, the characteristic is the height (M) or width (N) of the current video block. In an example, the reverse RST process is not applied, and wherein (i) M=8 and N=4, or (ii) M=4 and N=8.

[0869] Fig.26A A flow chart of an exemplary method of video processing is shown. The method 2610 includes, at step 2612, for conversion between a current video block of a video unit of a video and a coded representation of the video, determining a default intra-frame prediction mode for a video unit coded and decoded using a certain intra-frame prediction mode, so that a prediction block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction. The method 2610 also includes, at step 2614, performing conversion based on the determination.

[0870] Fig.26B A flow chart of an exemplary method of video processing is shown. The method 2620 includes, at step 2622, using a rule to determine a predetermined position at which a luma block of a video covers a chroma block of the video. The method 2620 also includes, at step 2624, performing a conversion between the video and a codec representation of the video based on the determination. In some implementations, the chroma block is represented in a codec representation using an intra-frame prediction mode.

[0871] Fig.26C A flow chart of an exemplary method of video processing is shown. The method 2630 includes, at step 2632, using a rule to derive an intra-frame prediction mode of a chroma block of a video based on a codec mode of a luminance block corresponding to the chroma block. The method 2630 also includes, at step 2634, performing a conversion between the chroma block and a codec representation of the video based on the derived intra-frame prediction mode. In some implementations, the rule specifies using a default intra-frame prediction mode in case the codec mode of the luminance block is a certain intra-frame prediction mode, wherein a prediction block of the luminance block is generated without extrapolating the neighborhood pixels of the luminance block along a direction.

[0872] Fig.27AA flow chart of an exemplary method for video processing is shown. Method 2710 includes, at step 2712, making a first determination whether to apply a non-normal chroma intra-frame prediction mode to a chroma block of the video. Method 2710 also includes, at step 2714, making a second determination to apply a luma intra-frame prediction mode to a luma block corresponding to the chroma block. Method 2710 also includes, at step 2716, making a third determination to apply a transform set or a transform matrix to the chroma block based on the luma intra-frame prediction mode. Method 2710 also includes, at step 2718, performing a conversion between a video and a coded representation of the video based on the third determination. In some implementations, the non-normal chroma intra-frame prediction mode includes encoding and decoding the chroma block without using neighborhood pixel values ​​extrapolated along the chroma prediction direction.

[0873] Fig.27B A flow chart of an exemplary method of video processing is shown. The method 2720 includes, at step 2722, for a chroma block of a video, making a first determination that a luma block corresponding to the chroma block is encoded and decoded using a non-normal luma intra prediction mode. The method 2720 also includes, at step 2724, making a second determination of a transform set or transform matrix for the chroma block based on the first determination according to a rule. The method 2720 also includes, at step 2726, performing conversion between the video and a codec representation of the video according to the second determination. In some implementations, the rule specifies that because the luma block is encoded and decoded using a non-normal luma intra prediction mode, one or more default modes or default transform sets associated with the chroma block determine a transform set or transform matrix in case the chroma block is encoded and decoded using a non-normal chroma intra prediction mode, wherein the non-normal luma intra prediction mode includes encoding and decoding the luma block without using neighboring pixel values ​​extrapolated along a luma prediction direction, and wherein the non-normal chroma intra prediction mode includes encoding and decoding the chroma block without using neighboring pixels extrapolated along a chroma prediction direction.

[0874] Fig.27C A flow chart of an exemplary method of video processing is shown. Method 2730 includes, at step 2732, determining the applicability of a secondary transform tool applied to the current video block of a color component for conversion between a current video block of a video and a codec representation of the video based on at least one of: 1) a codec mode of a corresponding block of another color component, or 2) a codec mode of the current video block. Method 2730 also includes, at step 2734, performing the conversion based on the determination. In some implementations, a second transform tool is used: during encoding, a forward secondary transform is applied to the output of a forward primary transform applied to a residual of the current video block before quantization, or during decoding, an inverse secondary transform is applied to a dequantized output of the current video block before applying an inverse primary transform.

[0875] Fig.27D A flow chart of an exemplary method of video processing is shown. Method 2740 includes, at step 2742, for a chrominance block of a video, making a first determination that a luma block at a predefined position covering the chrominance block is encoded using a non-normal luma intra-frame prediction mode. Method 2740 also includes, at step 2744, based on the first determination, making a second determination to apply the predefined intra-frame prediction mode to the chrominance block because the chrominance block is encoded using a non-normal luma intra-frame prediction mode. Method 2740 also includes, at step 2746, performing a conversion between the video and a codec representation of the video based on the second determination. In some implementations, the non-normal luma intra-frame prediction mode includes encoding the luma block without using neighborhood pixel values ​​extrapolated along a luma prediction direction.

[0876] Fig.27E A flow chart of an exemplary method of video processing is shown. Method 2750 includes, at step 2752, for a chroma block of the video, making a first determination that a luma block at a predefined position covering the chroma block is encoded using a normal luma intra prediction mode. Method 2750 also includes, at step 2754, based on the first determination, making a second determination to derive a chroma intra prediction mode based on a normal luma intra prediction mode of the luma block. Method 2750 also includes, at step 2756, performing a conversion between the video and a coded representation of the video based on the second determination, wherein the normal luma intra prediction mode includes encoding the luma block using neighborhood pixel values ​​extrapolated along a luma prediction direction.

[0877] Fig.23A 2300. The device 2300 may be used to implement one or more of the methods described herein. The device 2300 may be implemented as a smart phone, a tablet computer, an Internet of Things (IoT) receiver, etc. The device 2300 may include one or more processors 2302, one or more memories 2304, and video processing hardware 2306. The (one or more) processors 2302 may be configured to implement one or more methods described in this document (including but not limited to methods 2200 to 2750). The (one or more) memories 2304 may be used to store data and code, which is used to implement the methods and techniques described herein. The video processing hardware 2306 may be used to implement some of the techniques described in this document in hardware circuits. In some embodiments, the hardware 2306 may be at least partially located in the processor 2302, such as a graphics co-processor.

[0878] Fig. 23B is another example of a block diagram of a video processing system in which the disclosed technology may be implemented. Fig. 23B2400. The system 2400 may include an input 2402 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be received in a compressed or encoded format. The input 2402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0879] System 2400 may include a codec component 2404 that can implement various codecs or encoding methods described in this document. Codec component 2404 can reduce the average bit rate of the video from input 2402 to the output of codec component 2404 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 2404 can be stored or sent via the connected communication, as represented by component 2406. The bitstream (or codec) representation of the storage or communication of the video received at input 2402 can be used by component 2408 to generate pixel values ​​or displayable video sent to display interface 2410. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations of the reverse codec results will be performed by the decoder.

[0880] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be implemented in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0881] In some embodiments, the video processing methods discussed in this patent document may use Fig.23A or a device implemented on the hardware platform described in 23B.

[0882] Some embodiments of the technology of the present disclosure include deciding or determining to enable a video processing tool or mode. In an example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of the video block, but it is not necessary to modify the resulting bitstream based on the use of the tool or mode. In other words, when a video processing tool or mode is enabled based on a decision or determination, the conversion of the bitstream from the video block to the video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream using the knowledge that the bitstream has been modified based on the video processing tool or mode. In other words, the conversion from the bitstream of the video to the video block will be performed using the video processing tool or mode enabled based on the decision or determination.

[0883] Some embodiments of the disclosed techniques include deciding or determining to disable a video processing tool or mode. In an example, when the video processing tool or mode is disabled, the encoder will not use the tool or mode in the conversion of video blocks to a bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream with the knowledge that the bitstream has not been modified using the video processing tool or mode that was decided or determined to be disabled.

[0884] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. A bitstream representation or a codec representation of a current video block may correspond, for example, to co-located or spread bits in different positions within the bitstream, as defined by the syntax. For example, a macroblock may be encoded based on transformed and coded error residual values ​​and also using bits in the header and other fields in the bitstream.

[0885] Various techniques and embodiments may be described using the following clause-based format: The first set of clauses describes certain features and aspects of the techniques disclosed in the previous sections.

[0886] 1. A method of video processing, comprising: selecting a transform set or transform matrix to apply a reduced secondary transform to the current video block based on characteristics of the current video block; and applying the selected transform set or transform matrix to a portion of the current video block as part of a conversion between the current video block and a bitstream representation of a video including the current video block.

[0887] 2. The method of clause 1, wherein the portion of the current video block is an upper right sub-region, a lower right sub-region, a lower left sub-region, or a middle sub-region of the current video block.

[0888] 3. The method of clause 1 or 2, wherein the characteristic of the current video block is an intra prediction mode or a primary transform matrix of the current video block.

[0889] 4. The method of clause 1, wherein the characteristic is a color component of the current video block.

[0890] 5. The method of clause 4, wherein a first transform set is selected for a luma component of the current video block, and wherein a second transform set different from the first transform set is selected for one or more chroma components of the current video block.

[0891] 6. The method of clause 1, wherein the characteristic is an intra prediction mode or an intra coding method of the current video block.

[0892] 7. The method of clause 6, wherein the intra prediction method comprises a multiple reference line (MRL) based prediction method or a matrix based intra prediction method.

[0893] 8. The method of clause 6, wherein a first transform set is selected when the current video block is a cross-component linear model (CCLM) coded block, and wherein a second transform set different from the first transform set is selected when the current video block is a non-CCLM coded block.

[0894] 9. A method as described in clause 6, wherein, when the current video block is encoded and decoded using a joint chroma residual codec method, a first transform set is selected, and wherein, when the current video block is not encoded and decoded using a joint chroma residual codec method, a second transform set different from the first transform set is selected.

[0895] 10. The method of clause 1, wherein the characteristic is a transform of the current video block.

[0896] 11. A method of video processing, comprising: making a decision based on one or more coefficients associated with a current video block, the decision regarding selectively including signaling side information to apply a reduced secondary transform (RST) to a bitstream representation of the current video block; and performing a conversion between the current video block and a video including the bitstream representation of the current video block based on the decision.

[0897] 12. The method of clause 11, wherein the one or more coefficients comprise a last non-zero coefficient in a scan order for the current video block.

[0898] 13. The method of clause 11, wherein the one or more coefficients include a plurality of coefficients within a partial region of the current video block.

[0899] 14. The method of clause 13, wherein the partial region includes one or more codec groups to which RST is applicable.

[0900] 15. The method of clause 13, wherein the partial region comprises the first M codec groups or the last M codec groups in a scanning order of the current video block.

[0901] 16. The method of clause 13, wherein the partial region comprises the first M codec groups or the last M codec groups in a reverse scan order of the current video block.

[0902] 17. The method of clause 13, wherein the determination is further made based on an energy of one or more non-zero coefficients in the plurality of coefficients.

[0903] 18. A method for video processing, comprising: configuring a bitstream representation of a current video block to apply a reduced secondary transform (RST) to the current video block, wherein syntax elements related to the RST are signaled in the bitstream representation before encoding and decoding residual information; and performing conversion between the current video block and the bitstream representation of the current video block based on the configuration.

[0904] 19. The method of clause 18, wherein signaling of RST-related syntax elements is based on usage of at least one codec block flag or a transform selection mode.

[0905] 20. The method of clause 18, wherein the bitstream representation excludes codec residual information corresponding to codec groups having all zero coefficients.

[0906] 21. The method of clause 18, wherein the coding / decoding residual information is based on application of RST.

[0907] 22. A method for video processing, comprising: configuring a bitstream representation of a current video block to apply a reduced secondary transform (RST) to the current video block, wherein a syntax element related to the RST is signaled in the bitstream representation before a transform skip indication or a multiple transform set (MTS) index; and performing conversion between the current video block and the bitstream representation of the current video block based on the configuration.

[0908] 23. The method of clause 22, wherein the transform skip indication or the MTS index is based on RST-related syntax elements.

[0909] 24. A method of video processing, comprising: configuring a context model based on characteristics of a current video block to encode and decode an index of a reduced secondary transform (RST); and performing conversion between the current video block and a bitstream representation of a video including the current video block based on the configuration.

[0910] 25. The method of clause 24, wherein the characteristic is to explicitly or implicitly enable a multiple transform selection (MTS) process.

[0911] 26. The method of clause 24, wherein the characteristic is enabling a cross-component linear model (CCLM) codec mode in the current video block.

[0912] 27. The method of clause 24, wherein the characteristic is a size of the current video block.

[0913] 28. The method of clause 24, wherein the characteristic is a partition depth of a segmentation process applied to the current video block.

[0914] 29. The method of clause 28, wherein the partitioning process is a quadtree (QT) partitioning process, a binary tree (BT) partitioning process, or a ternary tree (TT) partitioning process.

[0915] 30. The method of clause 24, wherein the characteristic is a color format or a color component of the current video block.

[0916] 31. The method of clause 24, wherein the characteristics exclude an intra prediction mode and an index of a multiple transform selection (MTS) process for the current video block.

[0917] 32. A method of video processing, comprising: making a decision based on characteristics of a current video block, the decision being about selectively applying a reverse downscaled secondary transform (RST) to the current video block; and performing a conversion between the current video block and a bitstream representation of a video including the current video block based on the decision.

[0918] 33. The method of clause 32, wherein the characteristic is a codec block flag of a codec group of the current video block.

[0919] 34. The method of clause 33, wherein a reverse RST process is not applied, and wherein the codec block flag of the top left codec group is zero.

[0920] 35. The method of clause 33, wherein a reverse RST process is not applied, and wherein the codec block flags of the first and second codec groups in the scan order of the current video block are zero.

[0921] 36. The method of clause 32, wherein the characteristic is a height (M) or a width (N) of the current video block.

[0922] 37. The method of clause 36, wherein a reverse RST process is not applied, and wherein (i) M=8 and N=4, or (ii) M=4 and N=8.

[0923] 38. A method of video processing, comprising: making a decision based on characteristics of a current video block, the decision being about selectively applying a reverse downscaled secondary transform (RST) to the current video block; and performing a conversion between the current video block and a bitstream representation of a video including the current video block based on the decision; wherein the bitstream representation includes side information about the RST, wherein the side information is included based on a single color component or a luminance component of the current video block.

[0924] 39. The method of clause 38, wherein the side information is included further based on a dimension of the current video block.

[0925] 40. The method of any of clauses 38 or 39, wherein the side information is included without regard to block information of the current video block.

[0926] 41. A method as described in any of clauses 1 to 40, wherein converting comprises generating a bitstream representation from the current video block.

[0927] 42. A method as described in any of clauses 1 to 40, wherein converting includes generating the current video block from a bitstream representation.

[0928] 43. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any of clauses 1 to 42.

[0929] 44. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for implementing the method of any one of clauses 1 to 42.

[0930] The second group of items describes certain features and aspects of the technology disclosed in the previous sections, such as Example Items 4 and 5.

[0931] 1. A method for video processing, comprising: for conversion between a current video block of a video unit of a video and a coded representation of the video, determining a default intra-frame prediction mode for a video unit coded and decoded using a certain intra-frame prediction mode, so that a prediction block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction; and performing conversion based on the determination.

[0932] 2. The method of clause 1, wherein the video unit corresponds to a codec unit, a prediction unit, a codec block, or a prediction block.

[0933] 3. The method of clause 1, wherein the video unit is encoded and decoded using matrix-based intra prediction (MIP), wherein the matrix-based intra prediction generates the prediction block using matrix-vector multiplication.

[0934] 4. The method of clause 1, wherein the video unit is encoded and decoded using an intra block copy (IBC) mode, the intra block copy mode generating the prediction block using at least a block vector pointing to a video frame containing the current video block.

[0935] 5. The method of clause 1, wherein the video unit is encoded or decoded using a palette mode that allows the current video block to be represented or reconstructed using a palette of representative sample values.

[0936] 6. The method of clause 1, wherein the default intra prediction mode is determined based on a codec mode of the current video block.

[0937] 7. A method as described in clause 1, wherein information about the default intra-frame prediction mode is included in the codec representation.

[0938] 8. The method of clause 1, wherein information about the default intra prediction mode is derived rather than signaled.

[0939] 9. The method of clause 1, wherein the determining the default intra prediction mode is also used in the derivation of a chroma derivation mode (DM).

[0940] 10. The method of clause 1, wherein the determining the default intra-frame prediction mode is also used to predict the intra-frame prediction mode of other blocks of the video.

[0941] 11. The method of clause 1, wherein the determining a default intra prediction mode assigned to the current video block of one color component is used in the derivation of a transform set or transform index for another color component.

[0942] 12. The method of clause 1, wherein the default intra prediction mode is stored with prediction modes for blocks of the video.

[0943] 13. The method of clause 1, wherein the default intra prediction mode is not assigned to an inter-codec block.

[0944] 14. The method of clause 1, wherein the default intra prediction mode is a planar intra prediction mode.

[0945] 15. A method for video processing, comprising: using a rule to determine a predetermined position where a luminance block of a video covers a chrominance block of the video; and performing a conversion between the video and a codec representation of the video based on the determination, wherein the chrominance block is represented by the codec representation using an intra-frame prediction mode.

[0946] 16. The method of clause 15, wherein performing the converting comprises checking a prediction mode or a codec mode of the luma block and extracting an intra prediction mode of the luma block.

[0947] 17. The method of clause 15, wherein the luma block comprises luma samples positioned in correspondence with chroma samples at a middle of the chroma block.

[0948] 18. The method of clause 15, wherein the luma block comprises luma samples positioned in correspondence with chroma samples at an upper left of the chroma block.

[0949] 19. A method for video processing, comprising: using rules to derive an intra-frame prediction mode of a chrominance block of a video based on a codec mode of a luminance block corresponding to the chrominance block; and performing conversion between the luminance block and a codec representation of the video based on the derived intra-frame prediction mode, and wherein the rules specify the use of a default intra-frame prediction mode in case the codec mode of the luminance block is a certain intra-frame prediction mode, wherein a prediction block of the luminance block is generated without extrapolating neighboring pixels of the luminance block along a direction.

[0950] 20. The method of clause 19, wherein the rule specifies the use of a decoded intra prediction mode in case the encoding / decoding mode of the luma block is not the certain intra prediction mode.

[0951] 21. The method of clause 19, wherein the coding mode of the luma block corresponds to matrix-based intra prediction (MIP), the matrix-based intra prediction using matrix-vector multiplication to generate the prediction block.

[0952] 22. The method of clause 21, wherein the decoded intra prediction mode is a planar intra prediction mode.

[0953] 23. The method of clause 19, wherein the coding mode of the luma block corresponds to an intra block copy (IBC) mode, the intra block copy mode generating the prediction block using at least a block vector pointing to a video frame containing a current video block.

[0954] 24. The method of clause 19, wherein the coding mode of the luma block corresponds to a palette mode that allows a current video block to be represented or reconstructed using a palette of representative sample values.

[0955] 25. The method of clause 23 or 24, wherein the decoded intra prediction mode is a DC intra prediction mode.

[0956] 26. A method as described in any of clauses 1 to 25, wherein performing the conversion includes applying a secondary transform tool to the current video block, and wherein the secondary transform tool is used: during encoding, a forward secondary transform is applied to the output of a forward primary transform applied to the residual of the current video block before quantization, or during decoding, an inverse secondary transform is applied to the dequantized output of the current video block before applying an inverse primary transform.

[0957] 27. A method as described in any of clauses 1 to 26, wherein performing the conversion includes generating the codec representation from the video.

[0958] 28. A method as described in any of clauses 1 to 26, wherein performing the conversion includes generating the video from the codec representation.

[0959] 29. A video processing device comprising a processor configured to implement the method described in any one or more of clauses 1 to 28.

[0960] 30. A computer readable medium storing program code which, when executed, causes a processor to implement the method of any one or more of clauses 1 to 28.

[0961] The third group of items describes certain features and aspects of the technology disclosed in the previous sections, such as Example Items 5 and 6.

[0962] 1. A method for video processing, comprising: for a chroma block of a video, making a first determination whether to apply a non-normal chroma intra-frame prediction mode to the chroma block of the video; for a luminance block corresponding to the chroma block, making a second determination to apply the luminance intra-frame prediction mode to the luminance block; making a third determination to apply a transform set or a transform matrix to the chroma block based on the luminance intra-frame prediction mode; and performing conversion between a video and a coded representation of the video according to the third determination, and wherein the non-normal chroma intra-frame prediction mode includes encoding and decoding the chroma block without using neighborhood pixel values ​​extrapolated along a chroma prediction direction.

[0963] 2. A method as described in clause 1, wherein the transform set or transform matrix is ​​used in a secondary transform tool applied to the chrominance blocks.

[0964] 3. A method as described in clause 2, wherein the secondary transform tool corresponds to a low frequency non-separable transform (LFNST) tool, and wherein the secondary transform tool is used: during encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the chroma block before quantization, or during decoding, an inverse secondary transform is applied to the dequantized output of the chroma block before applying the inverse primary transform.

[0965] 4. The method of clause 1, wherein the non-normal chroma intra prediction mode corresponds to a cross-component linear model (CCLM) prediction mode that uses a linear model to derive a prediction value for a chroma component from another component.

[0966] 5. The method of clause 1, wherein the luma block covers luma samples corresponding to chroma samples located at predefined positions of the chroma blocks.

[0967] 6. The method of clause 1, wherein the luma block covers luma samples corresponding to middle chroma samples of the chroma blocks.

[0968] 7. The method of clause 1, wherein the luma block overlays luma samples corresponding to top-left chroma samples of the current video block.

[0969] 8. The method of clause 1, wherein the luma intra prediction is a non-normal luma intra prediction mode, and wherein the non-normal luma intra prediction mode comprises encoding and decoding the luma block without using neighboring pixel values ​​extrapolated along a luma prediction direction.

[0970] 9. The method of clause 8, wherein the non-normal luma intra prediction mode corresponds to matrix-based intra prediction (MIP) that generates prediction values ​​using matrix-vector multiplication.

[0971] 10. The method of clause 8, wherein the non-normal luma intra prediction mode corresponds to an intra block copy (IBC) mode that generates prediction values ​​using at least a block vector pointing to a video frame containing a luma block.

[0972] 11. The method of clause 8, wherein the non-normal luma intra prediction mode corresponds to a palette mode that allows a luma block to be represented or reconstructed using a palette of representative sample values.

[0973] 12. A method as described in any of clauses 8 to 11, wherein, in response to the luma intra-frame prediction mode being an abnormal luma intra-frame prediction mode, a predefined intra-frame prediction mode is assigned to the chroma block, and wherein, instead of the abnormal chroma intra-frame prediction mode, the predefined intra-frame prediction mode is also used to derive a transform set or transform matrix applied to the chroma block.

[0974] 13. The method of clause 12, wherein the predefined intra prediction mode is determined based on a non-normal luma intra prediction mode of the luma block.

[0975] 14. The method of clause 13, wherein, in response to the non-normal luma intra prediction mode being an IBC mode or a palette mode, the predefined intra prediction mode is determined to be a DC mode.

[0976] 15. The method of clause 13, wherein, in response to the non-normal luma intra prediction mode being a MIP mode, the predefined intra prediction mode is determined to be a planar mode.

[0977] 16. The method of clause 1, wherein the luma intra prediction is a normal luma intra prediction mode, and wherein the normal luma intra prediction mode comprises encoding and decoding the luma block using neighborhood pixel values ​​extrapolated along a luma prediction direction.

[0978] 17. A method as described in clause 8, wherein, in response to the luma intra prediction mode being a normal luma intra prediction mode, the luma intra prediction is assigned to the chroma block, and wherein, instead of the non-normal chroma intra prediction mode, the luma intra prediction mode is also used to derive a transform set or transform matrix applied to the chroma block.

[0979] 18. A method for video processing, comprising: for a chroma block of a video, making a first determination that a luminance block corresponding to the chroma block is encoded and decoded using an abnormal luminance intra-frame prediction mode; based on the first determination, making a second determination of a transform set or a transform matrix of the chroma block according to a rule; and performing conversion between a video and a coded representation of the video according to the second determination, and wherein the rule specifies one or more default modes or default transform sets associated with the chroma block because the luminance block is encoded and decoded using an abnormal luminance intra-frame prediction mode to determine the transform set or the transform matrix in case the chroma block is encoded and decoded using an abnormal chroma intra-frame prediction mode, wherein the abnormal luminance intra-frame prediction mode includes encoding and decoding the luminance block without using neighborhood pixel values ​​extrapolated along a chroma prediction direction.

[0980] 19. The method of clause 18, wherein the non-normal chroma intra prediction mode corresponds to a cross-component linear model (CCLM) prediction mode that uses a linear model to derive a predicted value for a chroma component from another component.

[0981] 20. The method of clause 18, wherein the non-normal luma intra prediction mode corresponds to matrix-based intra prediction (MIP) that generates prediction values ​​using matrix-vector multiplication.

[0982] 21. The method of clause 18, wherein the non-normal luma intra prediction mode corresponds to an intra block copy (IBC) mode that generates prediction values ​​using at least a block vector pointing to a video frame containing a luma block.

[0983] 22. The method of clause 18, wherein the non-normal luma intra prediction mode corresponds to a palette mode that allows a luma block to be represented or reconstructed using a palette of representative sample values.

[0984] 23. The method of clause 18, wherein the luma block covers luma samples corresponding to chroma samples located at predefined positions of the chroma blocks.

[0985] 24. The method of clause 18, wherein the luma block covers luma samples corresponding to middle chroma samples of the chroma blocks.

[0986] 25. The method of clause 18, wherein the luma block overlays luma samples corresponding to top-left chroma samples of the current video block.

[0987] 26. The method of clause 18, wherein the rule specifies whether to use at least one of a default transform set or a default transform matrix or to derive the transform set or transform matrix from an intra luma prediction mode of the luma block based on a codec mode of the luma block.

[0988] 27. A method as described in clause 18, wherein the rule specifies that a transform set or transform matrix is ​​derived based on an intra-frame luma prediction mode of a luma block, because the luma block is encoded using a normal intra-frame prediction mode that generates a prediction block for the luma block by extrapolating neighboring pixels of the luma block.

[0989] 28. The method of clause 18, wherein the rule specifies that the transform set or transform matrix is ​​derived based on an intra luma prediction mode of the luma block because the luma block is encoded using a Block Delta Pulse Codec Modulation (BDPCM) mode.

[0990] 29. The method of clause 26, wherein the rule specifies use of at least one of a default transform set or a default transform matrix because the luma block is encoded using a non-normal luma intra prediction mode.

[0991] 30. A method of video processing, comprising: for conversion between a current video block of a video and a codec representation of the video, determining the suitability of a secondary transform tool applied to the current video block of a color component based on at least one of: 1) a codec mode of a corresponding block of another color component, or 2) a codec mode of the current video block; and performing the conversion based on the determination, and wherein the secondary transform tool is used: during encoding, a forward secondary transform is applied to the output of a forward primary transform applied to a residue of the current video block before quantization, or during decoding, an inverse secondary transform is applied to a dequantized output of the current video block before applying an inverse primary transform.

[0992] 31. The method of clause 30, wherein the quadratic transform tool corresponds to a low frequency non-separable transform (LFNST) tool.

[0993] 32. The method of clause 30, wherein the current video block corresponds to a chrominance block, and the corresponding block corresponds to a luma block corresponding to the chrominance block.

[0994] 33. A method as described in clause 31, wherein it is determined that the secondary transform tool is applied to the current video block because the coding mode of the corresponding block is i) a normal intra-frame prediction mode, which includes encoding the corresponding block using neighborhood pixel values ​​extrapolated along a prediction direction, and / or ii) a block-based delta pulse codec modulation (BDPCM) mode.

[0995] 34. A method as described in clause 31, wherein it is determined that the secondary transform tool is disabled for the current video block because the coding mode of the corresponding block is an abnormal intra-frame prediction mode, which includes encoding the corresponding block without using neighboring pixel values ​​extrapolated along the prediction direction.

[0996] 35. The method of clause 30, wherein the coding mode of the current video block corresponds to a cross-component linear model (CCLM) prediction mode that uses a linear model to derive a predicted value for a chroma component from another component.

[0997] 36. A method for video processing, comprising: making a first determination for a chrominance block of a video to encode a luminance block at a predefined position covering the chrominance block using an abnormal luminance intra-frame prediction mode; based on the first determination, making a second determination to apply a predefined intra-frame prediction mode to the luminance block because the luminance block is encoded using an abnormal luminance intra-frame prediction mode; and performing conversion of the video to a codec representation of the video based on the second determination, wherein the abnormal luminance intra-frame prediction mode includes encoding the luminance block without using neighborhood pixel values ​​extrapolated along a luminance prediction direction.

[0998] 37. The method of clause 36, wherein a transform set or transform matrix used in a secondary transform tool applied to a chroma block is selected based on a predefined intra prediction mode.

[0999] 38. A method as described in clause 36, wherein the secondary transform tool corresponds to a low frequency non-separable transform (LFNST) tool, and wherein the secondary transform tool is used: during encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the chroma block before quantization, or during decoding, an inverse secondary transform is applied to the dequantized output of the chroma block before applying the inverse primary transform.

[1000] 39. A method as described in any of clauses 36 to 38, wherein the predefined intra prediction mode applied to the chroma block depends on a non-normal luma intra prediction mode of the luma block.

[1001] 40. The method of any of clauses 36 to 38, wherein the predefined intra prediction mode corresponds to at least one of the following modes: DC mode, planar mode, vertical mode, horizontal mode, 45 degree mode or 135 degree mode.

[1002] 41. A method as described in any of clauses 36 to 38, wherein the abnormal luma intra prediction mode corresponds to at least one of the following modes: intra block copy (IBC) mode, palette mode, matrix-based intra prediction (MIP), or block-based delta pulse codec modulation (BDPCM) mode.

[1003] 42. A method as described in any of clauses 36 to 38, wherein the abnormal luma intra prediction mode corresponds to a matrix-based intra prediction (MIP) mode, and the MIP mode is mapped to a specified intra prediction mode based on the MIP mode and the dimension of the luma block.

[1004] 43. The method of any of clauses 36 to 38, wherein a predefined intra-prediction mode is selected among the candidates.

[1005] 44. A method as described in any of clauses 36 to 38, wherein the signaling signals a predefined intra-frame prediction mode.

[1006] 45. A method as described in any of clauses 36 to 38, wherein a predefined intra-frame prediction mode is derived.

[1007] 46. ​​A method for video processing, comprising: for a chrominance block of a video, making a first determination to encode a luminance block at a predefined position covering the chrominance block using a normal luminance intra-frame prediction mode; based on the first determination, making a second determination to derive a chrominance intra-frame prediction mode based on the normal luminance intra-frame prediction mode of the luminance block; and performing conversion between a video and a codec representation of the video according to the second determination, wherein the normal luminance intra-frame prediction mode includes encoding the luminance block using neighborhood pixel values ​​extrapolated along a luminance prediction direction.

[1008] 47. The method of clause 46, wherein a transform set or transform matrix used in a secondary transform tool applied to a chroma block is selected based on a chroma intra prediction mode.

[1009] 48. A method as described in clause 46, wherein the secondary transform tool corresponds to a low frequency non-separable transform (LFNST) tool, and wherein the secondary transform tool is used: during encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the current video block before quantization, or during decoding, an inverse secondary transform is applied to the dequantized output of the current video block before applying the inverse primary transform.

[1010] 49. A method as described in any of clauses 46 to 48, wherein the luma block comprises luma samples located in correspondence with chroma samples at the middle of the chroma blocks.

[1011] 50. A method as described in any of clauses 46 to 48, wherein the luma block comprises luma samples positioned corresponding to chroma samples at the top left of the chroma block.

[1012] 51. A method as described in any of clauses 1 to 50, wherein performing the conversion includes generating a codec representation from the video or generating the video from the codec representation.

[1013] 52. A video processing device comprising a processor configured to implement the method described in any one or more of clauses 1 to 51.

[1014] 53. A computer readable medium storing program code which, when executed, causes a processor to implement the method of any one or more of clauses 1 to 51.

[1015] From the foregoing, it will be appreciated that specific embodiments of the disclosed technology have been described for purposes of illustration, but that various modifications may be made without departing from the scope of the invention. Accordingly, the disclosed technology is not limited as claimed in the appended claims.

[1016] The implementation and functional operation of the subject matter described in this patent document can be implemented in various systems, digital electronic circuits or in computer software, firmware or hardware, including the structures disclosed in this specification and the equivalents of their structures, or a combination of one or more of them. The implementation of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium, for execution by a data processing device or for controlling the operation of a data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composite that affects a machine-readable propagable signal, or a combination of one or more thereof. The term "data processing unit" or "data processing device" covers all devices, devices and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device can also include code that creates an execution environment for the computer program in question, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[1017] A computer program (also referred to as a program, software, software application, script or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including stand-alone programs or modules, components, subroutines or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that stores other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple collaborative files (e.g., files storing one or more modules, subroutines or code portions). A computer program can be deployed to execute on one computer or on multiple computers, which are located at one site or distributed across multiple sites and interconnected by a communication network.

[1018] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be implemented as, special purpose logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[1019] Processors suitable for the execution of computer programs include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (such as magnetic, magneto-optical disks or optical disks) for storing data, or be operably coupled to receive data from a mass storage device (such as magnetic, magneto-optical disks or optical disks) or transfer data to a mass storage device (such as magnetic, magneto-optical disks or optical disks), or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM and flash memory devices). The processor and memory can be supplemented by a dedicated logic circuit, or incorporated in a dedicated logic circuit.

[1020] Instructions and Appendix Figure 1 The above are considered to be exemplary only, where exemplary means example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[1021] Although this patent document contains many details, these details should not be interpreted as limitations on any invention or the scope of protection that can be claimed, but as descriptions of features specified for specific embodiments of specific inventions. In this patent document, certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments, respectively, or in various suitable sub-combinations. In addition, although features may be described as working in certain combinations as described above and even initially claimed in the same manner, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[1022] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, in order to achieve the desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1023] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: During conversion between a current chroma block of a video unit of a video and a bitstream of the video, determining whether a codec mode of a luma block covering luma samples at positions corresponding only to chroma samples at a middle portion of the current chroma block is a specified codec mode; For the derivation process of the chroma intra prediction mode of the current chroma block, deriving a variable denoted as lumaIntraPredMode based on the determination, wherein the variable denoted as lumaIntraPredMode is equal to a default intra prediction mode when the luma block is encoded and decoded using the specified codec mode, and the variable denoted as lumaIntraPredMode is equal to a luma intra prediction mode of the luma block when the luma block is not encoded and decoded using the specified codec mode; deriving the chroma intra prediction mode for the current chroma block, wherein the chroma intra prediction mode is derived using a first flag ccm_mode_flag indicating whether a cross-component linear model prediction mode is applied to the current chroma block, a second flag ccm_mode_idx indicating which of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM is applied to the current chroma block, a third flag intra_chroma_pred_mode indicating an intra prediction mode for chroma samples of the current chroma block, and the variable denoted as lumaIntraPredMode; and performing the converting based on the chroma intra prediction mode; Wherein, when the current chroma block is encoded and decoded using the cross-component linear model prediction mode, when the encoding and decoding mode of the luminance block is the specified encoding and decoding mode, the default intra-frame prediction mode is adopted in the secondary transformation process of the current chroma block, wherein the transform set used in the secondary transformation process is determined based on the default intra-frame prediction mode; wherein, in a case where the designated coding mode of the luma block is a matrix-based intra prediction (MIP) mode, the default intra prediction mode is determined to be a planar intra prediction mode, wherein the matrix-based intra prediction mode uses matrix-vector multiplication to generate prediction samples of the luma block; wherein, in a case where the designated coding mode of the luma block is an intra block copy (IBC) mode, the default intra prediction mode is determined to be a DC intra prediction mode, wherein the intra block copy mode derives prediction samples of the luma block from a block of sample values ​​of the same video region determined by a block vector; Wherein, when the designated encoding and decoding mode of the luminance block is a palette mode, the default intra-frame prediction mode is determined to be a DC intra-frame prediction mode, wherein the palette mode allows the luminance block to be represented or reconstructed using a palette of representative sample values.

2. The method according to claim 1, wherein: When the specified coding mode of the luma block is neither the matrix-based intra prediction (MIP) mode, the intra block copy (IBC) mode, nor the palette mode, the luma intra prediction mode of the luma block is utilized in the derivation of the chroma intra prediction mode for the current chroma block.

3. The method according to claim 1, wherein: The chroma intra prediction mode is derived using cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode specified in Table 1 below; Table 1 。 4. The method according to claim 3, wherein: When the chroma format of the video unit is 4:2:2, the chroma intra prediction mode X in Table 1 is mapped to the chroma intra prediction mode Y according to a predefined mapping table, and then the chroma intra prediction mode X in Table 1 is set equal to the chroma intra prediction mode Y.

5. The method according to claim 1, wherein: The default intra prediction mode is not allocated to the inter codec block.

6. The method according to claim 1, wherein: The converting includes encoding the current chroma block into the bitstream.

7. The method according to claim 1, wherein: The converting includes decoding the current chroma block from the bitstream.

8. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: During conversion between a current chroma block of a video unit of a video and a bitstream of the video, determining whether a codec mode of a luma block covering luma samples at positions corresponding only to chroma samples at a middle portion of the current chroma block is a specified codec mode; For the derivation process of the chroma intra prediction mode of the current chroma block, a variable denoted as lumaIntraPredMode is derived based on the determination, wherein: In the case where the luma block is encoded and decoded using the specified codec mode, the variable denoted as lumaIntraPredMode is equal to a default intra prediction mode, and in the case where the luma block is not encoded and decoded using the specified codec mode, the variable denoted as lumaIntraPredMode is equal to a luma intra prediction mode of the luma block; deriving the chroma intra prediction mode for the current chroma block, wherein the chroma intra prediction mode is derived using a first flag ccm_mode_flag indicating whether a cross-component linear model prediction mode is applied to the current chroma block, a second flag ccm_mode_idx indicating which of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM is applied to the current chroma block, a third flag intra_chroma_pred_mode indicating an intra prediction mode for chroma samples of the current chroma block, and the variable denoted as lumaIntraPredMode; and performing the converting based on the chroma intra prediction mode; Wherein, when the current chroma block is encoded and decoded using the cross-component linear model prediction mode, when the encoding and decoding mode of the luminance block is the specified encoding and decoding mode, the default intra-frame prediction mode is adopted in the secondary transformation process of the current chroma block, wherein the transform set used in the secondary transformation process is determined based on the default intra-frame prediction mode; wherein, in a case where the designated coding mode of the luma block is a matrix-based intra prediction (MIP) mode, the default intra prediction mode is determined to be a planar intra prediction mode, wherein the matrix-based intra prediction mode uses matrix-vector multiplication to generate prediction samples of the luma block; wherein, in a case where the designated coding mode of the luma block is an intra block copy (IBC) mode, the default intra prediction mode is determined to be a DC intra prediction mode, wherein the intra block copy mode derives prediction samples of the luma block from a block of sample values ​​of the same video region determined by a block vector; Wherein, when the designated encoding and decoding mode of the luminance block is a palette mode, the default intra-frame prediction mode is determined to be a DC intra-frame prediction mode, wherein the palette mode allows the luminance block to be represented or reconstructed using a palette of representative sample values.

9. The device according to claim 8, wherein: When the specified coding mode of the luma block is neither the matrix-based intra prediction (MIP) mode, the intra block copy (IBC) mode, nor the palette mode, the luma intra prediction mode of the luma block is utilized in the derivation of the chroma intra prediction mode for the current chroma block.

10. The device according to claim 8, wherein: The chroma intra prediction mode is derived using ccm_mode_flag, ccm_mode_idx, intra_chroma_pred_mode, and lumaIntraPredMode as specified in Table 1 below: Table 1 Wherein, when the chroma format of the video unit is 4:2:2, the chroma intra prediction mode X in Table 1 is mapped to the chroma intra prediction mode Y according to a predefined mapping table, and then the chroma intra prediction mode X in Table 1 is set equal to the chroma intra prediction mode Y; and The default intra-frame prediction mode is not allocated to the inter-frame coding and decoding block.

11. A non-transitory computer-readable storage medium storing instructions that cause a processor to: During conversion between a current chroma block of a video unit of a video and a bitstream of the video, determining whether a codec mode of a luma block covering luma samples at positions corresponding only to chroma samples at a middle portion of the current chroma block is a specified codec mode; For the derivation process of the chroma intra prediction mode of the current chroma block, a variable denoted as lumaIntraPredMode is derived based on the determination, wherein: In the case where the luma block is encoded and decoded using the specified codec mode, the variable denoted as lumaIntraPredMode is equal to a default intra prediction mode, and in the case where the luma block is not encoded and decoded using the specified codec mode, the variable denoted as lumaIntraPredMode is equal to a luma intra prediction mode of the luma block; deriving the chroma intra prediction mode for the current chroma block, wherein the chroma intra prediction mode is derived using a first flag ccm_mode_flag indicating whether a cross-component linear model prediction mode is applied to the current chroma block, a second flag ccm_mode_idx indicating which of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM is applied to the current chroma block, a third flag intra_chroma_pred_mode indicating an intra prediction mode for chroma samples of the current chroma block, and the variable denoted as lumaIntraPredMode; and performing the converting based on the chroma intra prediction mode; Wherein, when the current chroma block is encoded and decoded using the cross-component linear model prediction mode, when the encoding and decoding mode of the luminance block is the specified encoding and decoding mode, the default intra-frame prediction mode is adopted in the secondary transformation process of the current chroma block, wherein the transform set used in the secondary transformation process is determined based on the default intra-frame prediction mode; wherein, in a case where the designated coding mode of the luma block is a matrix-based intra prediction (MIP) mode, the default intra prediction mode is determined to be a planar intra prediction mode, wherein the matrix-based intra prediction mode uses matrix-vector multiplication to generate prediction samples of the luma block; wherein, in a case where the designated coding mode of the luma block is an intra block copy (IBC) mode, the default intra prediction mode is determined to be a DC intra prediction mode, wherein the intra block copy mode derives prediction samples of the luma block from a block of sample values ​​of the same video region determined by a block vector; Wherein, when the designated encoding and decoding mode of the luminance block is a palette mode, the default intra-frame prediction mode is determined to be a DC intra-frame prediction mode, wherein the palette mode allows the luminance block to be represented or reconstructed using a palette of representative sample values.

12. The non-transitory computer-readable storage medium of claim 11, wherein: When the specified coding mode of the luma block is neither the matrix-based intra prediction (MIP) mode, the intra block copy (IBC) mode, nor the palette mode, the luma intra prediction mode of the luma block is utilized in the derivation of the chroma intra prediction mode for the current chroma block.

13. The non-transitory computer-readable storage medium of claim 11, wherein: The chroma intra prediction mode is derived using ccm_mode_flag, ccm_mode_idx, intra_chroma_pred_mode, and lumaIntraPredMode as specified in Table 1 below: Table 1 Wherein, when the chroma format of the video unit is 4:2:2, the chroma intra prediction mode X in Table 1 is mapped to the chroma intra prediction mode Y according to a predefined mapping table, and then the chroma intra prediction mode X in Table 1 is set equal to the chroma intra prediction mode Y; and The default intra-frame prediction mode is not allocated to the inter-frame coding and decoding block.

14. A method for storing a bit stream of a video, comprising: During conversion between a current chroma block of a video unit of a video and a bitstream of the video, determining whether a codec mode of a luma block covering luma samples at positions corresponding only to chroma samples at a middle portion of the current chroma block is a specified codec mode; For the derivation process of the chroma intra prediction mode of the current chroma block, deriving a variable denoted as lumaIntraPredMode based on the determination, wherein the variable denoted as lumaIntraPredMode is equal to a default intra prediction mode when the luma block is encoded and decoded using the specified codec mode, and the variable denoted as lumaIntraPredMode is equal to a luma intra prediction mode of the luma block when the luma block is not encoded and decoded using the specified codec mode; deriving the chroma intra prediction mode for the current chroma block, wherein the chroma intra prediction mode is derived using a first flag ccm_mode_flag, a second flag ccm_mode_idx, a third flag intra_chroma_pred_mode, and the variable denoted as lumaIntraPredMode, the first flag ccm_mode_flag indicating whether the cross-component linear model prediction mode is applied to the current chroma block, the second flag ccm_mode_idx indicating which of INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM is applied to the current chroma block, and the third flag intra_chroma_pred_mode indicating the intra prediction mode of the chroma samples of the current chroma block; generating the bitstream based on the chroma intra prediction mode; and storing the bitstream in a non-transitory computer-readable recording medium; Wherein, when the current chroma block is encoded and decoded using the cross-component linear model prediction mode, when the encoding and decoding mode of the luminance block is the specified encoding and decoding mode, the default intra-frame prediction mode is adopted in the secondary transformation process of the current chroma block, wherein the transform set used in the secondary transformation process is determined based on the default intra-frame prediction mode; wherein, in a case where the designated coding mode of the luma block is a matrix-based intra prediction (MIP) mode, the default intra prediction mode is determined to be a planar intra prediction mode, wherein the matrix-based intra prediction mode uses matrix-vector multiplication to generate prediction samples of the luma block; wherein, in a case where the designated coding mode of the luma block is an intra block copy (IBC) mode, the default intra prediction mode is determined to be a DC intra prediction mode, wherein the intra block copy mode derives prediction samples of the luma block from a block of sample values ​​of the same video region determined by a block vector; Wherein, when the designated encoding and decoding mode of the luminance block is a palette mode, the default intra-frame prediction mode is determined to be a DC intra-frame prediction mode, wherein the palette mode allows the luminance block to be represented or reconstructed using a palette of representative sample values.

15. The method according to claim 14, wherein: When the specified coding mode of the luma block is neither the matrix-based intra prediction (MIP) mode, the intra block copy (IBC) mode, nor the palette mode, the luma intra prediction mode of the luma block is utilized in the derivation of the chroma intra prediction mode for the current chroma block.

16. The method according to claim 14, wherein: The chroma intra prediction mode is derived using ccm_mode_flag, ccm_mode_idx, intra_chroma_pred_mode, and lumaIntraPredMode as specified in Table 1 below: Table 1 Wherein, when the chroma format of the video unit is 4:2:2, the chroma intra prediction mode X in Table 1 is mapped to the chroma intra prediction mode Y according to a predefined mapping table, and then the chroma intra prediction mode X in Table 1 is set equal to the chroma intra prediction mode Y; and The default intra-frame prediction mode is not allocated to the inter-frame coding and decoding block.

17. A video processing device, comprising a processor, wherein the processor is configured to implement the method according to any one of claims 4 to 7.

18. A computer readable medium storing program code, wherein when the program code is executed, the processor is caused to implement the method according to any one of claims 4 to 7.

Images