Video data processing method, device, and storage medium

Abstract

To provide a video processing method.SOLUTION: A method is related to video chroma blocking includes the steps of making first determination whether a non-normal chroma intra prediction mode is applied to a chroma block of video, making second determination that a luma intra prediction mode is applied to a luma block with respect to a luma block corresponding to the chroma block, making third decision in which a transform set or a transform matrix is applied to the chroma block on the basis of the luma intra prediction mode, and performing conversion between the video and a coded representation of the video according to the third determination, and the non-normal chroma intra prediction mode codes a chroma block without using values of neighboring pixels extrapolated along a chroma prediction direction.SELECTED DRAWING: Figure 27A

Description

[Background technology]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Under applicable patent laws and / or regulations relating to the Paris Convention, this application has been timely filed to claim priority to and the benefit of International Patent Application No. PCT / CN2019 / 099158, filed August 3, 2019. For all purposes under law, the entire disclosure of the aforementioned application is incorporated by reference as part of the disclosure of this application.

[0002] Technical Field This patent document relates to video processing techniques, devices and systems.

[0003] background Despite advances in video compression, digital video still accounts for the largest amount of bandwidth in the Internet and other digital communications networks. It is expected that the bandwidth demands for digital video usage will continue to increase as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention

[0004] The devices, systems, and methods relate to digital video coding, and in particular to context modeling for residual coding in video coding. The described methods may be applied to both existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards or video codecs.

[0005] In one exemplary aspect, the disclosed techniques can be used to provide a video processing method for transforming between a current video block of a video unit of a video and a coded representation of the video, the method including determining a default intra-prediction mode for a video unit coded using a particular intra-prediction mode such that a predictive block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction, and performing the transform based on the determination.

[0006] In another representative aspect, the disclosed techniques can be used to provide a video processing method that includes using the rule to make a decision about a luma block of the video covering a given location of a chroma block of the video, and based on the decision, performing a conversion between the video and a coded representation of the video, where the chroma block is represented in the coded representation using an intra prediction mode.

[0007] In yet another representative aspect, the disclosed techniques can be used to provide a video processing method, the method including deriving an intra-prediction mode for a chroma block of a video based on a coding mode of a luma block corresponding to the chroma block using a rule, and performing a conversion between the chroma block and a coded representation of the video based on the derived intra-prediction mode, the rule specifying using a default intra-prediction mode if the coding mode of the luma block is a particular intra-prediction mode in which a predictive block of the luma block is generated without extrapolating neighboring pixels of the luma block along a direction.

[0008] In yet another representative aspect, the disclosed techniques can be used to provide a video processing method, the method including: making, for a chroma block of a video, a first determination whether a non-normal chroma intra prediction mode is applied to the chroma block of the video; making, for a luma block corresponding to the chroma block, a second determination that a luma intra prediction mode is applied to the luma block; making a third determination that a transform set or transform matrix is ​​applied to the chroma block based on the luma intra prediction mode; and performing a transformation between the video and a coded representation of the video in accordance with the third determination, the non-normal chroma intra prediction mode including coding the chroma block without using values ​​of neighboring pixels extrapolated along a chroma prediction direction.

[0009] In yet another representative aspect, the disclosed techniques can be used to provide a video processing method, the method including the steps of: making a first determination, for a chroma block of a video, that a luma block corresponding to the chroma block is coded using a non-normal luma intra prediction mode; The method includes making a second determination of a transform set or transform matrix for the chroma block according to a rule based on the first determination; and performing a transform between the video and the coded representation of the video according to the second determination, the rule specifying that one or more default modes or default transform sets associated with the chroma block determine the transform set or transform matrix when the chroma block is coded using a non-normal chroma intra prediction mode due to the luma block being coded using a non-normal chroma intra prediction mode, the non-normal luma intra prediction mode including coding the luma block without using values ​​of neighboring pixels extrapolated along a luma prediction direction, the non-normal chroma intra prediction mode including coding the chroma block without using values ​​of neighboring pixels extrapolated along a chroma prediction direction.

[0010] In yet another representative aspect, the disclosed techniques can be used to provide a video processing method for converting between a current video block of a video and a coded representation of the video, the method including determining applicability of a second transform tool to be applied to a current video block of one color component based on at least one of 1) a coding mode of a corresponding block of another color component, or 2) a coding mode of the current video block, and performing the conversion based on the determination, using the secondary transform tool: during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block prior to quantization, or during decoding, an inverse secondary transform is applied to an output of an inverse quantization of the current video block prior to applying the inverse primary transform.

[0011] In yet another representative aspect, the disclosed techniques can be used to provide a video processing method, comprising: making a first determination for a chroma block of a video that a luma block covering a predetermined position of the chroma block is coded using a non-normal luma intra prediction mode, making a second determination based on the first determination, applying the predetermined intra prediction mode to the chroma block due to the luma block being coded using the non-normal luma intra prediction mode, and performing a conversion between the video and a coded representation of the video according to the second determination, wherein the non-normal luma intra prediction mode includes coding the luma block without using values ​​of neighboring pixels extrapolated along a luma prediction direction.

[0012] In yet another representative aspect, the disclosed techniques can be used to provide a video processing method, the method including: making a first decision for a chroma block of a video where a luma block covering a predetermined position of the chroma block is coded using a normal luma intra prediction mode, making a second decision based on the first decision to derive a chroma intra prediction mode based on the normal luma intra prediction mode of the luma block, and performing a conversion between the video and a coded representation of the video according to the second decision, the normal luma intra prediction mode including coding the luma block using values ​​of neighboring pixels extrapolated along a luma prediction direction.

[0013] In yet another exemplary embodiment, the above method is embodied in a form of code executable by a processor and stored on a computer readable program medium.

[0014] In yet another exemplary aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

[0015] In yet another exemplary embodiment, a video decoder device may implement the methods described herein.

[0016] These and other aspects and features of the disclosed technology are described in more detail in the drawings, specification and claims. [Brief description of the drawings]

[0017] [Figure 1] 1 shows a block diagram of an exemplary encoder.

[0018] [Diagram 2] 67 examples of intra prediction modes are shown.

[0019] [Diagram 3] An example of ALWIP for a 4x4 block is shown below.

[0020] [Figure 4] Here is an example of ALWIP for an 8x8 block.

[0021] [Diagram 5] An example of ALWIP for an 8x4 block is shown below.

[0022] [Figure 6] Here is an example of ALWIP for a 16x16 block.

[0023] [Figure 7] An example of four reference lines adjacent to a predicted block is shown.

[0024] [Figure 8] An example of division into 4x8 and 8x4 blocks is shown.

[0025] [Figure 9] Examples of block division for all blocks except 4x8, 8x4, and 4x4 are shown.

[0026] [Figure 10] Here is an example of a quadratic transformation in JEM.

[0027] [Figure 11] An example of the proposed reduced secondary transform (RST) is given below.

[0028] [Figure 12] Examples of forward and inverse contraction transformations are given below.

[0029] [Figure 13] Here is an example of a forward RST 8x8 process using a 16x48 matrix.

[0030] [Figure 14] Here is an example of a zero-out region for an 8x8 matrix.

[0031] [Figure 15] Examples of sub-block transform modes SBT-V and SBT-H are shown.

[0032] [Figure 16] An example of a diagonal-upper scan order for a 4x4 coding group is shown.

[0033] [Figure 17] An example of a diagonal-upper scan order for an 8x8 block with a coding group of size 4x4 is shown.

[0034] [Figure 18] 1 shows an example of a template used to select a probabilistic model.

[0035] [Figure 19] Here is an example of two scalar quantizers used for dependent quantization.

[0036] [Figure 20] 13 shows examples of state transitions and quantizer selection for the proposed dependent quantization process.

[0037] [Figure 21] Here is an example of an 8x8 block with 4 coding groups.

[0038] [Figure 22A] 1 shows a flowchart of an exemplary method of video processing. [Figure 22B] 1 shows a flowchart of an exemplary method of video processing. [Figure 22C] 1 shows a flowchart of an exemplary method of video processing. [Figure 22D]1 shows a flowchart of an exemplary method of video processing. [Figure 22E] 1 shows a flowchart of an exemplary method of video processing. [Figure 22F] 1 shows a flowchart of an exemplary method of video processing.

[0039] [Figure 23A] 1 is a block diagram of an example hardware platform for implementing the visual media decoding or visual media encoding techniques described herein. [Figure 23B] 1 is a block diagram of an example hardware platform for implementing the visual media decoding or visual media encoding techniques described herein.

[0040] [Figure 24A] The former example shows one chroma block and its corresponding luma block under a dual tree partition. [Figure 24B] The latter example shows one chroma block and its corresponding luma block under a dual-tree partition.

[0041] [Diagram 25] An example of a 'CR' position for DM derivation from the corresponding luma block is shown.

[0042] [Figure 26A] 1 shows a flowchart of an exemplary method of video processing. [Figure 26B] 1 shows a flowchart of an exemplary method of video processing. [Figure 26C] 1 shows a flowchart of an exemplary method of video processing.

[0043] [Figure 27A] 1 shows a flowchart of an exemplary method of video processing. [Figure 27B] 1 shows a flowchart of an exemplary method of video processing. [Figure 27C] 1 shows a flowchart of an exemplary method of video processing. [Figure 27D] 1 shows a flowchart of an exemplary method of video processing. [Figure 27E] 1 shows a flowchart of an exemplary method of video processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0044] Embodiments of the disclosed techniques may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used herein to improve readability of the description and do not in any way limit the description or embodiments (and / or implementations) to the individual sections.

[0045] 2. Video Coding Introduction Due to the ever-increasing demand for higher resolution video, video coding methods and techniques are ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress or decompress digital video and are constantly being improved to provide higher coding efficiency. Video codecs convert uncompressed video to compressed formats and vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bit rate), the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, the ease of editing, random access, and end-to-end delay (latency). Compressed formats usually comply with a standard video compression standard, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the Versatile Video Coding (VVC) standard that is to be finalized, or other current and / or future video coding standards.

[0046] Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, while ISO / IEC created MPEG-1 and MPEG-4 Visual, and the two organizations jointly created H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure, in which temporal prediction and transform coding are used. In 2015, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG to explore future video coding technologies beyond HEVC. Since then, many new methods have been adopted by JVET and incorporated into a reference software named the Joint Exploration Model (JEM) [3][4]. In April 2018, the Joint Video Experts Team (JVET) was formed between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work on the VVC standard, which aims for a 50% bitrate reduction compared to HEVC.

[0047] 2.1 Typical video codec coding flow Figure 1 shows an example of an encoder block diagram for VVC, which includes three in-loop filtering blocks: Deblocking Filter (DF), Sample Adaptive Offset (SAO), and ALF. Unlike DF, which uses predefined filters, SAO and ALF use the original samples of the current picture to reduce the mean squared error between the original samples and the reconstructed samples by adding an offset and by applying a Finite Impulse Response (FIR) filter, respectively, and the coded side information signals the offset and filter coefficients. The ALF is located at the final processing stage of each picture and can be seen as a tool that tries to catch and repair artifacts produced in previous stages.

[0048] 2.2 Intra-coding in VVC 2.2.1 Intra-mode coding with 67 intra-prediction modes To capture any edge orientation present in natural video, the number of directional intra modes is expanded from 33 as used in HEVC to 65. The additional directional modes are depicted as dotted arrows in Figure 2, while the planar and DC modes remain the same. These denser directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction.

[0049] The traditional angular intra prediction direction is defined from 45 degrees to -135 degrees in a clockwise direction as shown in Figure 2. In VTM2, some traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original method and remapped to wide-angle mode indices after analysis. The total number of intra prediction modes remains unchanged, i.e., 67, and the intra mode coding remains unchanged.

[0050] In HEVC, all intra-coded blocks are square and the length of each side is a power of 2. Therefore, no splitting is required to generate an intra predictor using DC mode. In VVV2, blocks may have rectangular shapes, and splitting needs to be used for each block in the general case. To avoid splitting for DC prediction, only the long side is used to calculate the average for non-square blocks.

[0051] In addition to the 67 intra prediction modes, wide-angle intra prediction for non-square blocks (WAIP) and position-dependent intra prediction combined (PDPC) methods are further enabled for certain blocks. PDPC is applied without signaling for the following intra modes: planar, DC, horizontal, vertical, bottom-left angle mode and its 8 neighboring angle modes, and top-right angle mode and its 8 neighboring angle modes.

[0052] 2.2.2 Affine Linear Weighted Intra Prediction (ALWIP or Matrix-Based Intra Prediction) Affine linear weighted intra prediction (ALWIP, also known as matrix based intra prediction, MIP) was proposed in JVET-N0217.

[0053] 2.2.2.1 Generating a reduced prediction signal by matrix-vector multiplication First, the adjacent reference samples are down-sampled by averaging to obtain a reduced reference signal bdry red Then, the reduced prediction signal pred is generated by calculating the matrix-vector product and adding an offset. red is calculated:

[0054]

number

[0055] Here, A has 4 columns if W=H=4, and 8 columns in all other cases, and W red H red is a matrix with rows and columns. b is of size W red H red is a vector where

[0056] 2.2.2.2 Overall ALWIP Process Description The overall process of averaging, matrix vector multiplication, and linear interpolation is illustrated for various shapes in Figures 3-6. Note that the remaining shapes are treated the same as any of the cases shown.

[0057] 1. Given a 4x4 block, ALWIP takes two averages along each axis of the boundary. The resulting four input samples go into a matrix-vector multiplication. The matrix is ​​then added to the set S 0 After adding the offset, this results in 16 final predicted samples. No linear interpolation is required to generate the predicted signal. Therefore, a total of (4 16) / (4 4) = 4 multiplications are performed per sample.

[0058] 2. Given an 8x8 block, ALWIP takes 4 averages along each axis of the boundary. The resulting 8 input samples go into a matrix-vector multiplication. The matrix is ​​then added to the set S 1 This results in 16 samples in odd positions of the prediction block. So a total of (8·16) / (8·8)=2 multiplications are performed per sample. After adding the offset, these samples are interpolated vertically by using the reduced top border. Horizontal interpolation follows by using the original left border.

[0059] 3. Given an 8x4 block, ALWIP takes the four averages along the horizontal axis of the boundary and the four original boundary values ​​at the left boundary. The resulting eight input samples go into a matrix-vector multiplication. The matrix is ​​then added to the set S 1The input samples are taken from the left and right edges of the prediction block. This results in 16 samples at the odd horizontal positions and at each vertical position of the prediction block. Thus, a total of (8 16) / (8 4) = 4 multiplications are performed per sample. After adding the offset, these samples are horizontally interpolated by using the original left and right borders.

[0060] 4. Given a 16x16 block, ALWIP takes 4 averages along each axis of the boundary. The resulting 8 input samples go into a matrix-vector multiplication. The matrix is ​​then added to the set S 2 This results in 64 samples in odd positions of the prediction block. Thus, a total of (8·64) / (16·16)=2 multiplications are performed per sample. After adding the offset, these samples are vertically interpolated by using the 8 averages of the top border. Horizontal interpolation follows by using the original left border. In this case, the interpolation process does not add any multiplications. Thus, overall, 2 multiplications per sample are needed to compute the ALWIP prediction.

[0061] For larger shapes, the procedure is essentially the same, and it is easy to verify that the number of multiplications per sample is less than four.

[0062] For W×8 blocks (W>8), only horizontal interpolation is required since samples are provided at odd horizontal positions and at every vertical position.

[0063] Finally, for W × 4 blocks (W > 8), let A_kbe be the matrix resulting from excluding all rows corresponding to odd entries along the horizontal axis of the downsampled block, so that the output size is 32, again leaving only horizontal interpolation to be performed.

[0064] Transposed cases are treated accordingly. When creating an MPM list for an intra-predicted block, if the neighboring blocks are coded in MIP mode, the MIP mode will be mapped to the intra-prediction mode by using the following table: where widthNeig and hightNeig are the width and height of the neighboring luma block, respectively, MipSizeId is derived as follows: If widthNeig and hightNeig are equal to 4, MipSizeId is set equal to 0. Otherwise, if both widthNeig and hightNeig are less than or equal to 8, then MipSizeId is set equal to 1. Otherwise, MipSizeId is set equal to 2.

[0065]

number

[0066] 2.2.3 Multiple Reference Lines (MRLs) Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. In Figure 7, an example of four reference lines is depicted, where samples of segments A and F are padded with the nearest samples from segments B and E, respectively, instead of taking from reconstructed neighboring samples. HEVC intra picture prediction uses the nearest reference line (i.e. reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used.

[0067] The index of the selected reference line (mrl_idx) is signaled and used to generate the intra predictor. For reference line indexes greater than 0, we simply include additional reference line modes in the MPM list and signal the MPM index without using the remaining modes. The reference line index is signaled before the intra prediction modes, and planar and DC modes are excluded from intra prediction modes if a non-zero reference line index is signaled.

[0068] The MRL is disabled for the first line of a block inside the CTU, prohibiting the use of extended reference samples outside the current CTU line, and the PDPC is disabled if additional lines are used. 2.2.4 Intra-Sub-Block Partitioning (ISP) In JVET-M0102, ISP is proposed, which divides the luma intra prediction block into two or four sub-partitions vertically or horizontally depending on the dimension of the block size, as shown in Table 1. Figures 8 and 9 show examples of two possibilities. All sub-partitions meet the condition that they have at least 16 samples. If the block size is 4xN or Nx4 (N>8), there may be 1xN or Nx1 sub-partitions, if allowed.

[0069]

number

[0070]

number

[0071]

number

[0072] In VTM4, transforms with large block sizes up to 64x64 in size are enabled, which is mainly useful for high resolution videos, e.g., 1080p and 4K sequences. High frequency transform coefficients are zeroed so that only low frequency coefficients are kept for transform blocks whose size (width or height, or both width and height) is equal to 64. For example, for an MxN transform block, where M is the block width and N is the block height, only the left 32 columns of transform coefficients are kept if M is equal to 64. Similarly, only the top 32 rows of transform coefficients are kept if N is equal to 64. When transform skip mode is used for large blocks, the entire block is used without zeroing any values. In addition to the DCT-II adopted in HEVC, the Multiple Transform Selection (MTS) method is used for residual coding of both inter- and intra-coded blocks. It uses multiple transforms selected from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table 4 below shows the basis functions of the selected DST / DCT.

[0073]

number

[0074] To preserve the orthogonality of the transform matrices, they are quantized more precisely than those of HEVC. After horizontal and vertical transforms, all coefficients should have 10 bits to keep intermediate values ​​of the transformed coefficients within the 16-bit range.

[0075] To control the MTS scheme, separate enablement flags are specified at the SPS level for intra and inter respectively. When MTS is enabled at the SPS, a CU level flag is signaled to specify whether MTS is applied or not, where MTS is applied only for luma. The MTS CU level flag is signaled if the following conditions are met:

[0076] Both width and height must be 32 or less.

[0077] ○ CBF flag is equal to 1 If the MTS CU flag is equal to zero, the DCT2 is applied bidirectionally. However, if the MTS CU flag is equal to one, two other flags are additionally signaled to specify the transform type for the horizontal and vertical directions, respectively. The transform and signaling mapping is as shown in Table 5. In terms of transform matrix precision, an 8-bit linear transform core is used. Therefore, all transform cores used in HEVC, including 4-point DCT-2 and DST-7, 8-point, 16-point, and 32-point DCT-2, remain the same. Also, other transform cores, including 64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7, and DCT-8, use 8-bit linear transform cores.

[0078]

number

[0079] To reduce the complexity of large sizes of DST-7 and DCT-8, for DST-7 and DCT-8 blocks with size (width or height, or both width and height) equal to 32, the high frequency transform coefficients are zeroed. Only the coefficients in the 16x16 low frequency region are retained.

[0080] In addition to cases where different transforms are applied, VVC also supports a mode called Transform Skip (TS), which is similar to the concept of TS in HEVC. TS is treated as a special case of MTS.

[0081] 2.4.2 Reduced Quadratic Transformation (RST) proposed in JVET-N0193 2.4.2.1 Non-separable second-order transformation (NSST) in JEM In JEM, a secondary transform is applied between the forward linear transform and quantization (encoder side) and between the inverse quantization and the invert linear transform (decoder side). As shown in Figure 10, a 4x4 (or 8x8) secondary transform is performed depending on the block size. For example, a 4x4 secondary transform is applied to small blocks (i.e., min (width, height) < 8) and for every 8x8 block, an 8x8 secondary transform is applied to larger blocks (i.e., min (width, height) > 4).

[0082] As an example, we will explain the application of non-separable transform using the following input. To apply non-separable transform, first

[0083]

number

[0084]

number

[0085]

number

[0086]

number

[0087] 2.4.2.2 Reduced Secondary Transform (RST) in JVET-N0193 RST (Low Frequency Non-Separable Transform, LFNST) was introduced in JVET-K0099, and the mapping of four transform sets (instead of 35 transform sets) was introduced in JVET-L0133. In this JVET-N0193, 16×64 matrices (which are further reduced to 16×48 matrices) and 16×16 matrices are used. For notation convenience, the 16×64 (reduced to 16×48) transform is denoted as RST8×8, and the 16×16 transform is denoted as RST4×4. Figure 11 shows an example of RST.

[0088] 2.4.2.2.1 RST Calculation The main idea of the reduced transform (RT) is to map an N-dimensional vector to an R-dimensional vector in another space, where R / N (R < N) is the reduction factor.

[0089]

Equation

[0090] Here, the R rows of the transformation matrix are the R bases of the N-dimensional space. The inverse transformation matrix for RT is the transpose of its forward transformation. The forward and inverse RTs are depicted in Figure 12.

[0091] In this contribution, RST8x8 with a reduction factor of 4 (1 / 4 size) is applied. Thus, instead of the traditional 8x8 non-separable transform matrix size of 64x64, a 16x64 direct matrix is ​​used. In other words, a 64x16 invert RST matrix is ​​used at the decoder side to generate the core (primary) transform coefficients in the top-left region of the 8x8. The forward RST8x8 uses a 16x64 (or 8x64 for 8x8 blocks) matrix, which results in non-zero coefficients only in the top-left 4x4 region in a given 8x8 region. In other words, when RST is applied, the 8x8 region except the top-left 4x4 region will have only zero coefficients. For RST4x4, 16x16 (8x16 for 4x4 blocks), direct matrix multiplication is applied.

[0092] Inverted RST is conditionally applied if the following two conditions are met:

[0093] ○ Block size must be greater than or equal to a given threshold (W>=4 && H>=4)

[0094] ○ The conversion skip mode flag is equal to zero.

[0095] If both the width (W) and height (H) of a transform coefficient block are greater than 4, then RST8x8 is applied to the top-left 8x8 region of the transform coefficient block. Otherwise, RST4x4 is applied to the top-left min(8,W) x min(8,H) region of the transform coefficient block.

[0096] If the RST index is equal to 0, then the RST is not applied. Otherwise, the RST is applied and the kernel is selected using the RST index. The method for selecting the RST and the coding of the RST index are described below.

[0097] Furthermore, RST applies to intra CUs, and to both luma and chroma, in both intra and inter slices. If dual tree is enabled, RST indexes for luma and chroma are signaled separately. In the inter slice case (dual tree is disabled), a single RST index is signaled and used for both luma and chroma.

[0098] 2.4.2.2.2 RST Constraints When ISP mode is selected, RST is disabled and the RST index is not signaled because even if RST is applied to all appropriate partition blocks, the performance improvement is only marginal. Furthermore, disabling RST for ISP predicted residuals may reduce the coding complexity.

[0099] 2.4.2.2.3 RST Selection The RST matrix is ​​selected from four transform sets, each of which consists of two transforms. Which transform set is applied is determined by the intra prediction mode, as follows:

[0100] (1) If one of the three CCLM modes is specified, conversion set 0 is selected.

[0101]

number

[0102] The index to access the above table, denoted IntraPredMode, has the range [-14,83], which is the transform mode index used for wide-angle intra prediction.

[0103] Later, the selection of the Low Frequency Non-separable Transform (LFNST) (also known as RST) set for chroma blocks coded in CCLM mode is modified based on the variable IntraPredMode_CCLM, where IntraPredMode_CCLM has a range of [-14, 80], which is determined by the dimensions of the current chroma block and the co-located luma intra prediction mode.

[0104] When dual tree is enabled, a block (e.g., PU) that covers the corresponding luma sample of the top-left chroma sample in the current chroma block is defined as the co-located (or co-located) luma block. An example is shown in Figures 24a and 24b, where the location of the co-located is denoted by TL.

[0105] 2.4.2.2.4 RST Matrix with Reduced Dimensions For further simplification, the same set of transformations are used to apply a 16x48 matrix instead of a 16x64 matrix, each of which takes 48 pieces of data from three 4x4 blocks in the top left 8x8 block (excluding the bottom right 4x4 block) (as shown in Figure 13).

[0106] 2.4.2.2.5 RST signaling The forward RST8x8 uses a 16x48 matrix, which results in a non-zero coefficient only in the top-left 4x4 region among the first three 4x4 regions. In other words, when RST8x8 is applied, only the top-left 4x4 (due to RST8x8) and bottom-right 4x4 (due to linear transform) regions can have non-zero coefficients. As a result, if any non-zero elements are found in the top-right 4x4 and bottom-left 4x4 block regions (shown in Figure 14 and called "zero-out" regions), the RST index is not coded because it means that the RST was not applied. In such cases, the RST index is presumed to be zero.

[0107] 2.4.2.2.6 Zero-out areas in one CG Usually, before applying the inverted RST to a 4x4 sub-block, any coefficients in the 4x4 sub-block may be non-zero. However, in some cases, some coefficients of a 4x4 sub-block are restricted to be zero before the inverted RST is applied to the sub-block.

[0108] Let nonZeroSize be a variable that requires that, prior to the inversion RST, any coefficient that, when rearranged into a 1-D array, has an index not less than nonZeroSize must be zero.

[0109] If nonZeroSize is equal to 16, there are no zero-out constraints on the coefficients of the top-left 4x4 sub-block.

[0110]

number

[0111] 2.4.3 Sub-block transformation For inter predicted CUs with cu_cbf equal to 1, cu_sbt_flag may be signaled to indicate whether the entire residual block or a sub-part of the residual block is decoded. In the former case, the inter MTS information is further parsed to determine the transform type of the CU. In the latter case, part of the residual block is coded with an estimated adaptive transform and other parts of the residual block are zeroed out. SBT does not apply to mixed inter-intra modes.

[0112]

number

[0113] 2.4.4 Quantized Residual Domain Block Differential Pulse Code Modulation Coding (QR-BDPCM) JVET-N0413 proposes quantized residual domain BDPCM (hereafter referred to as RBDPCM). Similar to intra prediction, intra prediction is performed for the whole block by sample-copying in the prediction direction (horizontal or vertical prediction). The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) quantized value is coded.

[0114]

number

[0115]

number

[0116] For horizontal prediction, similar rules apply and the residual quantized samples are obtained by:

[0117]

number

[0118]

number

[0119]

number

[0120]

number

[0121] In the horizontal case:

[0122]

number

[0123]

number

[0124] If QR-BDPCM is selected, there is no transformation applied.

[0125] 2.5 Entropy coding of coefficients 2.5.1 Coefficient coding of transform application blocks In HEVC, the transform coefficients of a coding block are coded using non-overlapping coefficient groups (or sub-blocks), where each CG contains a 4x4 block of coefficients for the coding block. The CGs within a coding block and the transform coefficients within a CG are coded according to a predefined scan order.

[0126] The CG and intra-CG transform coefficients in a coding block are coded according to a predefined scan order. Both the CG and intra-CG coefficients follow a diagonal-upper scan order. Examples of 4x4 block and 8x8 scan orders are depicted in Figure 16 and Figure 17, respectively.

[0127] Note that the coding order is the reversed scan order (i.e., decoding from CG3 to CG0 in FIG. 17), and when decoding a block, the coordinates of the last non-zero coefficient are decoded first.

[0128] The coding of transform coefficient levels of CGs with at least one non-zero transform coefficient may be separated into multiple scan passes. In the first pass, the first bin (denoted by bin0, also referred to as significant_coeff_flag, which indicates that the coefficient magnitude is greater than 0) is coded. Then, two scan passes for the context coding the second / third bins (denoted by bin1 and bin2, respectively, also referred to as coeff_abs_greater1_flag and coeff_abs_greater2_flag) may be applied. Finally, two more scan passes for coding the coefficient level sign information and the remaining value (also referred to as coeff_abs_level_remaining) may be invoked, if necessary. Note that only the bins of the first three scan passes are coded in regular mode, and they are referred to as regular bins in the following description.

[0129] In VVC 3, for each CG, the regular and bypass coded bins are separated in the coding order; first all the regular coded bins for a subblock are transmitted, followed by the bypass coded bins. The transform coefficient levels of the subblock are coded in five passes over the scan positions as follows:

[0130] ○ Pass 1: The coding of the significance (sig_flag), the greater than 1 flag (gt1_flag), the parity (par_level_flag), and the greater than 2 flag (gt2_flag) are processed in coding order. If sig_flag is equal to 1, then gt1_flag is coded first (it specifies whether the absolute level is greater than 1). If gt1_flag is equal to 1, then par_flag is coded additionally (it specifies the parity of the absolute level minus 2).

[0131] Pass 2: The coding of the remaining absolute levels (remainders) is processed for all scan positions having gt2_flag equal to 1 or gt1_flag equal to 1. Non-binary syntax elements are binarized with a Golomb-Rice code and the resulting bins are coded in the bypass mode of the arithmetic coding engine.

[0132] Pass 3: The absolute levels (absLevel) of coefficients whose sig_flag was not coded in the first pass (due to reaching the limit of a regular coded bin) are fully coded in the bypass mode of the arithmetic coding engine using Golomb-Rice coding.

[0133] Pass 4: Code the sign (sign_flag) for all scan positions that have sig_coeff_flag equal to 1.

[0134] For 4x4 sub-blocks, it is guaranteed that no more than 32 regularly coded bins (sig_flag, par_flag, gt1_flag, and gt2_flag) are coded or decoded. For 2x2 chroma sub-blocks, the number of regularly coded bins is limited to 8.

[0135] The Rice parameter (ricePar) for coding the non-binary syntax element remainder (in pass 3) is derived similarly to HEVC. At the beginning of each subblock, ricePar is set equal to 0. After coding the syntax element remainder, the Rice parameter is modified according to a predefined formula. For coding the non-binary syntax element absLevel (in pass 4), the sum of absolute values ​​sumAbs in the local template is determined. The variables ricePar and posZero are determined based on the dependent quantization and sumAb by table lookup. The intermediate variable codeValue is derived as follows:

[0136] ○ if absLevel[k] is equal to 0, then codeValue is set equal to posZero;

[0137] Otherwise, if absLevel[k] is less than or equal to posZero, then codeValue is set equal to absLevel[k]-1;

[0138] Otherwise (absLevel[k] is greater than posZero), codeValue is set equal to absLevel[k].

[0139] The value of codeValue is coded using the Golomb-Rice code with the rice parameter ricePar.

[0140] 2.5.1.1 Context Modeling for Coefficient Coding The choice of the probability model for syntax elements related to the magnitude of the transform coefficient levels depends on the values ​​of the magnitude levels in a local neighborhood or on the values ​​of the partially reconstructed magnitude levels. The template used is shown in FIG.

[0141] The probability model chosen depends on the sum of the absolute levels (or partially reconstructed absolute levels) in the local neighborhood and the number of absolute levels greater than 0 in the local neighborhood (given by the number of sig_coeff_flags equal to 1). Context modeling and binarization depend on the following measures for the local neighborhood:

[0142] numSig: number of non-zero levels in the local neighborhood;

[0143] sumAbs1: the sum of the partially reconstructed absolute levels (absLevel1) after the first pass in the local neighborhood;

[0144] ○ sumAbs: the sum of the reconstructed absolute levels in the local neighborhood;

[0145] ○ Diagonal position (d): The sum of the horizontal and vertical coordinates of the current scan position within the transformation block.

[0146] Based on the values ​​of numSig, sumAbs1, and d, a probability model is selected for coding sig_flag, par_flag, gt1_flag, and gt2_flag. The Rice parameters for binarizing abs_remainder are selected based on the values ​​of sumAbs and numSig.

[0147] 2.5.1.2 Dependent Quantization (DQ) Furthermore, the same HEVC scalar quantization is used with a new concept called dependent scale quantization. Dependent scalar quantization refers to an approach in which the set of allowable reconstructed values ​​of a transform coefficient depends on the value of the transform coefficient level preceding the current transform coefficient level in the reconstruction order. The main effect of this approach is that allowable reconstructed vectors are packed more densely in an N-dimensional vector space (N represents the number of transform coefficients in a transform block) compared to traditional independent scalar quantization as used in HEVC. This means that for a given average number of allowable reconstructed vectors per N-dimensional unit volume, the average distortion between an input vector and the closest reconstructed vector is reduced. The dependent scalar quantization approach is realized by: (a) defining two scalar quantizers with different reconstruction levels, and (b) defining a process for switching between the two scalar quantizers.

[0148] Two scalar quantizers, denoted Q0 and Q1, are shown in Figure 19. The location of the available reconstruction levels is uniquely specified by the quantization step size Δ. The scalar quantizer used (Q0 or Q1) is not explicitly signaled in the bitstream. Instead, the quantizer used for the current transform coefficient is determined by the parity of the transform coefficient level that precedes the current transform coefficient in the coding / reconstruction order.

[0149]

number

[0150] 2.5.2 Coefficient Coding for TS-Coded and QR-BDPCM-Coded Blocks QR-BDPCM follows the context modeling method for TS coded blocks.

[0151] Modified transform coefficient level coding on TS residuals Compared to the regular residual coding case, the residual coding for TS includes the following changes:

[0152] (1) There is no signaling of the final x / y position.

[0153] (2) A coded_sub_block_flag that is coded for all sub-blocks except the last sub-block if all preceding flags are equal to 0.

[0154] (3) sig_coeff_flag context modeling using reduced templates

[0155] (4) Single-context model for abs_level_gt1_flag and par_level_flag

[0156] (5) Context modeling for sign flags, additional flags larger than 5, 7, and 9

[0157] (6) Modified Rice Parameter Derivation for Remainer Binarization (7) Limit on the number of context-coded bins per sample: Two bins per sample in a block.

[0158]

number

[0159] 2.5 Chroma Direct Mode In Direct Mode (DM), the prediction mode of the collocated luma block is used to derive the chroma intra prediction mode.

[0160] First, the intra prediction mode lumaIntraPredMode is derived: If the collocated luma block is coded in MIP mode, lumaIntraPredMode is set equal to planar mode. Otherwise, if the collocated luma block is coded in IBC mode or palette mode, lumaIntraPredMode is set equal to DC Mode. Otherwise, lumaIntraPredMode is set equal to the intra prediction mode of the co-located luma block that covers the corresponding luma sample in the center of the chroma block. An example is shown in Figure 25.

[0161]

number

[0162]

number

[0163]

number

[0164] 3 Shortcomings of existing implementations The current design has the following problems:

[0165] (1) The four predefined transform sets for the chroma components are the same as the transform sets for the luma components. Furthermore, luma and chroma blocks that utilize the same intra prediction mode use the same transform set. However, chroma signals are typically smoother compared to the luma components. Using the same set may not be optimal.

[0166] (2) RST is applied only to a specific CG, not to all CGs. However, the decision on signaling the RST index depends on the number of non-zero coefficients in the entire block. If all coefficients of the CG to which RST is applied are zero, there is no need to signal the RST index. However, the current design may still signal the index, wasting unnecessary bits.

[0167] (3) The RST index is signaled after the residual coding because it is necessary to record how many non-zero coefficients there are and whether non-zero coefficients exist at a given location (e.g., numZeroOutSigCoeff, numSigCoeff in Section 2.3.2.2.7). Such a design makes the analysis process more complicated.

[0168] (4) The RST index is context coded, and the context modeling depends on the coded luma / chroma intra prediction mode and the MTS index. Such a design introduces analysis delay in terms of intra prediction mode reconstruction. Also, eight contexts are introduced, which may be a burden for hardware implementation.

[0169] (a) Although DM and CCLM share the same context index and offset, this does not make sense as they are two different chroma-intra prediction methods.

[0170] (5) The current design of non-TS residual coding first codes the coefficient information, followed by coding the index of the RST (i.e., whether to use the RST or not, and if so, which matrix is ​​selected). In such a design, the on / off information of the RST cannot be taken into account in the entropy coding of the residual.

[0171] (6) RST is always applied to the top-left region of the transformation block on which the linear transformation was applied. However, when based on a different linear transformation, it is not always true that the energy is concentrated in the top-left region of the transformation block.

[0172] (7) The decision to signal RST-related information is made differently for the dual-tree and single-tree coding structures.

[0173] (8) If there is more than one TU in a CU (e.g., the CU size is 128x128), whether to parse the RST-related information can only be determined after decoding all TUs. For example, for a 128x128 CU, the first PB cannot be processed without waiting for the LFNST index that comes after the last PB. This does not necessarily break the overall 64x64-based decoder pipeline (if CABAC is separable), but it increases the data buffering by 4 times for a certain number of decoder pipeline stages, which is costly.

[0174] (9) In JVET-O0219, the intra prediction mode of the collocated luma block is used to determine the LFNST transform set of the CCLM mode coded chroma block. However, the collocated luma block may be coded in a non-intra prediction mode (i.e., not a traditional intra prediction method such as using DC / planar / angular prediction direction). For example, in the dual tree case, the collocated luma block may be coded in palette mode, IBC mode, etc. In this case, the intra prediction mode of the collocated luma block is undetermined. Therefore, it is unclear how to derive the secondary transform set for the chroma block.

[0175] (10) The derivation of a chroma intra prediction block defined in subclause 8.4.4 checks whether the luma block covering the corresponding top-left luma sample is coded in MIP / IBC / palette mode. If it is true, the default mode is given. Otherwise, the intra prediction mode of the central luma sample is used. That gives rise to two problems: a. If the coding block covering the top-left luma sample (e.g., TL in Figures 24(a) and 24(b), which is the corresponding luma sample of the top-left chroma sample of the current chroma block) is coded in MIP mode and the coding block covering the center luma sample (e.g., CR in Figure 25) is coded in normal intra mode, in this case, the default mode is used and set as the chroma intra prediction mode, which blocks the correlation between the current chroma block and the luma block covering CR. This will result in lower coding performance. b. If the coding block covering the top-left luma sample (e.g., TL in Figures 24(a) and 24(b), which is the corresponding luma sample of the top-left chroma sample of the current chroma block) is coded in intra mode and the coding block covering the center luma sample (e.g., CR in Figure 25) is coded in IBC / palette / MIP, in this case, the intra prediction mode from the coding block covering CR is used to derive the chroma DM mode. However, there is no definition of such intra prediction mode related to MIP / IBC / palette modes.

[0176] 4. Example Method for Context Modeling in Residual Coding Embodiments of the presently disclosed technology overcome shortcomings of existing implementations, thereby providing video coding with higher coding efficiency. Methods for context modeling for residual coding based on the disclosed technology may enhance both existing and future video coding standards, as elucidated in the following examples in which various implementations are described. The examples of the disclosed technology provided below illustrate general concepts and are not intended to be construed as limiting. Various features described in these examples can be combined, unless expressly specified otherwise in the examples.

[0177] In the following description, a "block" may refer to a coding unit (CU), a transform unit (TU), or any rectangular region of video data. A "current block" may refer to a decoded / encoded coding unit (CU), a transform unit (TU) currently being decoded / encoded, or any rectangular coding region of video data being decoded / encoded. A "CU" or "TU" may also be known as a "coding block" and a "transform block."

[0178] In these embodiments, the RST may be any variation of the design in JVET-N0193. The RST may be any technique capable of applying a secondary transform to a block or applying a transform to a transform-skip (TS) coded block (e.g., the RST proposed in JVET-N0193 applied to a TS coded block).

[0179] Hereinafter, "normal intra prediction mode" is used to refer to conventional intra prediction methods in which a prediction signal is generated by extrapolating neighboring pixels from a certain direction, such as DC mode, planar mode, and angular intra prediction mode (which may further include, for example, wide-angle intra prediction mode). For blocks coded without using normal intra prediction modes, the block may be coded with at least one coding method, for example, IBC, MIP, palette, BDPCM intra prediction mode.

[0180] Furthermore, 'zero-out regions' or 'zero-out CGs' may refer to those regions / CGs that always have zero coefficients due to the reduced transform size used in the secondary transform process. For example, if the secondary transform size is 16x32 and the CG size is 4x4, it will be applied to the first two CGs, but only the first CG may have non-zero coefficients, and the second 4x4 CG will also be referred to as zero-out CG. Selection of transformation matrices in RST 1. The sub-region to which RST is applied may be a sub-region that is not the top-left part of the block. In one example, RST may be applied to the top right, bottom right, bottom left, or center sub-region of a block. b. The sub-region to which the RST is applied may depend on the intra prediction mode and / or the linear transform matrix (eg, DCT-II, DST-VII, identity transform). 2. The choice of transformation set and / or transformation matrix used in the RST may depend on the color components. In one example, one set of transformation matrices may be used for the luma (or G) components and one set for the chroma (or B / R) components. b. In one example, each color component may correspond to one set. c. In one example, at least one matrix differs in any of the two or more sets for different color components. 3. The choice of the transform set and / or transform matrix used in RST may depend on the intra prediction method (e.g., CCLM, multi-reference line-based intra prediction, matrix-based intra prediction). In one example, one set of transformation matrices may be used for CCLM-coded blocks and another for non-CCLM-coded blocks. b. In one example, one set of transformation matrices may be used for normal intra-predictive coded blocks and another for blocks with multiple reference lines enabled (i.e., those that do not use neighboring lines for intra prediction). c. In one example, one set of transform matrices may be used for blocks with joint chroma residual coding, and another may be used for blocks to which joint chroma residual coding is not applied. d. In one example, at least one matrix differs in any of the two or more sets for different intra prediction methods. e. Alternatively, RST may be disabled for blocks coded with a particular intra-prediction direction and / or a particular coding tool, e.g., CCLM, and / or joint chroma residual coding, and / or a particular color component (e.g., chroma). 4. A default intra prediction mode may be assigned to blocks (e.g., CU / PU / CB / PB) that are not coded with a normal intra prediction mode such as MIP, IBC, or palette. a. The default mode may depend on the coding method (MIP / IBC / palette). b. The default mode may be signaled or derived on-the-fly. c. The default mode may be used in the derivation of the chroma derivation mode (DM). d. The default mode may be used to predict the intra-prediction modes of other blocks, for example to derive a most-probable-mode (MPM) list for coding blocks subsequent to the current block. e. A default mode assigned to a block of one color component (e.g., luma) may be used to derive a transform set or transform indices for another color component (e.g., chroma). f. Alternatively, the default mode may also be stored together with the prediction mode (e.g., intra / inter / IBC). g. The default mode is assigned to inter-coded blocks. do not have It's possible. 5. It is proposed that in all operations of the chroma intra prediction mode derivation process, the information of one luma coding block (sometimes known as corresponding luma block) covering the same pre-determined position of a chroma coding block is used, for example, the same luma coding block covering the same pre-determined position of a chroma coding block is used to check the prediction mode (or coding method such as IBC / MIP / palette) of the luma coding block and is used to fetch the intra prediction mode of the luma coding block. In one example, the same predefined position is defined to be associated with the corresponding luma sample of the center chroma sample of the current chroma block (eg, CR in FIG. 25). b. In one example, the same predefined position is defined to be associated with the corresponding luma sample of the top-left chroma sample of the current chroma block (eg, TL in FIG. 24b). Alternatively, if further coding blocks covering the same given position are coded in IBC / palette / MIP mode / some other non-normal intra prediction mode, the default intra prediction mode may be used to derive the chroma intra prediction mode, otherwise the decoded intra prediction mode of the coding blocks covering the same given position may be used to derive the chroma intra prediction mode. 6. For a chroma block coded in a non-normal intra prediction mode (e.g., CCLM), if the corresponding luma block is coded in a non-normal intra prediction mode (e.g., IBC / palette / MIP), the transform set / transform matrix used in the RST or other coding tool may be derived from one or more default modes or default transform sets. a. Whether to use a default transform set / transform matrix or to derive the transform set / transform matrix from the luma intra prediction mode of the corresponding luma block may depend on the coding mode of the corresponding luma block. i. In one example, if the corresponding luma block is coded in normal intra prediction mode and / or BDPCM, the transform set / transform matrix may be derived according to the luma intra prediction mode associated with the corresponding luma block (e.g., the luma block defined in invention item 5 or a luma block at an equivalent position). ii. In one example, if the corresponding luma block is coded in a non-normal intra prediction mode (e.g., IBC / palette and / or BDPCM), a default transform set (e.g., the one with a set index equal to K (e.g., K=0)) may be used. b. Whether RST is enabled for a block of a color component may depend on the coding method of one or more corresponding blocks in another color component and / or the coding method of the current block. i. Whether to enable RST for a chroma block may depend on the coding method of one or more corresponding luma blocks and / or the coding method of the current chroma block. ii. In one example, RST may be enabled for a chroma block if the corresponding luma block is coded in normal intra prediction mode and / or BDPCM. iii. In one example, RST may be disabled for a chroma block if the corresponding luma block is coded in a non-normal intra prediction mode (e.g., IBC / palette and / or BDPCM). 1) Alternatively, RST may be disabled for chroma blocks coded in non-normal intra prediction modes (e.g., CCLM) if the corresponding luma block is also coded in a non-normal intra prediction mode (e.g., IBC / palette and / or BDPCM). c. If the co-located luma block is not coded in normal intra prediction mode, a predefined intra prediction mode may be assigned, which is then used to select the transform set and / or transform matrix to be used in the RST (e.g., according to the transform set selection table shown in Section 2.4.2.2.3). i. In one example, if the collocated luma block is coded in IBC (intra block copy) mode, a first predefined intra prediction mode (eg, DC or planar mode) may be assigned. ii. In one example, if the collocated luma block is coded in palette mode, a second predefined intra-prediction mode (eg, DC or planar mode) may be assigned. iii. In one example, if the collocated luma block is coded in MIP mode, a third predefined intra prediction mode (eg, planar or DC mode) may be assigned. iv. In one example, if the collocated luma block is coded in a Block-based Delta Pulse Code Modulation (BDPCM) mode, a fourth predefined intra-prediction mode (e.g., planar or DC mode) may be assigned. v. In one example, for each given intra-prediction mode, it may be a DC mode, a planar mode, a vertical mode, a horizontal mode, a 45 degree mode, or a 135 degree mode. vi. Alternatively, if the collocated luma block is coded in MIP mode, the MIP mode may be mapped to an intra prediction mode according to the dimensions and MIP mode of the collocated luma block, for example by using the table in Section 2.2.2.2 “Specification of Mapping between MIP and Intra Prediction Modes”. vii. The predetermined mode may be adaptively selected from candidates such as DC mode and planar mode. 1) A given mode may be signaled from the encoder to the decoder. 2) The predefined modes may be derived from the encoder to the decoder. For example, if the decoder determines that the current picture is a screen content picture, the predetermined mode may be a DC mode, otherwise the predetermined mode may be a planar mode. viii. The predefined mode may be defined in the same manner as that used in the chroma DM derivation process (eg, lumaIntraPredMode in subclause 8.4.3). d. If the co-located luma block is coded in normal intra prediction mode or / and BDPCM mode, the chroma intra prediction mode may be derived depending on the intra prediction mode of the co-located luma block, similar to the DM mode (as specified in Section 2.7). The derived chroma intra prediction mode is then used to select the transform set and / or transform matrix to be used in the RST (e.g., according to the transform set selection table shown in Section 2.6). e. In the above items, the co-located luma block may be a coding block that covers a specific luma position, such as TL in FIG. 24b and CR in FIG. 25. 7. The choice of transformation set and / or transformation matrix used in the RST may depend on the linear transformation. a. In one example, if the linear transform applied to a block is an identity transform (e.g., a TS mode is applied to a block), the transform set and / or transform matrix used in the RST may be different from other types of linear transforms. b. In one example, if the horizontal and vertical 1-D linear transforms applied to a block are in the same basis (e.g., both DCT-II), the transform set and / or transform matrix used in the RST may be different from the linear transform with a different basis for a different direction (vertical or horizontal). Signaling and residual coding of RST side information 8. Whether and / or how to signal side information in the RST (eg, st_idx) may depend on the last non-zero coefficient in the block (in scanning order). In one example, a RST may be enabled and an index of the RST may be signaled only if the last non-zero coefficient is located within the CG to which the RST is applied. b. In one example, if the last non-zero coefficient is not located within the CG to which the RST is applied, the RST is disabled and the signaling of the RST is skipped. Whether to signal the side information of RST (e.g., st_idx) and how to do it may depend on the coefficients of specific color components, rather than all available color components within a CU. a. In one example, only side information may be used to determine whether to signal the side information of RST and how to do it. i. Alternatively, furthermore, the above method is only applied when the dimensions of the block meet specific conditions. 1) The condition is W < T1 or H < T2. 2) For example, T1 = T2 = 4. Therefore, for a 4x4 CU, the luma block size is 4x4, and the two chroma blocks in the 4:2:0 format are 2x2. In this case, only luma information may be used. ii. Alternatively, furthermore, the above method is only applied when the current partition type tree is a single tree. b. Whether to use the information of one color component or the information of all color components may depend on the block dimensions / coded information. 7. Whether to signal the side information of RST (e.g., st_idx) and how to do it may depend on the coefficients within a partial area of one block, rather than the entire block. a. In one example, the partial area may be defined as the CG to which RST is applied. b. In one example, the partial area may be defined as the first or last M (e.g., M = 1, or 2) CGs in the scanning order or reverse scanning order of the block. i. In one example, M may depend on the block dimensions. ii. In one example, when the block size is 4xN and / or Nx4 (N > 8), M is set to 2. iii. In one example, when the block size is 4x8 and / or 8x4 and / or WxH (W ≥ 8, H ≥ 8), M is set to 1. c. In one example, information about a block having dimensions W×H (e.g., the number of non-zero coefficients of the block) may not be allowed to be taken into account to determine the use of RST and / or the signaling of RST-related information. i. For example, when W < T1 or H < T2, the number of non-zero coefficients of the block may not be counted. For example, T1 = T2 = 4. d. In one example, a partial region may be defined as the upper left M×N region of the current block having dimensions W×H. i. In one example, M may be smaller than W, and / or N may be smaller than H. ii. In one example, M and N may be fixed numbers. For example, M = N = 4. iii. In one example, M and / or N may depend on W and / or H. iv. In one example, M and / or N may depend on the maximum allowable transform size. 1) For example, when W is greater than 8 and H is equal to 4, M = 8 and N = 4. 2) For example, when H is greater than 8 and W is equal to 4, M = 4 and N = 8. 3) For example, when neither of the above two conditions is satisfied, M = 4 and N = 4. v. Alternatively, further, these methods may be applied only for certain block dimensions where the condition of 7.c is not satisfied. e. In one example, the partial region may be the same for all blocks. i. Alternatively, this may be changed based on block dimensions and / or coded information. f. In one example, the partial region may depend on a given range of the scanning order index. i. In one example, the partial region may cover coefficients that lie within a particular range with their scanning order index within [dxS, IdxE] inclusive, based on the coefficient scanning order (e.g., reverse decoding order) of a current block having dimensions W×H. 1) In one example, IdxS is equal to 0. 2) In one example, IdxE may be less than W×H-1. 3) In one example, IdxE can be a fixed number, for example IdxE=15. 4) In one example, IdxE may depend on W and / or H. For example, if W is greater than 8 and H is equal to 4, then IdxE=31. b. For example, if H is greater than 8 and W is equal to 4, then IdxE=31. c. For example, if W is equal to 8 and H is equal to 8, then IdxE=7. d. For example, if W is equal to 4 and H is equal to 4, then IdxE=7. e. For example, if neither of the above two conditions a) nor b) is satisfied, then IdxE=15. f. For example, if none of the above two conditions a), b), c) and d) are satisfied, then IdxE=15. g. For example, if neither of the above two conditions c) nor d) is satisfied, then IdxE=15. ii. Alternatively, further, these methods may be applied only for specific block sizes for which the condition in 7.c is not satisfied. g. In one example, this may depend on the location of the non-zero coefficients within the partial region. h. In one example, this may depend on the energy (such as sum of squares or sum of absolute values) of the non-zero coefficients in the partial region. i. In one example, this may depend on the number of non-zero coefficients in a partial region of a block, rather than the entire block. i. Alternatively, it may depend on the number of non-zero coefficients in a partial region of one or more blocks in a CU. ii. If the number of non-zero coefficients in a partial region of a block is smaller than a threshold, the signaling of side information in the RST may be skipped. iii. In one example, the threshold is fixed to be N (eg, N=1 or 2). iv. In one example, the threshold may depend on slice type / picture type / partition tree type (dual or single) / video content (screen content or camera captured content). v. In one example, the threshold may depend on the color format, such as 4:2:0 or 4:4:4, and / or the color components, such as Y or Cb / Cr. 11. If there are no non-zero coefficients in the CG to which RST may be applied, RST shall be disabled. In one example, if RST is applied to a block, at least one CG to which the RST is applied must contain at least one non-zero coefficient. b. In one example, for 4×N and / or N×4 (N>8), if RST is applied, the first two 4×4 CGs must contain at least one non-zero coefficient. c. In one example, for 4x8 and / or 8x4, if RST is applied, the top-left 4x4 must contain at least one non-zero coefficient. d. In one example, for WxH (W>=8 and H>=8), if RST is applied, the top-left 4x4 must contain at least one non-zero coefficient. e. A conforming bitstream must meet one or more of the conditions above. 12. RST-related syntax elements may be signaled prior to coding the residual (eg, transform coefficients / directly quantized). a. In one example, counting the number of non-zero coefficients in the zero-out region (e.g., numZeroOutSigCoeff) and the number of non-zero coefficients in the entire block (e.g., numSigCoeff) is excluded in the coefficient analysis process. b. In one example, RST-related syntax elements (eg, st_idx) may be coded before the residual_coding. c. RST related syntax elements may be signaled conditionally (e.g. according to coded block flags, TS mode usage). vi. In one example, RST related syntax elements (eg, st_idx) may be coded after the signaling of coded block flags or after the signaling of TS / MTS related syntax elements. vii. In one example, if TS mode is enabled (eg, if the decoded transform_skip_flag is equal to 1), then signaling of RST related syntax elements is skipped. d. Residual-related syntax may not be signaled for zero-out CG. e. How the residual is coded (e.g., scanning order, binarization, decoded syntax, context modeling) may depend on the RST. i. In one example, a raster scanning order may be applied rather than a diagonal-upward scanning order. 1) The raster scanning order is from left to right, top to bottom, or vice versa. 2) Alternatively, instead of the diagonal-upward scanning order, a vertical scanning order (top-to-bottom, left-to-right, or vice versa) may be applied. 3) Alternatively, the context modeling may be further modified. In one example, context modeling may rely on previously coded information in the template, namely the N most recent neighbors in scan order, rather than using the right, bottom, and bottom-right neighbors. b. In one example, context modeling may rely on previously coded information in the template according to the scanned index (e.g., -1, -2, ... assuming the current index is equal to 0). ii. In one example, different binarization methods (e.g., Rice parameter derivation) may be applied to code the residuals associated with RST coded and non-RST coded blocks. iii. In one example, signaling of certain syntax elements may be skipped for RST coded blocks. 1) Signaling of the CG coded block flag (coded_sub_block_flag) for the CG to which the RST applies may be skipped. a. In one example, if RST8x8 is applied to the first three CGs in diagonal scan order, then signaling of the CG coded block flags is skipped for the second and third CGs, e.g., the top-right 4x4 CG and the bottom-left 4x4 CG of the top-left 8x8 region of the block. i. Alternatively, further, the corresponding CG coded block flag is presumed to be 0, i.e., all coefficients are zero. b. In one example, when RST is applied to a block, the signaling of the CG coded block flag is skipped for the first CG in scanning order (or the last CG in reverse scanning order). ii. Alternatively, further, the CG coded block flag for the top-left CG in the block is presumed to be 1, i.e., it contains at least one non-zero coefficient. An example of an 8x8 block is shown in Figure 21. When RST8x8 or RST4x4 is applied to an 8x8 block, coded_sub_block_flag of CG0 is inferred to be 1, and coded_sub_block_flag of CG1 and CG2 are inferred to be 0. 2) Signaling of coefficient magnitude and / or sign flags for certain coordinates may be skipped. In one example, if the index for one CG in the scan order is not less than the maximum allowed index at which a non-zero coefficient may exist (eg, nonZeroSize in section 0), the signaling of the coefficient may be skipped. b. In one example, signaling of syntax elements such as sig_coeff_flag, abs_level_gtX_flag, par_level_flag, abs_remainder, coeff_sign_flag, and dec_abs_level may be skipped. 3) Alternatively, the residual signaling (e.g., CG coded block flags, coefficient magnitudes, and / or sign flags of specific coordinates) may be maintained, but the context modeling may be modified to be different for one CG than for another. iv. In one example, the coding of the residual in the CG where the RST is applied and in other CGs may be different. 1) Regarding the above sub-items, they may only be applied to CGs to which the RST applies. 10. RST related syntax elements may be signaled before other transform indications such as transform skip and / or MTS index. In one example, the signaling of the transform skip may depend on the RST information. i. In one example, if a RST is applied within a block, the transform skip indication is not signaled and is presumed to be 0 for the block. b. In one example, the signaling of the MTS index may depend on the RST information. i. In one example, if a RST is applied within a block, one or more MTS transformation indications are not signaled and are presumed to be unused for the block. 11. It is proposed to use different context modeling methods in arithmetic coding for different parts of a block. In one example, the block is treated as being in two parts: the first M CGs in the scanning order and the remaining CG. In one example, M is set to 1. ii. In one example, M is set to 2 for 4xN and Nx4 (N>8) blocks, and is set to 1 for all other cases. b. In one example, the block is treated as if it were two parts: a sub-region to which RST is applied and a sub-region to which RST is not applied. i. When RST4x4 is applied, the sub-region to which RST is applied is the first one or two CGs of the current block. ii. If RST4x4 is applied, the sub-region to which RST is applied is the first three CGs of the current block. c. In one example, it is proposed to disable the use of previously coded information in the context modeling process for the first part in a block, but enable it for the second part. d. In one example, when decoding the first CG, information of the remaining one or more CGs may not be allowed to be used. i. In one example, when coding the CG coded block flag for the first CG, the value of the second CG (e.g. right or bottom) is not taken into account. ii. In one example, when coding the CG coded block flags for the first CG, the values ​​of the second and third CG (e.g., right and bottom CG for WxH (W≧8 and H≧8)) are not considered. iii. In one example, when coding the current coefficient, if a neighbor in the context template is in a different CG, information from this neighbor is prohibited from being used. e. In one example, when decoding coefficients in a region to which a RST is applied, information in the remaining regions to which a RST is not applied may be prohibited from being used. f. Alternatively, or in addition, the above method may be applied under certain conditions. i. The conditions may include whether RST is enabled. ii. A condition may include a block dimension. Context modeling in arithmetic coding of RST side information 15. When coding a RST index, context modeling may depend on whether explicit or implicit multiple transform selection (MTS) is enabled. In one example, when implicit MTS is enabled, different contexts may be selected for blocks coded in the same intra-prediction mode. i. In one example, block dimensions such as shape (square or non-square) are used to select the context. b. In one example, instead of checking the transform index (eg, tu_mts_idx) coded for an explicit MTS, the basis of the transform matrix may be used. i. In one example, for the basis of transformation matrices using DCT-II for both horizontal and vertical 1-D transformations, the corresponding context may be different from other types of transformation matrices. 16. When coding a RST index, context modeling may depend on whether CCLM is enabled (e.g., sps_cclm_enabled_flag). Alternatively, whether or how to enable context selection for RST index coding may depend on whether CCLM applies to a block. b. In one example, context modeling may depend on whether CCLM is enabled for the current block. i. The following is an example: intraModeCtx = sps_cclm_enabled_flag? ( intra_chroma_pred_mode[ x0 ][ y0 ] is CCLM: intra_chroma_pred_mode[ x0 ][ y0 ] is DM) ? 1 : 0. c. Alternatively, whether or how to enable selecting a context for RST index coding may depend on whether the current chroma block is coded in DM mode. i. The following is an example: intraModeCtx = ( intra_chroma_pred_mode[ x0 ][ y0 ] == (sps_cclm_enabled_flag ? 7:4) ) ? 1 : 0. 17. When coding the RST index, the context modeling may depend on the block dimension / partition depth (e.g., quadtree depth and / or BT / TT depth). 18. When coding the RST index, the context modeling may depend on the color format and / or color components. 19. When coding the RST index, the context modeling may be independent of the intra prediction mode and / or the MTS index. 20. When coding the RST index, the first and / or second bins may be context coded with only one context or may be bypass coded. Conditionally triggering the RST process 21. Whether or not to initiate an inverse RST process may depend on the CG coded block flags. In one example, if the top left CG coded block flag is zero, there is no need to launch the process. i. In one example, if the top-left CG coded block flag is zero and the block size is not equal to 4xN / Nx4 (N>8), then there is no need to invoke the process. b. In one example, if the first two CG coded block flags in the scanning order are both equal to zero, no process needs to be initiated. i. In one example, if the first two CG coded block flags in the scanning order are both equal to zero and the block size is equal to 4xN / Nx4 (N>8), then there is no need to initiate the process. 22. Whether to launch an inverse RST process may depend on the block dimensions. In one example, RST may be disabled for a particular block dimension such as 4x8 / 8x4. Alternatively, signaling of RST related syntax elements may also be skipped. Unified dual-tree and single-tree coding 23. The usage of the RST and / or the signaling of RST related information may be determined in the same manner in dual tree and single tree coding. a. For example, if the number of counted non-zero coefficients (e.g., numSigCoeff as specified in JVET-N0193) is not greater than T1 in the case of dual tree coding or not greater than T2 in the case of single tree coding, where T1 is equal to T2, then RST should not be applied and no related information is signaled. b. In one example, T1 and T2 are both set to N, e.g., N=1 or 2. Consider multiple TUs within a CU. 24. Whether and / or how RST is applied may depend on the block dimensions W×H. In one example, if W>T1 or H>T2, RST may not apply. b. In one example, if W>T1 and H>T2, RST may not apply. c. In one example, if W*H>=T, then RST may not apply. d. With respect to the above example, the following applies: i. In one example, the block is a CU. ii. In one example, T1=T2=64. iii. In one example, T1 and / or T2 may be dependent on the maximum allowed transform size, eg, T1=T2=maximum allowed transform size. iv. In one example, T is set to 4096. e. Alternatively, further, if it is determined that RST does not apply, the related information may not be signaled. 22. If there are N TUs in a CU (N>1), the coded information of only one of the N TUs is used to determine the usage of the RST and / or the signaling of RST-related information. In one example, the first TU of a CU in decoding order may be used to make the decision. b. In one example, the top-left TU of the CU in decoding order may be used to make the decision. c. In one example, the decision to use a particular TU may be made in the same manner as in the case where there is only one TU in a CU. 23. Signaling of RST usage and / or RST related information may be performed at the TU level or PU level instead of at the CU level. Alternatively, different TUs / PUs in a CU may also select different secondary transformation matrices or enable / disable control flags. b. Alternatively, and further for the dual tree case, chroma blocks may be coded and different color components may select different secondary transformation matrices or enable / disable control flags. c. Alternatively, at which video unit level RST-related information is signaled may depend on the partition tree type (dual or single). d. Alternatively, at which video unit level to signal RST-related information may depend on the relationship between the CU / PU / TU and the maximum allowed transform block size, e.g., larger or smaller.

[0181] 5. Examples of implementation of the disclosed technology In the following example embodiment, changes to the latest JVET-N0193 are highlighted in bold and italics. Text that is deleted is marked with double brackets (e.g., [[a]] indicates the deletion of the letter "a").

[0182]

number

[0183]

number

[0184]

number

[0185] The above examples may be incorporated in the context of the methods described below, such as methods 2200, 2210, 2220, 2230, 2240 and 2250, which may be implemented in a video decoder or a video encoder.

[0186] 22A illustrates a flowchart of an example method for video processing. The method 2200 includes, at step 2202, selecting a set of transforms or transform matrices for applying a contracting quadratic transform to a current video block based on characteristics of the current video block.

[0187] The method 2200 includes, at step 2204, applying a selected set of transforms or transform matrices to a portion of the current video block as part of a conversion between the current video block and a bitstream representation of the video that includes the current video block.

[0188] In some embodiments, the portion of the current video block is a top right sub-region, a bottom right sub-region, a bottom left sub-region, or a center sub-region of the current video block.

[0189] In some embodiments, the feature of the current video block is an intra-prediction mode or a primary transform matrix of the current video block.

[0190] In some embodiments, the features are color components of the current video block. In one example, a first set of transforms is selected for a luma component of the current video block, and a second set of transforms different from the first set is selected for one or more chroma components of the current video block.

[0191] In some embodiments, the feature is an intra prediction mode or intra coding method of the current video block. In one example, the intra prediction method includes a multiple reference line (MRL) based prediction method or a matrix based intra prediction method. In another example, the first transform set is selected if the current video block is a cross component linear model (CCLM) coded block, and a second transform set different from the first transform set is selected if the current video block is a non-CCLM coded block. In yet another example, the first transform set is selected if the current video block is coded with a joint chroma residual coding method, and a second transform set different from the first transform set is selected if the current video block is not coded with a joint chroma residual coding method.

[0192] In some embodiments, the feature is a linear transform of the current video block.

[0193] 22B shows a flowchart of an example method for video processing, comprising: making a decision, at step 2212, regarding selective inclusion of signaling of side information for application of a shrinking quadratic transform (RST) in a bitstream representation of a current video block based on one or more coefficients associated with the current video block.

[0194] The method 2210 includes, at step 2214, performing a conversion between the current video block and a video including a bitstream representation of the current video block based on the determination.

[0195] In some embodiments, the one or more coefficients include the last non-zero coefficient in scan order of the current video block.

[0196] In some embodiments, the one or more coefficients include a plurality of coefficients in a partial region of the current video block. In one example, the partial region includes one or more coding groups to which the RST may be applied. In another example, the partial region includes the top M coding groups or the bottom M coding groups in a scan order of the current video block. In yet another example, the partial region includes the top M coding groups or the bottom M coding groups in a reverse scan order of the current video block. In yet another example, the determining step is further based on an energy of one or more non-zero coefficients of the plurality of coefficients.

[0197] 22C shows a flowchart of an example method for video processing. The method 2220 includes, at step 2222, constructing a bitstream representation of the current video block to apply a contracting secondary transform to the current video block, where syntax elements related to the RST are signaled in the bitstream representation prior to coding the residual information.

[0198] The method 2220 includes, at step 2224, performing a conversion between the current video block and a bitstream representation of the current video block based on the configuration.

[0199] In some embodiments, signaling syntax elements associated with the RST is based on the use of at least one coded block flag or a transform selection mode.

[0200] In some embodiments, the bitstream representation excludes coding residual information corresponding to coding groups having all-zero coefficients.

[0201] In some embodiments, the coding residual information is based on the application of a RST.

[0202] 22D shows a flowchart of an example method for video processing. The method 2230 includes, at step 2232, configuring a bitstream representation of a current video block to apply a contracting secondary transform to the current video block, where a syntax element related to a RST is signaled in the bitstream representation before either a transform skip indication or a multiple transform set (MTS) index.

[0203] The method 2230 includes, at step 2234, performing a conversion between the current video block and a bitstream representation of the current video block based on the configuration.

[0204] In some embodiments, the transform skip indication or MTS index is based on a syntax element associated with the RST.

[0205] 22E illustrates a flowchart of an example method for video processing, comprising: constructing a context model for coding an index of a reduced quadratic transform (RST) based on characteristics of a current video block, at step 2242.

[0206] The method 2240 includes, at step 2244, performing a conversion between the current video block and a bitstream representation of the video that includes the current video block based on the configuration.

[0207] In some embodiments, the feature is the explicit or implicit enablement of a multiple transform selection (MTS) process.

[0208] In some embodiments, the feature is enablement of a cross-component linear model (CCLM) coding mode in the current video block.

[0209] In some embodiments, the characteristic is a size of the current video block.

[0210] In some embodiments, the characteristic is a partition depth of a partitioning process applied to the current video block. In one example, the partitioning process is a quadtree (QT) partitioning process, a binary tree (BT) partitioning process, or a ternary tree (TT) partitioning process.

[0211] In some embodiments, the feature is the color format or color components of the current video block.

[0212] In some embodiments, the features exclude an intra-prediction mode of the current video block and an index of a multiple transform selection (MTS) process.

[0213] 22F illustrates a flowchart of an example method for video processing 2250. The method 2250 includes, at step 2252, making a decision regarding selective application of an inverse secondary transform (RST) to a current video block based on characteristics of the current video block.

[0214] The method 2250 includes, at step 2254, performing a conversion between the current video block and the bitstream representation of the video that includes the current video block based on the determination.

[0215] In some embodiments, the feature is a coded block flag of a coding group of the current video block. In one example, an inverse RST process is not applied, where the coded block flag of the top-left coding group is zero. In another example, an inverse RST process is not applied, where the coded block flags for the first and second coding groups in the scan order of the current video block are zero.

[0216] In some embodiments, the feature is the height (M) or width (N) of the current video block. In one example, the inverse RST process is not applied, where (i) M = 8 and N = 4, or (ii) M = 4 and N = 8.

[0217] 26A shows a flowchart of an example method for video processing. The method 2610 includes, at step 2612, determining a default intra-prediction mode for a video unit coded using a particular intra-prediction mode for transforming between a current video block of a video unit of a video and a coded representation of the video such that a predictive block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction. The method 2610 further includes, at step 2614, performing the transform based on the determination.

[0218] 26B shows a flowchart of an example method for video processing. The method 2620 includes, at step 2622, using the rule to determine a luma block of the video that covers a given position of a chroma block of the video. The method 2620 further includes, at step 2624, performing a conversion between the video and a coded representation of the video based on the determination. In some implementations, the chroma block is represented in a coded representation using an intra prediction mode.

[0219] FIG. 26C illustrates a flowchart of an example method for video processing. The method 2630 includes, at step 2632, deriving an intra-prediction mode for a chroma block of a video based on a coding mode of a luma block corresponding to the chroma block using the rule. The method 2630 further includes, at step 2634, performing a conversion between the chroma block and a coded representation of the video based on the derived intra-prediction mode. In some implementations, the rule specifies using a default intra-prediction mode if the coding mode of the luma block is a particular intra-prediction mode in which a predictive block of the luma block is generated without extrapolating neighboring pixels of the luma block along a direction.

[0220] FIG. 27A illustrates a flowchart of an example method for video processing. The method 2710 includes, at step 2712, making a first determination, for a chroma block of the video, whether a non-normal chroma intra prediction mode is applied to the chroma block of the video. The method 2710 further includes, at step 2714, making a second determination, for a luma block corresponding to the chroma block, that a luma intra prediction mode is applied to the luma block. The method 2710 further includes, at step 2716, making a third determination, in which a transform set or transform matrix is ​​applied to the chroma block based on the luma intra prediction mode. The method 2710 further includes, at step 2718, performing a conversion between the video and a coded representation of the video according to the third determination. In some implementations, the non-normal chroma intra prediction mode includes coding the chroma block without using values ​​of neighboring pixels extrapolated along a chroma prediction direction.

[0221] 27B shows a flowchart of an example method for video processing. The method 2720 includes, at step 2722, making a first determination for a chroma block of a video that a luma block corresponding to the chroma block is coded using a non-normal luma intra prediction mode. The method 2720 further includes, at step 2724, making a second determination of a set of transforms or transform matrices for the chroma block according to the rule based on the first determination. The method 2720 further includes, at step 2726, performing a transform between the video and the coded representation of the video according to the second determination. In some implementations, the rules specify that due to the luma block being coded using a non-normal luma intra prediction mode, one or more default modes or default transform sets associated with the chroma block determine a transform set or transform matrix when the chroma block is coded using a non-normal chroma intra prediction mode, where the non-normal luma intra prediction mode includes coding the luma block without using values ​​of neighboring pixels extrapolated along the luma prediction direction, and where the non-normal chroma intra prediction mode includes coding the chroma block without using values ​​of neighboring pixels extrapolated along the chroma prediction direction.

[0222] 27C shows a flowchart of an example method for video processing. The method 2730 includes, at step 2732, determining applicability of a second transform tool applied to a current video block of one color component for converting between a current video block of the video and a coded representation of the video based on at least one of 1) a coding mode of a corresponding block of another color component, or 2) a coding mode of the current video block. The method 2730 further includes, at step 2734, performing the transform based on the determination. In some implementations, using a secondary transform tool: during encoding, a forward secondary transform is applied to an output of a forward primary transform applied to a residual of the current video block prior to quantization, or during decoding, an inverse secondary transform is applied to an output of an inverse quantization of the current video block prior to applying an inverse primary transform.

[0223] FIG. 27D shows a flowchart of an example method for video processing. The method 2740 includes, at step 2742, making a first determination with respect to a chroma block of a video that a luma block covering a predetermined location of the chroma block is encoded using a non-normal luma intra prediction mode. The method 2740 further includes, at step 2744, making a second determination based on the first determination, applying a predetermined intra prediction mode to the chroma block due to the luma block being encoded using the non-normal luma intra prediction mode. The method 2740 further includes, at step 2746, performing a conversion between the video and the coded representation of the video according to the second determination. In some implementations, the non-normal luma intra prediction mode includes encoding the luma block without using values ​​of neighboring pixels extrapolated along a luma prediction direction.

[0224] FIG. 27E shows a flowchart of an example method for video processing. The method 2750 includes making a first decision, with respect to a chroma block of a video, that a luma block covering a predetermined position of the chroma block is encoded using a normal luma intra prediction mode, at step 2752. The method 2750 further includes making a second decision, based on the first decision, at step 2754, to derive a chroma intra prediction mode based on the normal luma intra prediction mode of the luma block. The method 2750 further includes performing a conversion between the video and a coded representation of the video according to the second decision, at step 2756, where the normal luma intra prediction mode includes encoding the luma block using values ​​of neighboring pixels extrapolated along a luma prediction direction.

[0225] FIG. 23A is a block diagram of a video processing device 2300. The device 2300 may be used to implement one or more of the methods described herein. The device 2300 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, and the like. The device 2300 may include one or more processors 2302, one or more memories 2304, and video processing hardware 2306. The processor 2302 may be configured to implement one or more of the methods described herein, including but not limited to the methods 2200 through 2750. The memories 2304 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 2306 may be used to implement some of the techniques described herein in a hardware circuit. In some embodiments, the hardware 2306 may be at least partially within the processor 2302, such as a graphics co-processor.

[0226] FIG. 23B is a block diagram of another example of a video processing system in which the disclosed techniques may be implemented. FIG. 23B is a block diagram illustrating an example video processing system 2400 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 4100. System 2400 may include an input 2402 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or in a compressed or encoded format. Input 2402 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0227] The system 2400 may include a coding component 2404 capable of implementing various coding or encoding methods described herein. The coding component 2404 may reduce the average bit rate of the video from the input 2402 to the output of the coding component 2404 to generate a coded representation of the video. Thus, the coding techniques are sometimes referred to as video compression or video transcoding techniques. The output of the coding component 2404 may be stored or transmitted via a communication connection as represented by component 2406. The stored or communicated bitstream (or coded) representation of the video received at the input 2402 may be used by component 2408 to generate pixel values ​​or displayable video that are sent to the display interface 2410. The process of generating a user viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although certain video processing operations are referred to as "coding" operations or tools, it will be understood that the coding tools or operations are used in an encoder and corresponding decoding tools or operations that process the results of the coding in the reverse direction are performed by a decoder.

[0228] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI), Displayport, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The technology described in this document may be embodied in a variety of electronic devices, such as a mobile phone, a laptop, a smart phone, or other device capable of performing digital data processing and / or video display.

[0229] In some embodiments, the video processing methods described herein may be implemented using an apparatus implemented on a hardware platform such as those described with respect to Figures 23A or 23B.

[0230] Some embodiments of the disclosed techniques include making a judgment or decision to enable a video processing tool or mode. In one example, if a video processing tool or mode is enabled, an encoder will use or implement the tool or mode in processing blocks of video, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, conversion of blocks of video to a bitstream representation of video will use the video processing tool or mode if enabled based on the judgment or decision. In another example, if a video processing tool or mode is enabled, a decoder will process the bitstream with information that the bitstream has been modified based on the video processing tool or mode. That is, conversion of a bitstream representation of video to blocks of video will be performed using the video processing tool or mode that is enabled based on the judgment or decision.

[0231] Some embodiments of the disclosed techniques include making a judgment or decision to disable a video processing tool or mode. In one example, if a video processing tool or mode is disabled, an encoder will not use the tool or mode in converting blocks of video to a bitstream representation of the video. In another example, if a video processing tool or mode is disabled, a decoder will process the bitstream with information that the bitstream has not been modified using the video processing tool or mode that was disabled based on the judgment or decision.

[0232] In this document, the term "video processing" may refer to video encoding, video decoding, video compression or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation or vice versa. The bitstream representation of a current video block may correspond to bits that are at the same position or distributed at different locations in the bitstream, e.g. as defined by a syntax. For example, a macroblock may be coded in terms of a transformed coded error residual value and using bits from the header and other fields in the bitstream.

[0233] Various techniques and embodiments may be described using the following clause format: A first set of clauses describes particular features and aspects of the techniques disclosed in the sections above.

[0234] 1. A video processing method comprising: selecting a set of transforms or transform matrices for applying a reduction quadratic transform to the current video block based on characteristics of the current video block; and applying the selected transform set or transform matrix to a portion of the current video block as part of a transformation between the current video block and a bitstream representation of the video that includes the current video block; The method includes:

[0235] 2. The method of clause 1, wherein the portion of the current video block is a top right sub-region, a bottom right sub-region, a bottom left sub-region or a center sub-region of the current video block.

[0236] 3. The method of any one of clauses 1 to 2, wherein the feature of the current video block is an intra-prediction mode or a linear transform matrix of the current video block.

[0237] 4. The method according to clause 1, wherein said features are color components of the current video block.

[0238] 5. The method of clause 4, wherein a first set of transforms is selected for a luma component of the current video block, and a second set of transforms different from the first set is selected for one or more chroma components of the current video block.

[0239] 6. The method according to clause 1, wherein the feature is an intra-prediction mode or an intra-coding method of the current video block.

[0240] 7. The method of clause 6, wherein the intra prediction method includes a multiple reference line (MRL) based prediction method or a matrix-based intra prediction method.

[0241] 8. The method of claim 6, wherein a first transform set is selected if the current video block is a cross-component linear model (CCLM) coded block, and a second transform set different from the first transform set is selected if the current video block is a non-CCLM coded block.

[0242] 9. The method of claim 6, wherein a first transform set is selected if the current video block is coded in a joint chroma residual coding manner, and a second transform set different from the first transform set is selected if the current video block is not coded in a joint chroma residual coding manner.

[0243] 10. The method according to clause 1, wherein said characteristic is a linear transformation of the current video block.

[0244] 11. A video processing method comprising: making a decision regarding selective inclusion of signaling of side information for application of a reduction-secondary transform (RST) in a bitstream representation of the current video block based on one or more coefficients associated with the current video block; and performing, based on the determination, a conversion between the current video block and a video including a bitstream representation of the current video block; The method includes:

[0245] 12. The method of claim 11, wherein the one or more coefficients include a last non-zero coefficient in a scanning order of the current video block.

[0246] 13. The method of claim 11, wherein the one or more coefficients include a plurality of coefficients within a partial region of the current video block.

[0247] 14. The method according to clause 13, wherein the partial region comprises one or more coding groups to which the RST can be applied.

[0248] 15. The method of claim 13, wherein the partial region includes the first M coding groups or the last M coding groups in a scanning order of the current video block.

[0249] 16. The method of claim 13, wherein the partial region includes the first M coding groups or the last M coding groups in reverse scanning order of the current video block.

[0250] 17. The method of claim 13, wherein making the determination is further based on the energy of one or more non-zero coefficients of the plurality of coefficients.

[0251] 18. A video processing method comprising: constructing a bitstream representation of the current video block for applying a shrinking quadratic transform (RST) to the current video block, where syntax elements related to the RST are signaled in the bitstream representation prior to coding the residual information; and performing a conversion between the current video block and a bitstream representation of the current video block based on the construction; The method includes:

[0252] 19. The method according to clause 18, wherein signalling a syntax element associated with the RST is based on usage of at least one coded block flag or a transform selection mode.

[0253] 20. The method according to clause 18, wherein the bitstream representation excludes coding residual information corresponding to coding groups that are all-zero coefficients.

[0254] 21. The method of clause 18, wherein the coding residual information is based on application of a RST.

[0255] 22. A video processing method comprising: constructing a bitstream representation of a current video block to apply a reduction secondary transform (RST) to the current video block, where a syntax element related to the RST is signaled in the bitstream representation before either a transform skip indication or a multiple transform set (MTS) index; and performing a conversion between the current video block and a bitstream representation of the current video block based on the construction; The method includes:

[0256] 23. The method of clause 22, wherein the transform skip indication or MTS index is based on a syntax element associated with the RST.

[0257] 24. A video processing method comprising: constructing a context model for coding an index of a reduced quadratic transform (RST) based on features of the current video block; and performing a conversion between the current video block and a bitstream representation of the video that includes the current video block based on the construction; The method includes:

[0258] 25. The method according to clause 24, wherein said feature is the explicit or implicit enablement of a multiple transform selection (MTS) process.

[0259] 26. The method according to clause 24, wherein the feature is enablement of a Cross Component Linear Model (CCLM) coding mode in the current video block.

[0260] 27. The method according to clause 24, wherein the characteristic is a size of the current video block.

[0261] 28. The method according to clause 24, wherein the characteristic is a partition depth of a partitioning process applied to the current video block.

[0262] 29. The method of claim 28, wherein the partitioning process is a quad-tree (QT) partitioning process, a binary tree (BT) partitioning process, or a ternary tree (TT) partitioning process.

[0263] 30. The method according to clause 24, wherein the characteristic is a color format or color components of the current video block.

[0264] 31. The method of claim 24, wherein the above features exclude an intra prediction mode of the current video block and an index of a multiple transform selection (MTS) process.

[0265] 32. A video processing method comprising: making a decision regarding selective application of an inverse shrinking secondary transform (RST) process to the current video block based on characteristics of the current video block; and performing, based on the determination, a conversion between the current video block and a bitstream representation of the video that includes the current video block; The method includes:

[0266] 33. The method according to clause 32, wherein the feature is a coded block flag of a coding group of the current video block.

[0267] 34. The method according to clause 33, wherein the inverse RST process is not applied and the coded block flag of the top-left coding group is zero.

[0268] 35. The method of claim 33, wherein the inverse RST process is not applied and the coded block flags for the first and second coding groups in the scanning order of the current video block are zero.

[0269] 36. The method of claim 32, wherein the characteristic is a height (M) or a width (N) of the current video block.

[0270] 37. The method of clause 36, wherein the inverse RST process is not applied and (i) M=8 and N=4, or (ii) M=4 and N=8.

[0271] 38. A video processing method comprising: making a decision regarding selective application of an inverse shrinking secondary transform (RST) process to the current video block based on characteristics of the current video block; and performing, based on the determination, a conversion between the current video block and a bitstream representation of the video that includes the current video block; the bitstream representation includes side information regarding the RST, the side information being included based on a luma component or a single color coefficient of the current video block.

[0272] 39. The method of clause 38, wherein the side information is further based on a dimension of the current video block.

[0273] 40. The method of clause 38 or 39, wherein the side information is included without considering block information for the current video block.

[0274] 41. The method of any one of clauses 1-40, wherein the converting includes generating a bitstream representation from the current video block.

[0275] 42. The method of any one of clauses 1-40, wherein the transforming includes generating the current video block from a bitstream representation.

[0276] 43. An apparatus in a video system including a processor and a non-transitory memory with instructions, which when executed by the processor cause the processor to perform a method in any one of clauses 1-42.

[0277] 44. A computer program product stored on a non-transitory computer readable medium, comprising a program code for performing the method according to any one of clauses 1-42.

[0278] The second set of clauses describes particular features and aspects of the technology disclosed in the above sections, such as exemplary items 4 and 5.

[0279] 1. A video processing method comprising: determining a default intra-prediction mode for the coded video unit using a particular intra-prediction mode for a transformation between a current video block of a video unit of the video and a coded representation of the video, such that a predictive block of the current video block is generated without extrapolating neighboring pixels of the current video block along a direction; performing the conversion based on the determination; The method includes:

[0280] 2. The method of claim 1, wherein the video unit corresponds to a coding unit, a prediction unit, a coding block, or a prediction block.

[0281] 3. The method of clause 1, wherein the video units are coded using matrix-based intra prediction (MIP), which generates predictive blocks using matrix-vector multiplication.

[0282] 4. The method of claim 1, wherein the video unit is coded using an intra block copy (IBC) mode that generates a predictive block using at least a block vector that points to a video frame that contains the current video block.

[0283] 5. The method according to clause 1, wherein the video unit is coded using a palette mode that enables representing or reconstructing the current video block using a palette of representative sample values.

[0284] 6. The method of clause 1, wherein the default intra-prediction mode is determined based on a coding mode of the current video block.

[0285] 7. The method of claim 1, wherein information about the default intra-prediction mode is included in the coded representation.

[0286] 8. The method of claim 1, wherein information regarding the default intra-prediction mode is derived without being signaled.

[0287] 9. The method of claim 1, wherein the decision of the default intra prediction mode is further used in derivation of a chroma derivation mode (DM).

[0288] 10. The method of claim 1, wherein the default intra-prediction mode decision is further used to predict intra-prediction modes of other blocks of the video.

[0289] 11. The method of claim 1, wherein a determination of a default intra prediction mode assigned to a current video block of one color component is used to derive a transform set or transform index for another color component.

[0290] 12. The method of claim 1, wherein the default intra-prediction mode is stored along with the prediction mode of the block of video.

[0291] 13. The method of clause 1, wherein a default intra prediction mode is not assigned to inter-coded blocks.

[0292] 14. The method of claim 1, wherein the default intra-prediction mode is a planar intra-prediction mode.

[0293] 15. A video processing method comprising: determining, using the rules, a luma block of the video that covers a given location of a chroma block of the video; performing a conversion between the video and a coded representation of the video based on the determination; wherein the chroma blocks are represented in a representation coded using an intra prediction mode.

[0294] 16. The method according to claim 15, wherein the step of performing the transformation includes the steps of checking a prediction mode or coding mode of the luma block and fetching an intra prediction mode of the luma block.

[0295] 17. The method of claim 15, wherein the luma block includes a luma sample located to correspond to a chroma sample at a center of the chroma block.

[0296] 18. The method of claim 15, wherein the luma block includes luma samples positioned to correspond to chroma samples at the top left of the chroma block.

[0297] 19. A video processing method comprising: deriving an intra-prediction mode for a chroma block of the video based on a coding mode of a luma block corresponding to the chroma block using the rule; performing a conversion between the chroma block and a coded representation of the video based on the derived intra-prediction mode; and wherein if a coding mode of the luma block is a particular intra prediction mode in which a predictive block of the luma block is generated without extrapolating neighboring pixels of the luma block along a direction, the rule specifies using a default intra prediction mode.

[0298] 20. The method according to clause 19, wherein the rule specifies using a decoded intra-prediction mode if the coding mode of the luma block is not a particular intra-prediction mode.

[0299] 21. The method of clause 19, wherein the coding mode of the luma block corresponds to matrix-based intra prediction (MIP), which uses matrix-vector multiplication to generate the prediction block.

[0300] 22. The method of claim 21, wherein the decoded intra-prediction mode is a planar intra-prediction mode.

[0301] 23. The method of clause 19, wherein the coding mode of the luma block corresponds to an intra block copy (IBC) mode that generates a predictive block using at least a block vector that points to a video frame that includes the current video block.

[0302] 24. The method according to clause 19, wherein the coding mode of the luma block corresponds to a palette mode that enables representing or reconstructing the current video block using a palette of representative sample values.

[0303] 25. The method of any one of clauses 23 to 24, wherein the decoded intra-prediction mode is a DC intra-prediction mode.

[0304] 26. The method of any one of clauses 1 to 25, wherein the step of performing the transformation includes applying a secondary transform tool to the current video block, using the secondary transform tool to apply a forward secondary transform to an output of a forward primary transform applied to a residual of the current video block prior to quantization during encoding, or to apply an inverse secondary transform to an output of an inverse quantization of the current video block prior to applying an inverse primary transform during decoding.

[0305] 27. The method of any one of clauses 1 to 26, wherein performing the conversion includes generating a coded representation from the video.

[0306] 28. A method according to any one of clauses 1 to 26, wherein performing the transformation comprises generating a video from the coded representation.

[0307] 29. A video processing device comprising a processor configured to carry out the method according to any one of clauses 1 to 28.

[0308] 30. A computer readable medium having stored thereon program code which, when executed, causes a processor to perform the method according to any one or more of clauses 1 to 28.

[0309] The third set of clauses describes particular features and aspects of the technology disclosed in the above sections, such as exemplary items 5 and 6.

[0310] 1. A video processing method comprising: making a first determination for a chroma block of video whether a non-normal chroma intra prediction mode is applied to the chroma block of video; determining, for a luma block corresponding to the chroma block, that a luma intra prediction mode is applied to the luma block; determining whether a transform set or a transform matrix is ​​applied to the chroma block based on a luma intra prediction mode; performing a conversion between the video and a coded representation of the video according to the third determination; wherein a non-normal chroma intra prediction mode includes coding a chroma block without using values ​​of neighboring pixels extrapolated along a chroma prediction direction.

[0311] 2. The method according to clause 1, wherein the transformation set or transformation matrix is ​​used in a secondary transformation tool applied to the chroma blocks.

[0312] 3. The secondary conversion tool corresponds to the low frequency non-separable transform (LFNST) tool. During encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the chroma block before quantization, or The method according to clause 2, wherein during decoding, an inverse secondary transform is applied to the output of the inverse quantization of the chroma blocks before applying the inverse primary transform.

[0313] 4. The method according to clause 1, wherein the non-normal chroma intra prediction mode corresponds to a cross-component linear model (CCLM) prediction mode in which a linear model is used to derive a prediction value for a chroma component from other components.

[0314] 5. The method according to clause 1, wherein the luma block covers luma samples corresponding to chroma samples at predetermined positions of the chroma block.

[0315] 6. The method according to clause 1, wherein the luma block covers a luma sample that corresponds to a central chroma sample of the chroma block.

[0316] 7. The method according to clause 1, wherein the luma block covers the luma sample that corresponds to the top-left chroma sample of the current video block.

[0317] 8. The method of clause 1, wherein the luma intra prediction is a non-normal luma intra prediction mode, the non-normal luma intra prediction mode including coding the luma block without using values ​​of neighboring pixels extrapolated along the luma prediction direction.

[0318] 9. The method according to clause 8, wherein the non-normal luma intra prediction mode corresponds to a matrix-based intra prediction (MIP) that generates the predicted value using matrix-vector multiplication.

[0319] 10. The method according to clause 8, wherein the non-normal luma intra prediction mode corresponds to an intra block copy (IBC) mode in which the prediction value is generated using at least a block vector pointing to a video frame containing the luma block.

[0320] 11. The method according to clause 8, wherein the non-normal luma intra prediction mode corresponds to a palette mode enabling representing or reconstructing a luma block using a palette of representative sample values.

[0321] 12. The method of any one of clauses 8 to 11, wherein in response to the luma intra prediction mode being a non-normal luma intra prediction mode, a predetermined intra prediction mode is specified for the chroma block, and the predetermined intra prediction mode is further used instead of the non-normal chroma intra prediction mode to derive a set of transforms or a transform matrix to be applied to the chroma block.

[0322] 13. The method of claim 12, wherein the predetermined intra prediction mode is determined based on a non-normal luma intra prediction mode of the luma block.

[0323] 14. The method of claim 13, wherein in response to the non-normal luma intra prediction mode being an IBC mode or a palette mode, the predetermined intra prediction mode is determined as a DC mode.

[0324] 15. The method of claim 13, wherein in response to the non-normal luma intra prediction mode being a MIP mode, the predetermined intra prediction mode is determined as a planar mode.

[0325] 16. The method of clause 1, wherein the luma intra prediction is a normal luma intra prediction mode, the normal luma intra prediction mode including coding the luma block using values ​​of neighboring pixels extrapolated along a luma prediction direction.

[0326] 17. The method of claim 16, wherein in response to the luma intra prediction mode being a non-normal luma intra prediction mode, luma intra prediction is specified for the chroma block, and the luma intra prediction mode is further used to derive a set of transforms or a transform matrix to be applied to the chroma block, instead of the non-normal chroma intra prediction mode.

[0327] 18. A video processing method comprising: making a first determination for a chroma block of the video that a luma block corresponding to the chroma block is coded using a non-normal luma intra prediction mode; determining a second set of transformations or transformation matrices for the chroma blocks according to a rule based on the first determination; performing a conversion between the video and a coded representation of the video according to the second determination; wherein the rules specify that due to the luma block being coded using a non-normal luma intra prediction mode, one or more default modes or default transform sets associated with the chroma blocks determine a transform set or transform matrix when the chroma block is coded using a non-normal chroma intra prediction mode; A non-normal luma intra prediction mode involves coding a luma block without using values ​​of neighboring pixels extrapolated along the luma prediction direction; A method, wherein a non-normal chroma intra prediction mode includes coding a chroma block without using values ​​of neighboring pixels extrapolated along a chroma prediction direction.

[0328] 19. The method according to clause 18, wherein the non-normal chroma intra prediction mode corresponds to a cross-component linear model (CCLM) prediction mode in which a linear model is used to derive a predicted value for a chroma component from other components.

[0329] 20. The method according to clause 18, wherein the non-normal luma intra prediction mode corresponds to matrix-based intra prediction (MIP), which uses matrix-vector multiplication to generate the predicted value.

[0330] 21. The method according to clause 18, wherein the non-normal luma intra prediction mode corresponds to an intra block copy (IBC) mode in which the prediction value is generated using at least a block vector pointing to a video frame containing the luma block.

[0331] 22. The method according to clause 18, wherein the non-normal luma intra prediction mode corresponds to a palette mode that enables representing or reconstructing a luma block using a palette of representative sample values.

[0332] 23. The method of claim 18, wherein the luma block covers luma samples that correspond to chroma samples at predetermined positions of the chroma block.

[0333] 24. The method according to clause 18, wherein the luma block covers a luma sample that corresponds to a central chroma sample of the chroma block.

[0334] 25. The method according to clause 18, wherein the luma block covers the luma sample that corresponds to the top-left chroma sample of the current video block.

[0335] 26. The method of clause 18, wherein the rules specify whether to use at least one of a default transform set or a default transform matrix or to derive the transform set or transform matrix from an intra-luma prediction mode of the luma block based on a coding mode of the luma block.

[0336] 27. The method according to clause 18, wherein the rule specifies deriving the transform set or transform matrix in accordance with an intra-luma prediction mode of the luma block, due to the luma block being coded using a normal intra prediction mode that generates a predictive block of the luma block by extrapolating neighboring pixels of the luma block.

[0337] 28. The method according to clause 18, wherein the rule specifies deriving the transform set or transform matrix according to an intra-luma prediction mode of the luma block due to the luma block being coded using a block-based delta-pulse code modulation (BDPCM) mode.

[0338] 29. The method according to clause 26, wherein the rule specifies using at least one of a default transform set or a default transform matrix due to the luma block being coded using a non-normal luma intra prediction mode.

[0339] 30. A video processing method comprising: determining applicability of a second transformation tool applied to a current video block of one color component for transforming between a current video block of the video and a coded representation of the video based on at least one of: 1) a coding mode of a corresponding block of another color component; or 2) a coding mode of the current video block; performing said conversion based on the determination; and use secondary conversion tools, During encoding, a forward secondary transform is applied to the output of a forward primary transform applied to the residual of the current video block before quantization, or A method in which, during decoding, an inverse secondary transform is applied to the output of the inverse quantization of the current video block before applying the inverse primary transform.

[0340] 31. The method of claim 30, wherein the secondary transformation tool corresponds to a low frequency non-separable transformation (LFNST) tool.

[0341] 32. The method of clause 30, wherein the current video block corresponds to a chroma block and the corresponding block corresponds to a luma block that corresponds to the chroma block.

[0342] 33. The method of claim 31, wherein the determining step determines to apply a second transformation tool to the current video block because the coding mode of the corresponding block is i) a normal intra prediction mode, which involves encoding the corresponding block using values ​​of neighboring pixels extrapolated along a prediction direction, and / or ii) a block-based delta pulse code modulation (BDPCM) mode.

[0343] 34. The method of clause 31, wherein the determining step determines to disable the second transform tool for the current video block because a coding mode of the corresponding block is a non-normal intra prediction mode that includes encoding the corresponding block without using values ​​of neighboring pixels extrapolated along a prediction direction.

[0344] 35. The method according to clause 30, wherein the coding mode of the current video block corresponds to a cross-component linear model (CCLM) prediction mode, in which a predicted value of a chroma component is derived from other components using a linear model.

[0345] 36. A video processing method comprising: making a first determination, with respect to a chroma block of the video, that a luma block covering a predetermined position of the chroma block is coded using a non-normal luma intra prediction mode; making a second decision to apply a predetermined intra prediction mode to the chroma block due to the luma block being coded using a non-normal luma intra prediction mode based on the first decision; performing a conversion between the video and a coded representation of the video according to the second determination; wherein a non-normal luma intra prediction mode includes encoding the luma block without using values ​​of neighboring pixels extrapolated along the luma prediction direction.

[0346] 37. The method of claim 36, wherein a set of transforms or transform matrices used in the secondary transform tool applied to the chroma blocks are selected based on a predetermined intra prediction mode.

[0347] 38. The secondary transformation tool corresponds to the low frequency non-separable transformation (LFNST) tool, and the secondary transformation tool is used to During encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the chroma block before quantization, or 37. The method according to clause 36, wherein during decoding, an inverse secondary transform is applied to an output of the inverse quantization of the chroma blocks before applying the inverse primary transform.

[0348] 39. The method of any one of clauses 36 to 38, wherein the predetermined intra prediction mode to be applied to the chroma block is dependent on a non-normal luma intra prediction mode of the luma block.

[0349] 40. The method of any one of clauses 36 to 38, wherein the predetermined intra-prediction mode corresponds to at least one of a DC mode, a planar mode, a vertical mode, a horizontal mode, a 45 degree mode, or a 135 degree mode.

[0350] 41. The method of any one of clauses 36 to 38, wherein the non-normal luma intra prediction mode corresponds to at least one of an intra block copy (IBC) mode, a palette mode, a matrix-based intra prediction (MIP), or a block-based delta pulse code modulation (BDPCM) mode.

[0351] 42. The method of any one of clauses 36 to 38, wherein if the non-normal luma intra prediction mode corresponds to matrix-based intra prediction (MIP), the MIP mode is mapped to a specific intra prediction mode based on the dimensions of the luma block and the MIP.

[0352] 43. The method of any one of clauses 36 to 38, wherein the predetermined intra-prediction mode is selected from among a plurality of candidates.

[0353] 44. The method of any one of clauses 36 to 38, wherein the predetermined intra-prediction mode is signaled.

[0354] 45. The method of any one of clauses 36 to 38, wherein the predetermined intra prediction mode is derived.

[0355] 46. ​​A video processing method comprising: making a first determination, with respect to a chroma block of the video, that a luma block covering a predetermined position of the chroma block is coded using a normal luma intra prediction mode; making a second decision based on the first decision to derive a chroma intra prediction mode based on a normal luma intra prediction mode of the luma block; performing a conversion between the video and a coded representation of the video according to the second determination; wherein a normal luma intra prediction mode includes encoding the luma block using values ​​of neighboring pixels extrapolated along a luma prediction direction.

[0356] 47. The method of claim 46, wherein a transform set or transform matrix used in a secondary transform tool applied to a chroma block is selected based on a chroma intra prediction mode.

[0357] 48. The secondary transformation tool corresponds to the Low Frequency Non-separable Transform (LFNST) tool. During encoding, a forward secondary transform is applied to the output of a forward primary transform applied to the residual of the current video block before quantization, or 47. The method of claim 46, wherein during decoding, an inverse secondary transform is applied to an output of the inverse quantization of the current video block before applying the inverse primary transform.

[0358] 49. The method of any one of clauses 46 to 48, wherein the luma block includes a luma sample at a position corresponding to a chroma sample at the center of the chroma block.

[0359] 50. The method of any one of clauses 46 to 48, wherein the luma block includes a luma sample at a position corresponding to a chroma sample at the top left of the chroma block.

[0360] 51. A method according to any one of clauses 1 to 50, wherein the step of performing a conversion comprises the step of generating a coded representation from a video, or generating a video from a coded representation.

[0361] 52. A video processing device comprising a processor configured to perform the method described in any one of clauses 1 to 51.

[0362] 53. A computer readable storage medium storing program code which, when executed, causes a processor to carry out the method described in any one of clauses 1 to 51.

[0363] From the foregoing, it will be appreciated that, although specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, various modifications may be made without departing from the scope of the invention. Accordingly, the presently disclosed technology is not to be limited except as by the appended claims.

[0364] Implementations of the subject matter and functional operations described in this patent document can be realized in various systems, digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. Implementations of the subject matter described herein can be implemented as one or more computer program products, i.e., as one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for execution by or for controlling the operation of a data processing apparatus. A computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter affecting a machine-readable propagated signal, or one or more combinations thereof. The term "data processing unit" or "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. An apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code constituting a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0365] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in several coordinated files (e.g., a file that stores one or more modules, subprograms, or portions of code). A computer program can be deployed to run on one computer or on several computers, which can be located at one site or distributed across several sites and interconnected by a communication network.

[0366] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data to generate output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as, for example, an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0367] Processors suitable for executing a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices, e.g., magnetic, magnetic-optical, or optical disks, for storing data, or is operatively coupled to receive data from them, transfer data to them, or both. However, a computer need not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0368] It is intended that the specification, together with the drawings, be considered as exemplary only, where exemplary means exemplary. As used in this application, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0369] While this patent document contains many details, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features that are described in this patent document in the context of separate embodiments may also be implemented in a single embodiment in combination. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination, or even originally claimed as such, one or more features in a claimed combination may, in some cases, be carved out of the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0370] Similarly, although acts are depicted in a particular order in the figures, this should not be understood as requiring that such acts be performed in the particular order or sequence shown, or that all of the acts illustrated be performed, to achieve desired results.Furthermore, the division of various system components in the embodiments described in this patent document should not be understood as requiring such division in all embodiments.

[0371] Only a few implementations and examples have been described; other implementations, extensions and modifications are possible based on what is described and illustrated in this patent document.

Claims

1. 1. A method for processing video data, comprising: determining, during a first conversion between a first chroma block of a video unit of a video and a bitstream of the video, if a coding mode of a first luma block covering a luma sample at a position corresponding to a chroma sample at a center of the first chroma block is a first coding mode, a default intra prediction mode is used in a process of deriving a chroma intra prediction mode for the first chroma block; deriving the chroma intra-prediction mode for the first chroma block based on the default intra-prediction mode; and performing the first transform based on the chroma intra prediction mode; The method further comprises: making a first determination that during a second conversion between a second chroma block of the video and the bitstream, the second chroma block is coded in a cross-component linear model intra prediction mode; making a second determination that a predetermined intra prediction mode is used in a secondary transform process for the second chroma block if a coding mode of a second luma block covering a luma sample at a position corresponding to a chroma sample at a center of the second chroma block is a second coding mode; determining a transform set for the secondary transform process based on the predetermined intra-prediction mode; and performing the second transformation based on the transformation set; wherein during the secondary conversion process: In the case of encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the second chroma block prior to quantization; or A method wherein, in case of decoding, an inverse secondary transform is applied to an output of the inverse quantization of the second chroma block before applying an inverse primary transform.

2. 2. The method of claim 1, wherein when a first coding mode of the first luma block is a matrix-based intra prediction (MIP) mode that uses matrix-vector multiplication to generate predicted samples of the first luma block, the default intra prediction mode is determined as a planar intra prediction mode.

3. 2. The method of claim 1, wherein if a first coding mode of the first luma block is an intra block copy (IBC) mode that generates predictive samples for the first luma block derived from blocks of sample values ​​of the same video domain as determined by a block vector, the default intra prediction mode is determined as a DC intra prediction mode.

4. 2. The method of claim 1, wherein if a first coding mode of the first luma block is a palette mode that enables representing or reconstructing the first luma block using a palette of representative sample values, the default intra prediction mode is determined as a DC intra prediction mode.

5. 2. The method of claim 1, wherein if a coding mode of the first luma block is not a matrix-based intra prediction (MIP) mode, not an intra block copy (IBC) mode, or not a palette mode, then the luma intra prediction mode of the first luma block is used in the derivation process of the chroma intra prediction mode for the first chroma block.

6. The method of claim 1 , wherein the chroma intra prediction mode is: a first flag (cclm_mode_flag) indicating whether a cross-component linear model prediction mode is applied to the first chroma block; a second flag (cclm_mode_idx) indicating which of the INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM chroma intra prediction modes is applied to the first chroma block; a third flag (intra_chroma_pred_mode) indicating an intra prediction mode for chroma samples of the first chroma block; and a variable, denoted as lumaIntraPredMode, that is equal to the default intra prediction mode if the first luma block is coded in the first coding mode, or is equal to the luma intra prediction mode of the first luma block if the first luma block is not coded in the first coding mode; The method is derived using

7. The method of claim 6, wherein the chroma intra prediction mode is determined by using cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode, and lumaIntraPredMode as shown in Table 1 below: Table 1 【Table 1】 The method is derived as shown in.

8. 8. The method of claim 7, wherein, if a chroma format of the video unit is 4:2:2, chroma intra prediction mode X in Table 1 is mapped to chroma intra prediction mode Y according to a predetermined mapping table, and chroma intra prediction mode X in Table 1 is thereafter set equal to the chroma intra prediction mode Y.

9. 2. The method of claim 1, wherein the default intra-prediction mode is not specified for inter-coded blocks.

10. 2. The method of claim 1, wherein the transform set is selected from a predefined transform set selection table, and an index into the transform set selection table, indicated as IntraPredMode, is equal to a predefined intra prediction mode if the second luma block is coded in the second coding mode, or is equal to a luma intra prediction mode of the second luma block if the second luma block is not coded in the second coding mode.

11. 11. The method of claim 10, wherein the predetermined transformation set selection table comprises: Predefined transformation set selection table 【Table 2】 and IntraPredMode has a range of [-14,80].

12. 2. The method of claim 1, wherein the secondary transform is disabled for the second chroma block if a last non-zero coefficient is not located within a region of the second chroma block to which the secondary transform is applied.

13. The method of claim 1 , wherein the secondary transform corresponds to a Low Frequency Non-Separable Transform (LFNST) tool.

14. 14. The method of claim 1, wherein the first transform comprises encoding the first chroma block into the bitstream, and the second transform comprises encoding the second chroma block into the bitstream.

15. 14. The method of claim 1, wherein the first transform comprises decoding the first chroma block from the bitstream, and the second transform comprises decoding the second chroma block from the bitstream.

16. 1. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions, the instructions, when executed by the processor, causing the processor to: determining, during a first conversion between a first chroma block of a video unit of a video and a bitstream of the video, if a coding mode of a first luma block covering a luma sample at a position corresponding to a chroma sample at a center of the first chroma block is a first coding mode, a default intra prediction mode is used in a process of deriving a chroma intra prediction mode for the first chroma block; deriving the chroma intra-prediction mode for the first chroma block based on the default intra-prediction mode; and performing the first transform based on the chroma intra prediction mode; which instructions, when executed by the processor, further cause the processor to: making a first determination that during a second conversion between a second chroma block of the video and the bitstream, the second chroma block is coded in a cross-component linear model intra prediction mode; making a second determination that a predetermined intra prediction mode is used in a secondary transform process for the second chroma block if a coding mode of a second luma block covering a luma sample at a position corresponding to a chroma sample at a center of the second chroma block is a second coding mode; determining a transform set for the secondary transform process based on the predetermined intra-prediction mode; and performing the second transformation based on the transformation set; and during the secondary conversion process: In the case of encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the second chroma block prior to quantization; or The apparatus, wherein, in the case of decoding, an inverse secondary transform is applied to an output of the inverse quantization of the second chroma block before applying an inverse primary transform.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to: determining, during a first conversion between a first chroma block of a video unit of a video and a bitstream of the video, if a coding mode of a first luma block covering a luma sample at a position corresponding to a chroma sample at a center of the first chroma block is a first coding mode, a default intra prediction mode is used in a process of deriving a chroma intra prediction mode for the first chroma block; deriving the chroma intra-prediction mode for the first chroma block based on the default intra-prediction mode; and performing the first transform based on the chroma intra prediction mode; the instructions further cause the processor to: making a first determination that during a second conversion between a second chroma block of the video and the bitstream, the second chroma block is coded in a cross-component linear model intra prediction mode; making a second determination that a predetermined intra prediction mode is used in a secondary transform process for the second chroma block if a coding mode of a second luma block covering a luma sample at a position corresponding to a chroma sample at a center of the second chroma block is a second coding mode; determining a transform set for the secondary transform process based on the predetermined intra-prediction mode; and performing the second transformation based on the transformation set; and during the secondary conversion process: In the case of encoding, a forward secondary transform is applied to the output of the forward primary transform applied to the residual of the second chroma block prior to quantization; or A storage medium, wherein, in the case of decoding, an inverse secondary transform is applied to an output of the inverse quantization of the second chroma block before applying an inverse primary transform.

18. A method for storing a bitstream of video, comprising: determining, for a first chroma block of a video unit of a video, that a default intra prediction mode is to be used in a process of derivation of a chroma intra prediction mode for the first chroma block when a coding mode of a first luma block covering a luma sample at a position corresponding to a chroma sample at a center of the first chroma block is a first coding mode; deriving the chroma intra-prediction mode for the first chroma block based on the default intra-prediction mode; and generating the bitstream based on the chroma intra prediction mode; and storing the bitstream on a non-transitory computer readable storage medium; and the method further comprises: making a first determination that a second chroma block of the video is coded in a cross-component linear model intra prediction mode; making a second determination that a predetermined intra prediction mode is used in a secondary transform process for the second chroma block if a coding mode of a second luma block covering a luma sample at a position corresponding to a chroma sample at a center of the second chroma block is a second coding mode; determining a transform set for the secondary transform process based on the predetermined intra-prediction mode; and generating the bitstream based on the set of transformations; wherein during the secondary conversion process: A method according to claim 1, wherein a forward secondary transform is applied to an output of a forward primary transform applied to the residual of the second chroma block prior to quantization to perform the encoding.

Images