Derivation of Linear Parameters in Cross-Component Video Codecs

Abstract

A video processing method includes: for conversion between a chroma block of a video and a bitstream representation of the video, deriving parameters of a cross-component linear model by using downsampled luminance samples, the downsampled luminance samples being generated from N upper neighboring lines of a co-located luminance block of the chroma block using a downsampling filter, where N is a positive integer; and performing the conversion using a predicted chroma block generated using the cross-component linear model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 115034, filed on November 1, 2019, under applicable patent laws and / or regulations under the Paris Convention. The entire disclosure of the aforementioned application is incorporated by reference as a part of the disclosure of this application for all purposes prescribed by law. Technical Field

[0003] This document deals with video and image encoding and decoding techniques. Background Art

[0004] Digital video consumes the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video usage is expected to continue to grow. Summary of the Invention

[0005] The disclosed techniques may be used by video or image decoder or encoder embodiments for performing encoding or decoding using cross-component linear model prediction.

[0006] In one example aspect, a method for processing video is disclosed. The method includes: for converting between a chroma block of the video and a bitstream representation of the video, deriving parameters of a cross-component linear model using downsampled luma samples generated from N upper neighboring lines of a collocated luma block of the chroma block using a downsampled filter, where N is a positive integer; and performing the conversion using a predicted chroma block generated using the cross-component linear model.

[0007] In another example aspect, a method of processing video is disclosed, comprising: determining a maximum allowable block size for a video block encoded using a transform skip mode for conversion between a video region of a component of the video and a bitstream representation of the video; and performing the conversion based on the determination.

[0008] In another example aspect, a method of processing video is disclosed. The method includes performing conversion between a video comprising a video block and a bitstream representation of the video according to first and second rules, wherein a transform skip codec is used to encode a first portion of the video block, wherein a transform codec is used to encode a second portion of the video block, wherein the first rule specifies a maximum allowable block size for the first portion of the video block, and the second rule specifies a maximum allowable block size for the second portion of the video block, and wherein the maximum allowable block size for the first portion of the video block is different from the maximum allowable block size for the second portion of the video block.

[0009] In another example aspect, a method of processing video is disclosed. The method includes performing conversion between video comprising one or more chroma blocks and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule that specifies whether a syntax element indicating use of a transform skip tool is included in the bitstream representation based on a maximum allowable size of chroma blocks encoded using the transform skip tool.

[0010] In another example aspect, a method of processing video is disclosed. The method includes performing conversion between a video comprising one or more first video blocks of a first chroma component and one or more second video blocks of a second chroma component and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule that specifies use of a syntax element that jointly indicates availability of a transform skip tool for encoding and decoding the one or more first chroma blocks and the one or more second chroma blocks.

[0011] In another example aspect, the above method may be implemented by a video encoder apparatus comprising a processor.

[0012] In yet another example aspect, the methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.

[0013] These and other aspects are described further throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1A The nominal vertical and horizontal positions of the 4:2:2 luma and chroma samples in the image are shown.

[0015] Figure 1B An example of a video encoder is shown.

[0016] Figure 2 Examples of 67 intra prediction modes are shown.

[0017] Figure 3 Examples of horizontal and vertical traversal scans are shown.

[0018] Figure 4 Examples of sample point positions used for the derivation of α and β are shown.

[0019] Figure 5 An example of dividing a 4×8 sample block into two independently decodable regions is shown.

[0020] Figure 6 An example order of processing rows of pixels that maximizes throughput for a 4xN block using a vertical predictor is shown.

[0021] Figure 7 An example of a low frequency non-separable transform (LFNST) process is shown.

[0022] Figure 8 An example of adjacent chroma samples and downsampled collocated adjacent luma samples used in the derivation of CCLM parameters for 4:2:2 video is shown.

[0023] Figure 9 An example of a video processing apparatus is shown.

[0024] Figure 10 A block diagram of a video encoder is shown.

[0025] Figure 11 is a flowchart of an example of a video processing method based on some implementations of the disclosed technology.

[0026] Figure 12 is a block diagram of an example of a video processing system.

[0027] Figure 13 is a block diagram illustrating an example video encoding and decoding system.

[0028] Figure 14 is a block diagram illustrating an encoder according to some embodiments of the disclosed technology.

[0029] Figure 15 is a block diagram illustrating a decoder according to some embodiments of the disclosed technology.

[0030] Figure 16A and Figure 16B is a flowchart of an example of a video processing method based on some implementations of the disclosed technology. DETAILED DESCRIPTION

[0031] This document provides various techniques that decoders of image or video bitstreams can use to improve the quality of decompressed or decoded digital video or images. For brevity, the term "video" as used herein includes both sequences of images (traditionally referred to as video) and individual images. In addition, video encoders can also implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0032] The section headings used in this document are for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. Thus, embodiments from one section can be combined with embodiments from other sections.

[0033] 1. Brief Overview

[0034] The present invention relates to video coding and decoding technology. Specifically, it relates to cross-component linear model prediction and other coding and decoding tools in image / video coding and decoding. The present invention can be applied to existing video coding and decoding standards such as HEVC, or to a future standard (Multi-functional Video Codec). The present invention can also be applied to future video coding and decoding standards or video codecs.

[0035] 2. Introduction to Video Codec

[0036] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Codec (AVC), and H.265 / HEVC. Since H.262, video codec standards have been based on a hybrid video codec architecture that utilizes temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) established the Joint Video Experts Team (JVET) to work on the VVC standard, with the goal of reducing the bit rate by 50% compared to HEVC.

[0037] 2.1. Color Space and Chroma Subsampling

[0038] A color space, also known as a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values or color components (such as RGB). Fundamentally, a color space is an elaboration of a coordinate system and subspaces.

[0039] For video compression, the most commonly used color spaces are YCbCr and RGB.

[0040] YCbCr, Y'CbCr, or Y Pb / Cb Pr / Cr, also written as YCBCR or Y'CBCR, is a family of color spaces used as part of the color imaging pipeline in video and digital photography systems. Y' is the luma component, while CB and CR are the blue-difference and red-difference chroma components. Y' (with a prime) is distinguished from Y, which is luminance, meaning that light intensity is nonlinearly encoded based on the gamma-corrected RGB primaries.

[0041] Chroma subsampling is the practice of encoding an image at a lower resolution for chroma information than for luminance information by exploiting the fact that the human visual system is less sensitive to color differences than to luminance. 2.1.1. 4:4:4

[0043] Each of the three Y'CbCr components has the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and film post-production. 2.1.2. 4:2:2

[0045] The two chroma components are sampled at half the luminance sampling rate: the horizontal chroma resolution is halved, while the vertical chroma resolution remains unchanged. This reduces the bandwidth of the uncompressed video signal by one third, but there is almost no visual difference. Figure 1A Depicted are examples of nominal vertical and horizontal positions for a 4:2:2 color format. 2.1.3. 4:2:0

[0047] Compared to 4:1:1, horizontal sampling is doubled in 4:2:0, but vertical resolution is halved because the Cb and Cr channels are sampled only on alternate lines. The data rate remains the same. Cb and Cr are subsampled by a factor of 2 both horizontally and vertically. There are three variants of the 4:2:0 scheme, with different horizontal and vertical addressing.

[0048] In MPEG-2, Cb and Cr are co-located horizontally and located between pixels vertically (in the gap).

[0049] In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are located in the interstices, midway between alternating luminance samples.

[0050] In 4:2:0 DV, Cb and Cr are co-located horizontally and vertically on alternate lines.

[0051] Table 2-1. SubWidthC and SubHeightC values derived from chroma_format_idc and separate_colour_plane_flag

[0052]

[0053] 2.2. Encoding and decoding flow of typical video codecs

[0054] Figure 1BAn example of a VVC encoder block diagram is shown, which contains three in-loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO), and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture and, with the offset and filter coefficients signaled by the codec, reduce the mean square error between the original and reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively. ALF is located at the last processing stage of each picture and can be seen as a tool that attempts to capture and repair artifacts caused by previous stages.

[0055] Intra-mode codec with 67 intra-prediction modes

[0056] To capture arbitrary edge directions present in natural videos, the number of directional intra modes is increased from 33 used by HEVC to 65. The additional directional modes are Figure 2 These are depicted as red dashed arrows in , and planar and DC modes remain unchanged. These more dense directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction.

[0057] The traditional angular intra prediction direction is defined as from 45 degrees to -135 degrees in a clockwise direction, such as Figure 2 As shown in Figure 2. In VTM, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. The replaced modes are signaled using the original method and remapped to wide-angle mode indices after parsing. The total number of intra prediction modes remains unchanged at 67, and the intra mode codec remains unchanged.

[0058] In HEVC, each intra-coded block has a square shape, and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate the intra predictor using the DC mode. In VVC, blocks can have a rectangular shape, which makes it necessary to use a division operation for each block in the general case. To avoid the division operation for DC prediction, only the longer side is used to calculate the average for non-square blocks.

[0059] Figure 2 Examples of 67 intra prediction modes are shown.

[0060] 2.4. Inter-frame prediction

[0061] For each inter-predicted CU, the motion parameters include motion vector, reference picture index and reference picture list usage index, as well as additional information required by the new coding features of VVC for inter-prediction sample generation. The motion parameters can be signaled explicitly or implicitly. When a CU is encoded and decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increments or reference picture indexes. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs (including spatial and temporal candidates) and the additional scheduling introduced in VVC. The merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to the merge mode is the explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index and reference picture list usage flag for each reference picture list, and other required information are explicitly signaled per CU.

[0062] 2.5. Intra-block copy (IBC)

[0063] Intra Block Copy (IBC) is a tool adopted in the HEVC extension on SCC. It is well known that it significantly improves the encoding and decoding efficiency of screen content materials. Since the IBC mode is implemented as a block-level codec mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block, which has been reconstructed within the current picture. The luminance block vector of the CU encoded and decoded by IBC is integer precision. The chrominance block vector is also rounded to integer precision. When used in conjunction with AMVR, the IBC mode can switch between 1-pixel and 4-pixel motion vector precision. The CU encoded and decoded by IBC is regarded as a third prediction mode different from the intra or inter prediction mode. The IBC mode is applicable to CUs with a width and height less than or equal to 64 luminance samples.

[0064] On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks for blocks with a width or height no greater than 16 luma samples. For non-merge mode, a block vector search is first performed using a hash-based search. If the hash search does not return a valid candidate, a local search based on block matching is performed.

[0065] In hash-based search, hash key matching (32-bit CRC) between the current block and the reference block is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on a 4×4 sub-block. For larger current block sizes, a hash key is determined to be the hash key of the matching reference block when all hash keys of all 4×4 sub-blocks match the hash key in the corresponding reference position. If multiple reference blocks are found whose hash keys match the hash key of the current block, the block vector cost of each matching reference is calculated, and the one with the smallest cost is selected.

[0066] In the block matching search, the search range is set to cover both the previous and current CTUs.

[0067] At the CU level, the IBC mode is signaled with a flag and can be signaled as IBC AMVP mode or IBC skip / merge mode as shown below:

[0068] – IBC skip / merge mode: The merge candidate index is used to indicate which block vector from the list of neighboring candidate IBC coded blocks is used to predict the current block. The merge list includes spatial candidates, HMVP candidates, and pairwise candidates.

[0069] – IBC AMVP mode: Block vector differences are encoded and decoded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the upper neighbor (if IBC is used). When either neighbor is unavailable, the default block vector is used as the predictor. A flag is signaled to indicate the block vector predictor index.

[0070] 2.6. Palette Mode

[0071] For palette mode signaling, the palette mode is encoded as the prediction mode of the codec unit, that is, the prediction mode of the codec unit can be MODE_INTRA, MODE_INTER, MODE_IBC and MODE_PLT. If the palette mode is used, the pixel values in the CU are represented by a small set of representative color values. This set is called a palette. For pixels with values close to the palette colors, the palette index is signaled. For pixels with values outside the palette, the pixels are represented with an escape symbol, and the quantized pixel value is signaled directly.

[0072] To decode palette-encoded blocks, the decoder needs to decode the palette colors and indices. The palette colors are described by a palette table and encoded by the palette table codec. An escape symbol is signaled for each CU to indicate whether the escape symbol exists in the current CU. If the escape symbol exists, the palette table is incremented by 1 and the last index is assigned to the escape mode. The palette indices of all pixels in the CU form a palette index map and are encoded by the palette index map codec.

[0073] For encoding and decoding of the palette table, a palette predictor is maintained. The predictor is initialized at the beginning of each slice, where the predictor is reset to 0. For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. The reuse flag is sent using a run-length codec of zero. After that, the number of new palette entries is signaled using an exponential Golomb code of order 0. Finally, the component values of the new palette entries are signaled. After encoding the current CU, the palette predictor will be updated with the current palette, and entries from the previous palette predictor that are not reused in the current palette will be added to the end of the new palette predictor until the maximum allowed size is reached (palette padding).

[0074] To encode and decode the palette index map, use Figure 3 The horizontal and vertical traversal scans shown encode and decode the index. The scan order is explicitly signaled in the bitstream using palette_transpose_flag.

[0075] Figure 3 Examples of horizontal and vertical traversal scans are shown.

[0076] The palette index is encoded and decoded using two main palette sampling modes: 'INDEX' and 'COPY_ABOVE'. A flag is used to signal the mode, except when the top row is used when scanning horizontally, or the first column is used when scanning vertically, or before. In 'COPY_ABOVE' mode, the palette index of the sample in the row above is copied. In 'INDEX' mode, the palette index is signaled explicitly. For both 'INDEX' and 'COPY_ABOVE' modes, a running value is signaled that specifies the number of pixels encoded and decoded using the same mode.

[0077] The encoding order of the index map is as follows: First, the number of index values for the CU is signaled. Next, the actual index value for the entire CU is signaled using truncated binary codec. Both the number of indices and the index values are coded in bypass mode. This groups the bypass bins associated with the index together. Then, the palette mode (INDEX or COPY_ABOVE) and the run are signaled in an interleaved manner. Finally, the component escape values corresponding to the escape samples of the entire CU are grouped together and coded in bypass mode. After signaling the index values, an additional syntax element last_run_type_flag is signaled. This syntax element, combined with the number of indices, eliminates the need to signal the run value corresponding to the last run in the block.

[0078] In VTM, dual trees are enabled for I slices, which separate the codecs for luma and chroma. Therefore, in this proposal, the palettes are applied separately to luma (Y component) and chroma (Cb and Cr components). If dual trees are disabled, the palette is applied jointly to Y, Cb, and Cr components, the same as in HEVC.

[0079] 2.7. Cross-component linear model prediction

[0080] The cross-component linear model (CCLM) prediction mode is used in VVC. For this mode, the chrominance samples are predicted based on the reconstructed luma samples of the same CU using the following linear model:

[0081] pred C (i,j)=α·rec L ′(i,j)+β (2-1)

[0082] where pred C (i, j) represents the predicted chroma sample in the CU, and rec L (i, j) represents the downsampled and reconstructed luminance sample of the same CU.

[0083] Figure 4 An example of the positions of the left and upper samples involved in the LM mode and the samples of the current block is shown.

[0084] Figure 4 An example of the positions of the sample points used to derive α and β is shown.

[0085] In addition to the upper template and the left template being used together to calculate the linear model coefficients in the LM mode, they can also be used alternately in two other LM modes, called LM_A and LM_L modes. In the LM_A mode, only the upper template is used to calculate the linear model coefficients. In order to obtain more samples, the upper template is expanded to (W+H). In the LM_L mode, only the left template is used to calculate the linear model coefficients. In order to obtain more samples, the left template is expanded to (H+W). For non-square blocks, the upper template is expanded to W+W and the left template is expanded to H+H.

[0086] The CCLM parameters (α and β) are derived from up to four adjacent chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block size is W×H, W' and H' are set to

[0087] – When LM mode is applied, W'=W, H'=H

[0088] – When LM-A mode is applied, W'=W+H;

[0089] – When LM-L mode is applied, H'=H+W;

[0090] The upper adjacent position is represented as S[0, -1]...S[W'-1, -1], and the left adjacent position is represented as S[-1, 0]...S[-1, H'-1]. Then these four sample points are selected as

[0091] – When LM mode is applied and both upper and left adjacent samples are available, S[W' / 4, -1], S[3W' / 4, -1], S[-1, H' / 4], S[-1, 3H' / 4];

[0092] – When LM-A mode is applied or only upper adjacent samples are available, S[W' / 8, -1], S[3W' / 8, -1], S[5W' / 8, -1], S[7W' / 8, -1];

[0093] – When LM-L mode is applied or only left adjacent samples are available, S[-1, H' / 8], S[-1, 3H' / 8], S[-1, 5H' / 8], S[-1, 7H' / 8];

[0094] The four adjacent luminance samples at the selected position are downsampled and compared four times to find the two smaller values: x 0 A and x 1 A , and two larger values: x 0 B and x 1 BTheir corresponding chrominance sample values are represented as y 0 A 、y 1 A.y 0 B and y 1 B Then x A 、x B 、y A and y B It can be deduced as:

[0095] X a =(x 0 A +x 1 A +1)>>1;X b =(x 0 B +x 1 B +1)>>1;Y a =(y 0 A +y 1 A +1)>>1;

[0096] Y b =(y 0 B +y 1 B +1)>>1 (2-2)

[0097] Finally, the linear model parameters α and β are obtained according to the following equations.

[0098]

[0099] β=Y b -α·X b (2-4)

[0100] The division operation to calculate the parameter α is implemented using a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated using a 4-bit significand and an exponent. Therefore, for 16 values of the significand, the table of 1 / diff is simplified to 16 elements, as follows:

[0101] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (2-5)

[0102] This will help reduce the computational complexity and the memory size required to store the required tables.

[0103] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2 to 1 downsampling ratio in both the horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0104]

[0105]

[0106] Note that when the upper reference line is at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to form the downsampled luma samples.

[0107] This parameter calculation is performed as part of the decoding process, not just an encoder search operation. Therefore, no syntax is used to pass the α and β values to the decoder.

[0108] For chroma intra mode coding and decoding, chroma intra mode coding and decoding allows a total of 8 intra modes. These modes include five traditional intra modes and three cross-component linear model modes (LM, LM_A and LM_L). Table 2-2 shows the chroma mode signaling and derivation process. Chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0109] Table 2-2. Derivation of chroma prediction modes from luma modes when CCLM is enabled

[0110]

[0111] 2.8. Block Differential Pulse Codec Modulation Codec

[0112] BDPCM was proposed in JVET-M0057. Since the shape of the horizontal (respectively, vertical) predictor uses the left (A) (respectively, top (B)) pixel of the current pixel for prediction, the most throughput-efficient way to process the block is to process all pixels of a column (respectively, a line) in parallel and process these columns (respectively, these lines) sequentially. To increase throughput, we introduce the following procedure: when the predictor selected on the block is vertical, the block of width 4 is divided into two halves with a horizontal boundary, and when the predictor selected on the block is horizontal, the block of height 4 is divided into two halves with a vertical boundary.

[0113] When a block is partitioned, samples from one region are not allowed to use pixels from another region to compute a prediction: if this happens, the predicted pixel is replaced by a reference pixel in the prediction direction. Figure 5 The different positions of the current pixel X in the vertically predicted 4×8 block are shown in FIG.

[0114] Figure 5 An example of dividing a 4×8 sample block into two independently decodable regions is shown.

[0115] Due to this feature, it is now possible to process a 4×4 block in 2 cycles, and a 4×8 or 8×4 block in 4 cycles, and so on. Figure 6 shown.

[0116] Figure 6 An example order of processing rows of pixels that maximizes throughput for a 4xN block using a vertical predictor is shown.

[0117] Table 2-3 summarizes the number of cycles required to process a block, depending on the block size. It is easy to see that any block with two dimensions greater than or equal to 8 can be processed with 8 pixels or more per cycle.

[0118] Table 2-3. Worst-case throughput for blocks of size 4xN and Nx4

[0119]

[0120] 2.9. Quantized Residual Domain BDPCM

[0121] JVET-N0413 proposes quantized residual domain BDPCM (hereinafter referred to as RBDPCM). Intra-frame prediction is performed on the entire block by copying samples in the prediction direction (horizontal or vertical prediction) similar to intra-frame prediction. The residual is quantized, and the delta between the quantized residual and its predictor (horizontal or vertical) quantized value is encoded and decoded.

[0122] For a block of size M (rows) × N (columns), let r i,j , 0≤i≤M-1,0≤j≤N-1 is the prediction residual after performing intra prediction horizontally (copying the left neighboring pixel values line by line across the prediction block) or vertically (copying the top neighboring line to every line in the prediction block) using the unfiltered samples from the upper or left block boundary samples. Let Q(r i,j ), 0≤i≤M-1,0≤j≤N-1 represents the residual r i,j The quantized version of i,j is the difference between the original block and the predicted block value. Then, block DPCM is applied to the quantized residual samples, producing a The modified M×N array When vertical BDPCM is signaled:

[0123]

[0124] For horizontal prediction, similar rules apply, and the residual quantization samples are obtained by

[0125] Residual quantization samples is sent to the decoder.

[0126] At the decoder side, the above calculation is reversed to produce Q(r i,j ),0≤i≤M-1,0≤j≤N-1. For vertical prediction,

[0127]

[0128] For the horizontal case,

[0129]

[0130] Inverse quantization residual Q -1 (Q(r i,j )) is added to the intra block prediction value to produce the reconstructed sample value.

[0131] The main benefit of this approach is that the inverse DPCM transform can be done dynamically during coefficient analysis by simply adding predictors as the coefficients are solved, or it can be performed after analysis.

[0132] Transform skipping is always used in quantized residual domain BDPCM.

[0133] 2.10. VVC Multiple Transform Set (MTS)

[0134] In VTM, large block size transforms up to 64×64 are supported, which is mainly useful for higher resolution video, such as 1080p and 4K sequences. For transform blocks with a size (width or height, or both width and height) equal to 64, the high-frequency transform coefficients are zeroed, so that only the low-frequency coefficients are retained. For example, for an M×N transform block, where M is the block width and N is the block height, when M is equal to 64, only the left 32 columns of transform coefficients are retained. Similarly, when N is equal to 64, only the top 32 rows of transform coefficients are retained. When transform skip mode is used for large blocks, the entire block is used without zeroing any values. VTM also supports a configurable maximum transform size in SPS, so the encoder can flexibly choose a transform size of up to 16 lengths, 32 lengths, or 64 lengths according to the needs of the specific implementation.

[0135] In addition to the DCT-II already adopted in HEVC, the Multiple Transform Selection (MTS) scheme is used for residual coding of inter- and intra-frame codec blocks. It uses multiple selected transforms from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table 2-4 shows the selected DST / DCT basis functions.

[0136] Table 2-4. Transform basis functions of DCT-II / VIII and DSTVII for N-point input

[0137]

[0138] To maintain the orthogonality of the transform matrix, the transform matrix is quantized more accurately than the HEVC transform matrix. To keep the intermediate values of the transform coefficients within 16 bits, all coefficients will have 10 bits after horizontal and vertical transforms.

[0139] To control the MTS scheme, separate enable flags are specified at the SPS level for intra and inter frames. When MTS is enabled at the SPS, a CU-level flag is signaled to indicate whether MTS is applied. Here, MTS applies only to luma. The MTS CU-level flag is signaled when the following conditions are met.

[0140] - Both width and height are less than or equal to 32

[0141] -CBF flag equals one

[0142] If the MTS CU flag is equal to zero, DCT2 is applied in both directions. However, if the MTS CU flag is equal to 1, two other flags are additionally signaled to indicate the transform type used for the horizontal and vertical directions, respectively. The transform and signaling mapping table is shown in Table 2-5. The transform selection using unified ISP and implicit MTS is removed by removing the intra mode and block shape dependencies. If the current block is in ISP mode, or if the current block is an intra block and both intra and inter explicit MTS are turned on, only DST7 is used for both horizontal and vertical transform cores. For transform matrix precision, an 8-bit primary transform core is used. Therefore, all transform cores used in HEVC remain unchanged, including 4-point DCT-2 and DST-7, 8-point, 16-point and 32-point DCT-2. In addition, other transform cores including 64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7 and DCT-8 use 8-bit primary transform cores.

[0143] Table 2-5. Transformation and signaling mapping table

[0144]

[0145] To reduce the complexity of large-size DST-7 and DCT-8, high-frequency transform coefficients are zeroed for DST-7 and DCT-8 blocks with size (width or height, or both) equal to 32. Only coefficients in the 16×16 low-frequency region are retained.

[0146] As in HEVC, the residual of a block can be coded in transform skip mode. To avoid syntax coding redundancy, the transform skip flag is not signaled when the CU level MTS_CU_flag is not equal to zero. The block size restrictions for transform skip are the same as those for MTS in JEM4, which indicates that transform skip applies to a CU when both the block width and height are equal to or less than 32. Note that when LFNST or MIP is activated for the current CU, the implicit MTS transform is set to DCT2. In addition, when MTS is enabled for an inter-coded block, implicit MTS can still be enabled.

[0147] 2.11. Low-Frequency Non-Separable Transform (LFNST)

[0148] In VVC, LFNST (Low Frequency Non-separable Transform), known as a simplified secondary transform, is applied between the forward primary transform and quantization (at the codec) and between dequantization and inverse primary transform (at the decoder side), as shown in Figure 7 As shown in Figure 2. In LFNST, a 4×4 non-separable transform or an 8×8 non-separable transform is applied depending on the block size. For example, 4×4 LFNST is applied for small blocks (i.e., min(width, height) < 8), and 8×8 LFNST is applied for larger blocks (i.e., min(width, height) > 4).

[0149] Figure 7 An example of a low frequency non-separable transform (LFNST) process is shown.

[0150] The application of the non-separable transform used in LFNST is described as follows using the input as an example. To apply 4×4 LFNST, the 4×4 input block X

[0151]

[0152] First, it is represented as a vector

[0153]

[0154] The non-separable transformation is calculated as in Indicates the transform coefficient vector, and T is the 16×16 transform matrix. 16×1 coefficient vector Subsequently, the scan order (horizontal, vertical, or diagonal) of the block is reorganized into 4×4 blocks. Coefficients with smaller indices are placed in the 4×4 coefficient block together with smaller scan indices.

[0155] 2.11.1. Simplified Inseparable Transform

[0156] The LFNST (Low-Frequency Non-Separable Transform) applies the non-separable transform based on the direct matrix multiplication method, such that it is implemented in a single pass without multiple iterations. However, the dimension of the non-separable transform matrix needs to be reduced to minimize the computational complexity and the memory space for storing the transform coefficients. Therefore, the simplified non-separable transform (or RST) method is used in the LFNST. The main idea of the simplified non-separable transform is to map an N-dimensional vector (for an 8×8 NSST, N is usually equal to 64) to an R-dimensional vector in a different space, where N / R (R < N) is the simplification factor. Thus, instead of an N×N matrix, the RST matrix becomes an R×N matrix, as follows:

[0157]

[0158] Where the R rows of the transform are the R basis of the N-dimensional space. The inverse transform matrix of RT is the transpose of its forward transform. For 8×8 LFNST, a simplification factor of 4 is applied, and the 64×64 direct matrix, which is the size of the conventional 8×8 non-separable transform matrix, is simplified to a 16×48 direct matrix. Therefore, a 48×16 inverse RST matrix is used on the decoder side to generate the core (primary) transform coefficients in the 8×8 upper left region. When a 16×48 matrix is applied instead of a 16×64 matrix with the same transform set configuration, each matrix takes 48 input data from three 4×4 blocks in the upper left 8×8 block except for the lower right 4×4 block. With the help of the reduced dimensionality, the memory usage for storing all LFNST matrices is reduced from 10KB to 8KB, with a reasonable performance drop. To reduce complexity, LFNST is restricted to being applicable only when all coefficients outside the first coefficient subgroup are invalid. Therefore, when LFNST is applied, all primary transform coefficients must be zero. This allows for LFNST index signaling to be adjusted for the last significant position, thus avoiding the extra coefficient scan required in current LFNST designs, which is only needed to check for significant coefficients at specific positions. The worst-case treatment of LFNST (in terms of per-pixel multiplications) limits the non-separable transforms for 4×4 and 8×8 blocks to 8×16 and 8×48 transforms, respectively. In these cases, for other sizes less than 16, the last significant scan position must be less than 8 when applying LFNST. For blocks with shapes of 4×N and N×4 with N>8, the proposed restriction means that LFNST is now applied only once, and only to the top-left 4×4 region. Since all primary-only coefficients are zero when LFNST is applied, the number of operations required for the primary transform is reduced in this case. From a codec perspective, coefficient quantization is significantly simplified when testing the LFNST transform. Maximum rate-distortion-optimized quantization must be performed for the first 16 coefficients (in scan order), with the remaining coefficients forced to zero.

[0159] 2.11.2. LFNST Transform Selection

[0160] There are a total of 4 transform sets in LFNST, and each transform set uses 2 inseparable transform matrices (kernels). The mapping from intra prediction modes to transform sets is predefined, as shown in Table 2-6. If one of the three CCLM modes (INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM) is used for the current block (81 <= predModeIntra <= 83), transform set 0 is selected for the current chroma block. For each transform set, the selected inseparable secondary transform candidate is further specified by an explicitly signaled LFNST index. After the transform coefficients, the index is signaled once in the bitstream for each intra CU.

[0161] Table 2-6. Transformation selection table

[0162] Intra prediction mode Transform Set Index IntraPredMode<0 1 0<=IntraPredMode<=1 0 2<=IntraPredMode<=12 1 13<=IntraPredMode<=23 2 24<=IntraPredMode<=44 3 45<=IntraPredMode<=55 2 56<=IntraPredMode<=80 1 81<=IntraPredMode<=83 0

[0163] 2.11.3.LFNST Index Signaling and Interaction with Other Tools

[0164] Since LFNST is restricted to only apply when all coefficients outside the first coefficient subgroup are invalid, the LFNST index encoding depends on the position of the last significant coefficient. In addition, the LFNST index is context-encoded, but does not depend on the intra prediction mode, and only the first bin is context-encoded. In addition, LFNST applies to intra CUs in both intra and inter slices, and to both luma and chroma. If dual-tree is enabled, the LFNST indexes for luma and chroma are signaled separately. For inter slices (dual-tree disabled), a single LFNST index is signaled and used for luma and chroma.

[0165] When ISP mode is selected, LFNST is disabled and the RST index is not signaled, because even if RST is applied to every feasible partition, the performance improvement is negligible. In addition, disabling RST for the ISP prediction residual can reduce coding complexity. When MIP mode is selected, LFNST is also disabled and the index is not signaled.

[0166] Considering that large CUs larger than 64×64 are implicitly split (TU tiling) due to the existing maximum transform size limit (64×64), LFNST index search may increase the data buffer by four times for a certain number of decoding pipeline stages. Therefore, the maximum size allowed by LFNST is limited to 64×64. Please note that LFNST is only enabled in the DCT case.

[0167] 2.12. Transform skipping for chroma

[0168] Chroma Transform Skip (TS) was introduced in VVC. The motivation was to unify TS and MTS signaling between luma and chroma by relocating transform_skip_flag and mts_idx to the residual_coding section. A context model was added for the chroma TS. For mts_idx, no context model was used and the binarization was changed. In addition, when a chroma TS is used, the TS residual codec is also applied.

[0169] Semantics

[0170] transform_skip_flag[x0][y0][cIdx] specifies whether a transform is applied to the associated luma transform block. Array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the transform block under consideration relative to the top-left luma sample of the slice. transform_skip_flag[x0][y0][cIdx] equal to 1 specifies that no transform is applied to the current luma transform block. Array index cIdx specifies the indicator of the color component; it is equal to 0 for luma, 1 for Cb, and 2 for Cr. transform_skip_flag[x0][y0][cIdx] equal to 0 specifies that the decision of whether to apply a transform to the current transform block depends on other syntax elements. When transform_skip_flag[x0][y0][cIdx] is not present, it is inferred to be equal to 0.

[0171] 2.13. BDPCM for Chroma

[0172] In addition to chroma TS support, BDPCM is added for chroma components. If sps_bdpcm_enable_flag is 1, another syntax element, sps_bdpcm_chroma_enable_flag, is added to the SPS. These flags have the following behavior, as indicated in Table 2-7.

[0173] Table 2-7. SPS flags for luminance and chrominance BDPCM

[0174]

[0175] When BDPCM is only available for luma, the current behavior is unchanged. When BDPCM is also available for chroma, a bdpcm_chroma_flag is sent for each chroma block. This indicates whether BDPCM is used on the chroma block. When enabled, BDPCM is used for both chroma components and an additional bdpcm_dir_chroma flag is encoded to indicate the prediction direction used for both chroma components.

[0176] The deblocking filter is deactivated on the boundary between two Block-DPCM blocks, since neither block uses the transform level that is usually responsible for blocking artifacts. This deactivation occurs independently for the luminance and chrominance components.

[0177] 3. Examples of technical problems solved by the disclosed solutions

[0178] The current design of linear parameter derivation in CCLM and TS has the following problems:

[0179] 1. For non-4:4:4 color formats, the derivation of linear parameters in CCLM involves adjacent chroma samples and downsampled co-located adjacent luminance samples. Figure 8 As shown in Figure 1, in the current VVC, when the nearest line is not on a CTU boundary, for 4:2:2 video, the second line above the current block is used to derive the downsampled co-located adjacent top luma sample. However, for 4:2:2 video, the vertical resolution remains unchanged. Therefore, there is a phase shift between the downsampled co-located adjacent top luma sample and the adjacent chroma samples.

[0180] Figure 8 An example showing adjacent chroma samples and downsampled collocated adjacent luma samples used in the derivation of CCLM parameters for 4:2:2 video.

[0181] 2. In the current VVC, the same maximum block size is used for the conditional check of the signaling of the luma transform skip flag and the signaling of the chroma transform skip flag. This design does not take the color format into consideration, which is not desirable.

[0182] a. A similar problem exists for the signaling of the luma BDPCM flag and the signaling of the chroma BDPCM flag, where the same maximum block size is used in the conditional check.

[0183] 4. List of Examples and Techniques

[0184] The following list should be considered as examples to explain general concepts. These items should not be interpreted in a narrow sense. Furthermore, these items can be combined in any way.

[0185] In this document, the term "CCLM" refers to a codec that uses cross-color component information to predict the samples / residuals of the current color component, or to derive a reconstruction of the samples in the current color component. It is not limited to the CCLM technology described in VVC.

[0186] Derivation of linear parameters in CCLM

[0187] 1. When deriving CCLM parameters for a chroma block, one or more upper neighboring lines of its co-located luma block may be used to derive its downsampled co-located neighboring top luma samples.

[0188] a. In one example, when the current chroma block is not at the top CTU boundary, the top nearest line of the co-located luma block can be used instead of the second line above to derive the downsampled co-located adjacent top luma sample.

[0189] i. In one example, one and the same downsampling filter can be used to derive downsampled collocated adjacent top luma samples and downsampled collocated adjacent left luma samples.

[0190] 1) For example, a [1 2 1] filter can be used. More specifically, pDsY[x] = (pY[2*x–1][-1] + 2*pY[2*x][-1] + pY[2*x+1][-1] + 2) >> 2, where pY[2*x][-1], pY[2*x–1][-1], and pY[2*x+1][-1] are the luma samples from the nearest upper neighboring line, and pDstY[x] represents the downsampled co-located adjacent top luma sample.

[0191] ii. In one example, different downsampling filters (eg, different filter taps / different filter coefficients) may be used to derive the downsampled collocated adjacent top luma samples and the downsampled collocated adjacent left luma samples.

[0192] iii. In one example, one and the same downsampling filter may be used to derive downsampled co-located adjacent top luma samples regardless of the location of the chroma blocks (eg, the chroma blocks may or may not be located at the top CTU boundary).

[0193] iv. In one example, the above method can be applied only to images / videos in 4:2:2 format.

[0194] b. In one example, when the current chroma block is not at the top CTU boundary, the upper-neighboring luma samples, which include the nearest upper line of the collocated luma block but do not include the second line above, can be used to derive the downsampled collocated upper-neighboring luma samples.

[0195] c. In one example, the derivation of downsampled co-located adjacent top luma samples may depend on samples located in multiple lines.

[0196] i. In one example, it may depend on both the second nearest line and the nearest line above the co-located luma block.

[0197] ii. In one example, for different color formats (eg, 4:2:0 and 4:2:2), one and the same downsampling filter can be used to derive downsampled co-located adjacent top luma samples.

[0198] 1) In one example, a 6-tap filter (eg, [1 2 1; 1 2 1]) may be utilized.

[0199] a) In one example, the downsampled co-located adjacent top luma samples can be derived as: pDsY[x] = (pY[2*x–1][-2] + 2*pY[2*x][-2] + pY[2*x+1][-2] + pY[2*x–1][-1] + 2*pY[2*x][-1] + pY[2*x+1][-1] + 4)>>3, where pY is the corresponding luma sample and pDstY[x] represents the downsampled co-located adjacent top luma sample.

[0200] b) Alternatively, also when sps_cclm_colocated_chroma_flag is equal to 0, the above method may be applied.

[0201] 2) In one example, a 5-tap filter may be utilized (eg, [0 1 0; 1 4 1; 0 1 0]).

[0202] a) In one example, the downsampled co-located adjacent top luma samples may be derived as: pDsY[x] = (pY[2*x][-2] + pY[2*x-1][-1] + 4*pY[2*x][-1] + pY[2*x+1][-1] + pY[2*x][0] + 4)>>3, where pY is the corresponding luma sample and pDstY[x] represents the downsampled co-located adjacent top luma sample.

[0203] b) Alternatively, also when sps_cclm_colocated_chroma_flag is equal to 1, the above method may be applied.

[0204] iii. In one example, the above method can only be applied to images / videos in 4:2:2 format.

[0205] Maximum block size for transform skip codec blocks (e.g., where transform_skip_flag is equal to 1, or BDPCM or other modes that bypass the transform process / use the identity transform)

[0206] 2. The maximum block size of a transform skip codec block can depend on the color component. The maximum block size of the transform skip codec blocks for luma and chroma are represented by MaxTsSizeY and MaxTsSizeC respectively.

[0207] a. In one example, the maximum block sizes for luma and chroma components may be different.

[0208] b. In one example, the maximum block sizes of the two chroma components may be different.

[0209] c. In one example, the maximum block size of luma and chroma components or the maximum block size of each color component may be signaled separately.

[0210] i. In one example, MaxTsSizeC / MaxTsSizeY may be signaled at the sequence level / picture level / slice level / slice group level, such as in the sequence header / picture header / SPS / VPS / DPS / PPS / APS / slice header / slice group header.

[0211] ii. In one example, MaxTsSizeY may be signaled conditionally, such as depending on whether transform skip is enabled / BDPCM is enabled.

[0212] iii. In one example, MaxTsSizeC may be signaled conditionally, such as depending on color format / transform skip enabled / BDPCM enabled.

[0213] iv. Alternatively, predictive coding between the maximum block sizes of luma and chroma components can be utilized.

[0214] d. In one example, MaxTsSizeC may depend on MaxTsSizeY.

[0215] i. In one example, MaxTsSizeC may be set equal to MaxTsSizeY.

[0216] ii. In one example, MaxTsSizeC may be set equal to MaxTsSizeY / N (N is an integer). For example, N=2.

[0217] e. In one example, MaxTsSizeC may be set according to the chroma subsampling rate.

[0218] i. In one example, MaxTsSizeC is set equal to MaxTsSizeY>>SubWidthC, where SubWidthC is defined in Table 2-1.

[0219] ii. In one example, MaxTsSizeC is set equal to MaxTsSizeY>>SubHeightC, where SubHeightC is defined in Table 2-1.

[0220] iii. In one example, MaxTsSizeC is set equal to MaxTsSizeY>>max(SubWidthC, SubHeightC).

[0221] iv. In one example, MaxTsSizeC is set equal to MaxTsSizeY>>min(SubWidthC, SubHeightC).

[0222] 3. The maximum allowed block size width and height of the transform codec block can be defined differently.

[0223] a. In one example, the maximum allowed block size width and height may be signaled separately.

[0224] b. In one example, the maximum allowed block size width and height of the chroma transform codec block can be denoted as MaxTsSizeWC and MaxTsSizeHC, respectively. MaxTsSizeWC can be set equal to MaxTsSizeY>>SubWidthC, and MaxTsSizeHC can be set equal to MaxTsSizeY>>SubHeightC.

[0225] i. In one example, MaxTsSizeY is the one defined in bullet 2.

[0226] 4. Whether to signal the transform skip flag of the chroma block (e.g., transform_skip_flag[x0][y0][1] and / or transform_skip_flag[x0][y0][2]) may depend on the maximum allowed size of the chroma transform skip codec block.

[0227] a. In one example, the chroma transform skip flag may be conditionally signaled based on the following conditions:

[0228] i. In one example, the condition is: tbW is less than or equal to MaxTsSizeC, tbH is less than or equal to MaxTsSizeC, where tbW and tbH are the width and height of the current chroma block.

[0229] 1) In one example, MaxTsSizeC is defined as in bullet points 2-3.

[0230] ii. In one example, the condition is: tbW is less than or equal to MaxTsSizeWC, tbH is less than or equal to MaxTsSizeHC, where tbW and tbH are the width and height of the current chroma block, and MaxTsSizeWC and MaxTsSizeHC represent the maximum allowed block size width and height of the chroma transform skip codec block, respectively.

[0231] 1) In one example, MaxTsSizeWC and / or MaxTsSizeHC may be defined as in bullet point 3.

[0232] b. In one example, the above method can be applied to the encoding and decoding of the chroma BDPCM flag (eg, intra_bdpcm_chroma_flag) by replacing "transform skip" with "BDPCM".

[0233] 5. Instead of encoding and decoding two TS flags for two chrominance color components, it is proposed to use one syntax to indicate the use of TS for two chrominance color components.

[0234] a. In one example, instead of encoding and decoding transform_skip_flag[x0][y0][1] and / or transform_skip_flag[x0][y0][2], a single syntax element (eg, TS_chroma_flag) may be encoded and decoded.

[0235] i. In one example, the value of a single syntax element is a binary value.

[0236] 1) Alternatively, furthermore, two chroma component blocks share the same on / off control of the TS mode according to a single syntax element.

[0237] a) In one example, a single syntax element with a value equal to 0 indicates that TS is disabled for both.

[0238] b) In one example, a single syntax element with a value equal to 0 indicates that TS is enabled for both.

[0239] 2) Alternatively, in addition, the second syntax element may be further signaled based on whether the value of the single syntax element is equal to K (eg, K=1).

[0240] a) In one example, a single syntax element with a value equal to 0 indicates that TS is disabled for both; a single syntax element with a value equal to 0 indicates that TS is enabled for at least one of the two chroma components.

[0241] b) The second syntax element may be used to indicate to which of the two chroma components the TS applies and / or that the TS applies to both chroma components.

[0242] ii. In one example, the value of a single syntax element is a non-binary value.

[0243] 1) In one example, a single syntax element with a value equal to K0 indicates that TS is disabled for both

[0244] 2) In one example, a single syntax element with a value equal to K1 indicates that TS is enabled for the first chroma color component and disabled for the second color component.

[0245] 3) In one example, a single syntax element with a value equal to K2 indicates that TS is disabled for a first chroma color component and enabled for a second color component.

[0246] 4) In one example, a single syntax element with a value equal to K3 indicates that TS is enabled for both.

[0247] 5) In one example, a single syntax element can be encoded or decoded using fixed-length, unary, truncated unary, k-order EG binarization methods.

[0248] iii. In one example, the single syntax element and / or the second syntax element may be context coded or bypass coded.

[0249] General Statement

[0250] 6. Whether and / or how to apply the above disclosed method can be signaled at the sequence level / picture level / slice level / slice group level, such as in the sequence header / picture header / SPS / VPS / DPS / PPS / APS / slice header / slice group header.

[0251] 7. Whether and / or how to apply the above disclosed method may depend on codec information, such as color format, single / dual tree partitioning.

[0252] 5. Examples

[0253] This section shows example embodiments and how the current VVC standard is modified to describe these embodiments. Changes to the VVC specification are highlighted in bold and italics. Deleted text is marked with double brackets (e.g., [[a]] indicates the deletion of the character "a").

[0254] Example 1

[0255] The working draft specified in JVET-P2001-v9 may be changed as follows.

[0256] 8.4.5.2.13 Specification of INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM intra prediction modes

[0257] …

[0258] 3. The downsampled co-located luminance samples pDsY[x][y], where x = 0..nTbW-1 and y = 0..nTbH-1, are derived as follows:

[0259] – If SubWidthC and SubHeightC are both equal to 1, the following applies:

[0260] –pDsY[x][y], where x = 1..nTbW-1, y = 1..nTbH-1 is derived as follows:

[0261] pDstY[x][y]=pY[x][y] (8-159)

[0262] – Otherwise, the following applies:

[0263] – The one-dimensional filter coefficient arrays F1 and F2, and the two-dimensional filter coefficient arrays F3 and F4 are specified as follows.

[0264] F1[i]=1, where i=0..1 (8-160)

[0265] F2[0]=1,F2[1]=2,F2[2]=1 (8-161)

[0266] F3[i][j]=F4[i][j]=0, where i=0..2, j=0..2 (8-162)

[0267] – If SubWidthC and SubHeightC are both equal to 2, then the following applies:

[0268] F1[0]=1,F1[1]=1 (8-163)

[0269] F3[0][1]=1,F3[1][1]=4,F3[2][1]=1,F3[1][0]=1,

[0270] F3[1][2]=1 (8-164)

[0271] F4[0][1]=1, F4[1][1]=2, F4[2][1]=1 (8-165)

[0272] F4[0][2]=1, F4[1][2]=2, F4[2][2]=1 (8-166)

[0273] – Otherwise, the following applies:

[0274] F1[0]=2,F1[1]=0 (8-167)

[0275] F3[1][1]=8 (8-168)

[0276] F4[0][1]=2, F4[1][1]=4, F4[2][1]=2, (8-169)

[0277] …

[0278] 5. When numSampT is greater than 0, the selected adjacent top chroma sample pSelC[idx] is set equal to p[pickPosT[idx cntL]][-1], where idx = cntL..cntL+cntT-1, and the downsampled adjacent top luma sample pSelDsY[idx], where idx = 0..cntL+cntT-1 is specified as follows:

[0279] …

[0280] – Otherwise (sps_cclm_colocated_chroma_flag is equal to 0), the following applies:

[0281] – If x is greater than 0, the following applies:

[0282] – If bCTUboundary is equal to FALSE, the following applies:

[0283]

[0284]

[0285] – Otherwise (bCTUboundary equals TRUE), the following applies:

[0286]

[0287] – Otherwise (x is equal to 0), then the following applies:

[0288] – If availTL equals TRUE and bCTUboundary equals FALSE, the following applies:

[0289] pSelDsY[idx]=(F4[0][1]*pY[-1][[[-2]]-1]+F4[0][2]*pY[-1][[[-1]]-2]+F4[1][1]*pY[0][[[-2 ]]-1]+F4[1][2]*pY[0][[[-1]]-2]+F4[2][1]*pY[1][[[-2]]-1]+F4[2][2]*pY[1][[[-1]]-2]+4)>>3 (8-195)

[0290] – Otherwise, if availTL equals TRUE and bCTUboundary equals TRUE, the following applies:

[0291] pSelDsY[idx]=(F2[0]*pY[-1][-1]+F2[1]*pY[0][-1]+F2[2]*pY[1][-1]+2)>>2(8-196)

[0292] Otherwise, if availTL equals FALSE and bCTUboundary equals FALSE, the following applies:

[0293] pSelDsY[idx]=(F1[1]*pY[0][-2]+F1[0]*pY[0][-1]+1)>>1 (8-197)

[0294] – Otherwise (availTL equals FALSE and bCTUboundary equals TRUE), the following applies:

[0295] pSelDsY[idx]=pY[0][-1] (8-198)

[0296] …

[0297] 5.2. Example 2

[0298] This embodiment shows an example of chroma transform skip flag encoding and decoding according to the maximum allowed transform skip encoding and decoding block size. The working draft specified in JVET-P2001-v9 can be modified as follows.

[0299] 7.3.9.10 Transformation unit syntax

[0300] …

[0301]

[0302]

[0303] …

[0304] 5.3. Example 3

[0305] This embodiment shows an example of chroma BDPCM flag codec based on the maximum allowed chroma transform skip codec block size. The working draft specified in JVET-P2001-v9 can be modified as follows.

[0306] 7.3.9.5 Codec unit syntax

[0307] …

[0308]

[0309]

[0310] …

[0311] Figure 9 is a block diagram of a video processing device 900. The device 900 can be used to implement one or more of the methods described herein. The device 900 can be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 900 may include one or more processors 902, one or more memories 904, and video processing hardware 906. The processor(s) 902 can be configured to implement one or more of the methods described in this document. The memory(s) 904 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 906 can be used to implement some of the techniques described in this document (e.g., listed in the previous section) in hardware circuitry. In some embodiments, the hardware 906 can be partially or entirely within the processor 902, such as a graphics processor.

[0312] Figure 10 A block diagram of a video encoder is shown.

[0313] Figure 11 is a flow chart of a method 1100 for processing video. The method 1100 includes converting between a chroma block of the video and a codec representation of the video by deriving parameters of a cross-component linear model using downsampled collocated neighboring top luma samples generated from N top-neighboring lines of the collocated luma block using a downsampled filter, where N is a positive integer, and performing (1104) the conversion using a predicted chroma block generated using the cross-component linear model.

[0314] Figure 12 is a block diagram of an example video processing system in which the disclosed technology may be implemented.

[0315] Figure 12 1 is a block diagram illustrating an example video processing system 1200 in which the various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1200. System 1200 may include an input 1202 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. Input 1202 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0316] System 1200 may include a codec component 1204 that can implement the various codecs or encoding methods described in this document. The codec component 1204 can reduce the average bit rate of the video from the input 1202 to the output of the codec component 1204 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. As represented by component 1206, the output of the codec component 1204 can be stored or sent via a connected communication. Component 1208 can use the stored or transmitted bitstream (or codec) representation of the video received at the input 1202 to generate pixel values or displayable video sent to the display interface 1210. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that the encoding tools or operations are used at the encoder, and the corresponding decoding tools or operations opposite to the encoding results will be performed by the decoder.

[0317] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described in this document may be embodied in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0318] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of the video block, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from the video block to the bitstream representation of the video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to the video block will be performed using the video processing tool or mode enabled based on the decision or determination.

[0319] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when the video processing tool or mode is disabled, the encoder will not use the tool or mode in converting video blocks to a bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode enabled based on the decision or determination.

[0320] The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware (including the structures disclosed herein and their structural equivalents), or in a combination of one or more thereof. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing apparatus or for controlling the operation of the data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials that implement a machine-readable propagated signal, or a combination of one or more thereof. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a suitable receiver apparatus.

[0321] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files that store one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers that are located at one location or distributed across multiple locations and interconnected by a communications network.

[0322] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0323] By way of example, processors suitable for executing computer programs include both general-purpose and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, to receive data from or transfer data to the mass storage devices, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated into, special-purpose logic circuitry.

[0324] Figure 13 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure.

[0325] like Figure 13 As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data and may be referred to as a video encoding device. The destination device 120 may decode the encoded video data generated by the source device 110 and may be referred to as a video decoding device.

[0326] Source device 110 may include a video source 112 , a video encoder 114 , and an input / output (I / O) interface 116 .

[0327] The video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a codec representation of the video data. The bitstream may include codec pictures and associated data. A codec picture is a codec representation of a picture. Associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be directly transmitted to the destination device 120 via the network 130a via the I / O interface 116. The encoded video data may also be stored on a storage medium / server 130b for access by the destination device 120.

[0328] Destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0329] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120, where destination device 120 is configured to interface with an external display device.

[0330] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other current and / or future standards.

[0331] Figure 14 is a block diagram illustrating an example of a video encoder 200, which may be Figure 13 The video encoder 114 in the system 100 is shown.

[0332] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. Figure 14 In the example of FIG, video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0333] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0334] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode in which at least one reference image is a picture in which the current video block is located.

[0335] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be highly integrated but are not shown for purposes of explanation. Figure 14 In the example, they are represented separately.

[0336] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0337] The mode selection unit 203 may select one of the coding modes (intra or inter) based on the error result, for example, and provide the resulting intra- or inter-coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combined intra and inter prediction (CIIP) mode, in which prediction is based on both an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may also select a resolution for the motion vector for the block (e.g., sub-pixel or integer pixel precision).

[0338] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures other than the picture associated with the current video block from the buffer 213.

[0339] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block, eg, depending on whether the current video block is in an I slice, a P slice, or a B slice.

[0340] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search for a reference video block for the current video block in the reference pictures in list 0 or list 1. Motion estimation unit 204 may then generate a reference index and a motion vector, where the reference index indicates the reference picture in list 0 or list 1 that includes the reference video block, and the motion vector indicates a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 may generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.

[0341] In other examples, motion estimation unit 204 may perform bidirectional prediction on the current video block. Motion estimation unit 204 may search for a reference video block for the current video block in the reference pictures in list 0 and may also search for another reference video block for the current video block in the reference pictures in list 1. Motion estimation unit 204 may then generate a reference index and a motion vector, where the reference index indicates the reference pictures in list 0 and list 1 that contain the reference video block, and the motion vector indicates the spatial displacement between the reference video block and the current video block. Motion estimation unit 204 may output the reference index and motion vector for the current video block as motion information for the current video block. Motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0342] In some examples, motion estimation unit 204 may output a complete set of motion information for use in the decoding process of the decoder.

[0343] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, motion estimation unit 204 may reference motion information of another video block to signal the motion information of the current video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0344] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0345] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0346] As described above, the video encoder 200 may predictively signal motion vectors.Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0347] Intra-prediction unit 206 may perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, intra-prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a predicted video block and various syntax elements.

[0348] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the predicted video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0349] In other examples, there may be no residual data for the current video block, such as in skip mode, and residual generation unit 207 may not perform a subtraction operation.

[0350] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0351] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0352] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block for storage in the buffer 213.

[0353] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blockiness artifacts in the video block.

[0354] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0355] Figure 15 is a block diagram illustrating an example of a video decoder 300, which may be Figure 13 The video decoder 114 in the system 100 is shown.

[0356] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 15 In the example of FIG, video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0357] exist Figure 15 In the example of FIG, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform the same operations that are generally performed for the video encoder 200 (e.g., Figure 14 ) is the opposite of the encoding process described in .

[0358] The entropy decoding unit 301 can retrieve an encoded bitstream. The encoded bitstream can include entropy-encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy-encoded video data, and based on the entropy-encoded video data, the motion compensation unit 302 can determine motion information, including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine such information, for example, by implementing AMVP and merge modes.

[0359] Motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter to be used with sub-pixel precision may be included in the syntax element.

[0360] Motion compensation unit 302 may calculate interpolated values for sub-integer pixels of a reference block using interpolation filters such as those used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 based on received syntax information and use the interpolation filters to generate a prediction block.

[0361] The motion compensation unit 302 may use some syntax information to determine the sizes of blocks used to encode frame(s) and / or slice(s) of the coded video sequence, partitioning information describing how each macroblock of an image of the coded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the coded video sequence.

[0362] The intra prediction unit 303 can form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 303 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0363] The reconstruction unit 306 may add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter may be applied to the decoded block to remove blocking artifacts. The decoded video block is then stored in the buffer 307, which provides reference blocks for subsequent motion compensation.

[0364] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the codec will use or implement the tool or mode in the processing of the video block, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from the video block to the bitstream representation of the video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to the video block will be performed using the video processing tool or mode enabled based on the decision or determination.

[0365] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. As defined by the syntax, the bitstream representation of the current video block may correspond to bits that are co-located or dispersed across different locations within the bitstream. For example, a macroblock may be encoded based on a transformed and encoded error residual value, and also using bits from the header and other fields in the bitstream.

[0366] A list of preferred clauses for some embodiments is provided below.

[0367] The first set of clauses illustrates example embodiments of the techniques discussed in the previous sections.

[0368] 1. A video processing method, comprising: for conversion between a chroma block of a video and a codec representation of the video, deriving parameters of a cross-component linear model by using downsampled co-located adjacent top luma samples, the downsampled co-located adjacent top luma samples being generated from N top-neighboring lines of the co-located luma block using a downsampling filter, where N is a positive integer; and performing the conversion using a predicted chroma block generated using the cross-component linear model.

[0369] 2. The method of clause 1 , wherein the N above-neighboring lines correspond to the nearest above lines of the co-located luma block due to the chroma block not being at a top codec tree unit boundary.

[0370] 3. A method according to any of clauses 1-2, wherein a downsampling filter is further applied to generate downsampled collocated adjacent left luma samples.

[0371] 4. A method according to any of clauses 1-2, wherein the downsampling filter is different from another downsampling filter used to generate the downsampled co-located adjacent left luma samples.

[0372] 5. A method according to any of clause 1, wherein the downsampling filter is independent of the position of the chroma block relative to the top boundary of the codec tree unit.

[0373] 6. The method of any one of clauses 1, wherein the method is selectively applied because the video has a 4:2:2 format.

[0374] 7. The method of clause 1, wherein N is greater than one.

[0375] 8. The method of clause 7, wherein the N upper neighboring lines include a nearest upper line and a second nearest upper line.

[0376] 9. The method of clause 1, wherein the downsampling filter depends on the color format of the video.

[0377] 10. A method according to any of clauses 1-9, wherein the downsampling filter is a 6-tap filter.

[0378] 11. A method according to any of clauses 1-9, wherein the downsampling filter is a 5-tap filter.

[0379] 12. A method according to any of clauses 1 to 11, wherein converting comprises encoding the video into a codec representation.

[0380] 13. A method according to any of clauses 1 to 11, wherein converting comprises decoding the codec representation to generate pixel values of the video.

[0381] 14. A video decoding apparatus comprising a processor configured to implement the method of one or more of clauses 1 to 13.

[0382] 15. A video encoding apparatus comprising a processor configured to implement the method of one or more of clauses 1 to 13.

[0383] 16. A computer program product having computer code stored thereon which, when executed by a processor, causes the processor to carry out the method of any one of clauses 1 to 13.

[0384] 17. A method, apparatus or system as herein described.

[0385] The second set of clauses describes certain features and aspects of the technology disclosed in previous sections (e.g., item 1).

[0386] 1. A video processing method, comprising: for conversion between a chroma block of a video and a bitstream representation of the video, deriving 1602 parameters of a cross-component linear model by using downsampled luma samples, the downsampled luma samples being generated from N upper neighboring lines of a co-located luma block of the chroma block using a downsampling filter, where N is a positive integer; and performing the conversion using a predicted chroma block generated using the cross-component linear model.

[0387] 2. The method of clause 1, wherein, since the chroma block is not at a top codec tree unit boundary, the N upper neighboring lines correspond to the nearest upper lines of the co-located luminance block.

[0388] 3. A method according to any of clauses 1-2, wherein the downsampling filter is further applied to generate further downsampled luma samples, the further downsampled luma samples being generated from a left adjacent line of the co-located luma block.

[0389] 4. A method according to any of clauses 1-2, wherein a further downsampling filter is applied to generate further downsampled luma samples, the further downsampled luma samples being generated from a left neighbouring line of the co-located luma block.

[0390] 5. A method according to any of clauses 1-4, wherein the downsampling filter has filter coefficients [1, 2, 1].

[0391] 6. A method according to any of clauses 1 to 5, wherein the downsampled luma samples pDsY[x] satisfy the equation pDsY[x] = (pY[2*x–1][-1] + 2*pY[2*x][-1] + pY[2*x+1][-1] + 2)>>2, where pY[2*x][-1], pY[2*x–1][-1] and pY[2*x+1][-1] are luma samples from the nearest upper neighboring line and x is an integer

[0392] 7. A method according to any of clauses 1-6, wherein the downsampling filter is independent of the position of the chroma block relative to the top boundary of the codec tree unit.

[0393] 8. A method according to any of clauses 1-6, wherein the method is selectively applied due to a 4:2:2 color format of the video.

[0394] 9. The method of clause 1 , wherein the N above-neighboring lines include the nearest above line of the co-located luma block, but not the second nearest above line, because the chroma block is not at a top codec tree unit boundary.

[0395] 10. The method of clause 1, wherein N is greater than one.

[0396] 11. The method of clause 10, wherein the N upper neighboring lines include a nearest upper line and a second nearest upper line.

[0397] 12. The method of clause 1, wherein the downsampling filter depends on the color format of the video.

[0398] 13. A method according to any of clauses 1-12, wherein the downsampling filter is a 6-tap filter.

[0399] 14. A method according to any of clauses 1-12, wherein the downsampling filter is a 5-tap filter.

[0400] 15. A method according to any of clauses 1 to 14, wherein converting comprises encoding the video into a bitstream representation.

[0401] 16. A method as described in any of clauses 1 to 14, wherein converting comprises decoding the video from a bitstream representation.

[0402] 17. A video processing apparatus comprising a processor configured to implement the method of any one or more of clauses 1 to 16.

[0403] 18. A computer-readable medium storing program code which, when executed, causes a processor to implement the method of any one or more of clauses 1 to 16.

[0404] 19. A computer readable medium storing a bitstream representation generated according to any of the above methods.

[0405] The third group of clauses describes certain features and aspects of the technology disclosed in previous sections (e.g., items 2 to 7).

[0406] 1. A video processing method (e.g., Figure 16A The method 1610 shown includes: determining 1612 a maximum allowed block size of a video block encoded using a transform skip mode for converting between a video region of a component of a video and a bitstream representation of the video; and performing 1614 the conversion based on the determination.

[0407] 2. A method according to clause 1, wherein the transform skip mode comprises, during encoding, encoding and decoding a residual of a video block without applying a non-identity transform, or, during decoding, determining a decoded video block without applying a non-identity inverse transform to a residual encoded in a bitstream representation.

[0408] 3. The method of clause 1, wherein the transform skip mode comprises BDPCM (Block Differential Pulse Coded Modulation), which corresponds to an intra codec using Differential Pulse Coded Modulation (DPCM) at the block level.

[0409] 4. A method according to clause 1, wherein the maximum allowed block size depends on whether the transform skipped block is a chroma block or a luma block.

[0410] 5. A method according to clause 1, wherein the maximum allowed block size depends on the chroma component of the transform skipped block.

[0411] 6. A method according to clause 1, wherein the maximum allowed block size of luma blocks (MaxTsSizeY) and the maximum allowed block size of chroma blocks (MaxTsSizeC) are signalled separately in the bitstream representation.

[0412] 7. A method according to clause 6, wherein MaxTsSizeC and / or MaxTsSizeY are signalled at sequence level, picture level, slice level or slice group level.

[0413] 8. A method according to clause 6, wherein MaxTsSizeY is signalled conditionally based on an enabled state of transform skip mode.

[0414] 9. A method according to clause 6, wherein MaxTsSizeY is signalled conditionally based on the colour format and / or the enabled state of the transform skip mode.

[0415] 10. The method of clause 1, wherein the conversion is performed by utilizing a predictive codec between maximum block sizes of the luma component and the chroma components.

[0416] 11. The method of clause 1, wherein the video block is a chroma video block, and wherein a maximum allowed block size (MaxTsSizeC) of the video block is dependent on a maximum allowed block size (MaxTsSizeY) of another video block that is a luma component.

[0417] 12. The method of clause 11, wherein MaxTsSizeC is set equal to MaxTsSizeY.

[0418] 13. The method of clause 11, wherein MaxTsSizeC is set equal to MaxTsSizeY / N, where N is an integer.

[0419] 14. The method of clause 1, wherein the video blocks are chroma video blocks, and wherein a maximum allowed block size (MaxTsSizeC) of the video blocks is set according to a chroma subsampling rate.

[0420] 15. A method according to clause 14, wherein MaxTsSizeC is set equal to i) MaxTsSizeY>>SubWidthC, ii) MaxTsSizeY>>SubHeightC, iii) MaxTsSizeY>>max(SubWidthC, SubHeightC), iv) MaxTsSizeY>>min(SubWidthC, SubHeightC), where MaxTsSizeY indicates the maximum block size of the luma video block, and SubWidthC and SubHeightC are predefined.

[0421] 16. A video processing method (e.g., Figure 16A The method 1610 shown includes performing conversion between a video comprising a video block and a bitstream representation of the video according to a first rule and a second rule, wherein a transform skip codec is used to encode and decode a first portion of the video block, and a transform codec is used to encode and decode a second portion of the video block, wherein the first rule specifies a maximum allowable block size for the first portion of the video block, the second rule specifies a maximum allowable block size for the second portion of the video block, and the maximum allowable block size for the first portion of the video block is different from the maximum allowable block size for the second portion of the video block.

[0422] 17. The method of clause 16, wherein the maximum allowed block size corresponds to the width and height of the corresponding block.

[0423] 18. A method according to clause 17, wherein the width and height of the maximum allowed block size are signalled separately.

[0424] 19. A method according to clause 17, wherein, for the second portion of the video block that is a chroma block, the width (MaxTsSizeWC) is set equal to MaxTsSizeY>>SubWidthC, and the height (MaxTsSizeHC) is set equal to MaxTsSizeY>>SubHeightC, where MaxTsSizeY indicates the maximum allowed block size for the luma block.

[0425] 20. A video processing method (e.g., Figure 16AThe method 1610 shown in FIG. 1 includes performing conversion between a video including one or more chroma blocks and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule that specifies whether a syntax element indicating use of a transform skip tool is included in the bitstream representation based on a maximum allowable size of a chroma block encoded and decoded using the transform skip tool.

[0426] 21. The method of clause 20, wherein the transform skipping tool comprises bypassing a transform or applying an identity transform.

[0427] 22. A method according to clause 20, wherein the syntax element is signalled in the case where tbW is less than or equal to MaxTsSizeC and tbH is less than or equal to MaxTsSizeC, where tbW and tbH are the width and height of the chroma blocks, respectively, and MaxTsSizeC is the maximum allowed size of the chroma blocks, respectively.

[0428] 23. A method according to clause 20, wherein, in the case where tbW is less than or equal to MaxTsSizeWC and tbH is less than or equal to MaxTsSizeHC, a syntax element is signalled, where tbW and tbH are the width and height of the chroma block, respectively, and MaxTsSizeWC and MaxTsSizeHC represent the width and height of the maximum allowed size of the chroma block, respectively.

[0429] 24. The method of clause 20, wherein the transform skip tool comprises a BDPCM (Block Differential Pulse Coded Modulation) mode, the BDPCM mode corresponding to an intra codec tool using Differential Pulse Coded Modulation (DPCM) at the block level.

[0430] 25. A video processing method (e.g., Figure 16A The method 1610 shown includes performing conversion between a video comprising one or more first video blocks of a first chroma component and one or more second video blocks of a second chroma component and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule specifying the use of a syntax element that jointly indicates the availability of a transform skip tool for encoding and decoding the one or more first chroma blocks and the one or more second chroma blocks.

[0431] 26. A method according to clause 25, wherein the syntax elements have binary values.

[0432] 27. The method of clause 25, wherein a transform skip tool is enabled or disabled in the one or more first video blocks and the one or more second video blocks according to a syntax element.

[0433] 28. A method according to clause 25, wherein the format rule further specifies that additional syntax elements are included in the bitstream representation based on whether the value of the syntax element is equal to K, where K is an integer.

[0434] 29. The method of clause 28, wherein the second syntax element is used to indicate to which of the one or more first video blocks and the one or more second video blocks the transform skip tool applies.

[0435] 30. The method of clause 25, wherein the syntax element has a non-binary value.

[0436] 31. The method of clause 30, wherein the syntax elements are encoded or decoded using fixed-length, unary, truncated unary, or k-th order Exp-Golomb (EG) binarization methods.

[0437] 32. The method of clause 25, wherein the syntax element is context coded or bypass coded.

[0438] 33. A method according to any preceding clause, wherein whether and / or how to apply the method is signalled at sequence level, picture level, slice level or slice group level.

[0439] 34. A method according to any of the preceding clauses, wherein the method is further based on codec information.

[0440] 35. A method according to any of clauses 1 to 34, wherein converting comprises encoding the video into a bitstream representation.

[0441] 36. A method according to any of clauses 1 to 34, wherein converting comprises decoding the video from a bitstream representation.

[0442] 37. A video processing device comprising a processor configured to implement the method of any one or more of clauses 1 to 36.

[0443] 38. A computer-readable medium storing program code which, when executed, causes a processor to implement the method of any one or more of clauses 1 to 36.

[0444] 39. A computer readable medium storing a codec representation or a bitstream representation generated according to any of the above methods.

[0445] Although this patent document contains many details, these details should not be construed as limitations on the scope of any subject matter or of what is claimed, but rather as descriptions of features unique to particular embodiments of particular technologies. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, alone or in any suitable subcombination. Furthermore, although features may be described above as functioning in certain combinations, and even initially claimed as such, one or more features from a claimed combination may in some cases be deleted from that combination, and a claimed combination may be directed to subcombinations or variations of subcombinations.

[0446] Similarly, while operations are described in a particular order in the drawings, this should not be understood as requiring that these operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, in order to achieve the desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0447] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For a first conversion between a chroma block of a video and a bitstream of the video, determine a prediction mode to apply to the chroma block, wherein in the prediction mode, predicted samples of the chroma block are derived based on reconstructed luma samples of a collocated luma block of the chroma block; deriving parameters of the prediction mode based on adjacent chroma samples of the chroma block and downsampled adjacent top luma samples of the collocated luma block; as well as performing a first conversion based on the parameters, wherein, in the case where the chroma block has a 4:2:2 color format, the downsampled adjacent top luma sample is derived based on the nearest upper adjacent line of the co-located luma block, Wherein, when the chroma co-location flag is equal to 0, the chroma block is not at the top codec tree unit boundary and the chroma block has a 4:2:0 color format, (pY[SubWidthC*x-1][-1]+pY[SubWidthC*x-1][-2]+2*pY[SubWidthC*x][-1]+2*pY[SubWidthC*x][-2]+pY[SubWidthC*x+1][-1]+pY[SubWidthC*x+1][-2]+4)>>3 is used to derive at least one downsampled adjacent top luminance sample, where pY[SubWidthC*x-1][-1], pY[SubWidthC*x-1][-2], pY[SubWidthC*x][-1], pY[SubWidthC*x][-2], pY[SubWidthC*x+1][-1] and pY[SubWidthC*x+1][-2] represent luma samples from adjacent lines of the collocated luma block, and Wherein, x is an integer and SubWidthC is equal to 2 when the chroma block has a 4:2:0 color format.

2. The method according to claim 1, wherein In response to the chroma block having a 4:2:2 color format, a second upper neighboring line of the co-located luma block is excluded from deriving the downsampled adjacent top luma sample.

3. The method according to claim 1, wherein In response to the chroma block having a 4:2:2 color format, the downsampled adjacent top luma samples are derived using the same downsampling filter regardless of whether the chroma block is at a top codec tree unit boundary.

4. The method according to claim 3, wherein: In response to the chroma block being at the top codec tree unit boundary, the downsampled neighboring top luma samples are derived based on the nearest upper neighboring line of the collocated luma block.

5. The method according to claim 4, wherein In response to the chroma block being at the top codec tree unit boundary or the chroma block having a 4:2:2 color format, pDsY[x]=(pY[2*x–1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2, where pDsY[x] represents the adjacent top luminance sample of the downsampled point, and Where pY[2*x][-1], pY[2*x–1][-1] and pY[2*x+1][-1] represent the luma samples from the nearest upper neighboring line of the co-located luma block.

6. The method according to claim 1, wherein further deriving the parameters of the prediction mode based on downsampled adjacent left luma samples of the co-located luma block; In response to the chroma block having a 4:2:2 color format, the downsampled adjacent top luma samples and the downsampled adjacent left luma samples are derived using a downsampling filter having the same filter coefficient.

7. The method according to claim 6, wherein: The same filter coefficients are [1 2 1].

8. The method according to claim 1, wherein A first syntax element specifying a maximum block size for a transform skip mode is included in the bitstream in response to a first condition based on a value of a transform skip enable flag in a sequence parameter set included in the bitstream.

9. The method according to any one of claims 1 to 8, wherein The converting includes decoding the video from the bitstream.

10. The method according to any one of claims 1 to 8, wherein The converting includes encoding the video into the bitstream.

11. An apparatus for processing video data, comprising a processor and non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: For a first conversion between a chroma block of a video and a bitstream of the video, determine a prediction mode to apply to the chroma block, wherein in the prediction mode, predicted samples of the chroma block are derived based on reconstructed luma samples of a collocated luma block of the chroma block; deriving parameters of the prediction mode based on adjacent chroma samples of the chroma block and downsampled adjacent top luma samples of the collocated luma block; as well as performing a first conversion based on the parameters, wherein, in the case where the chroma block has a 4:2:2 color format, the downsampled adjacent top luma sample is derived based on the nearest upper adjacent line of the co-located luma block, Wherein, when the chroma co-location flag is equal to 0, the chroma block is not at the top codec tree unit boundary and the chroma block has a 4:2:0 color format, (pY[SubWidthC*x-1][-1]+pY[SubWidthC*x-1][-2]+2*pY[SubWidthC*x][-1]+2*pY[SubWidthC*x][-2]+pY[SubWidthC*x+1][-1]+pY[SubWidthC*x+1][-2]+4)>>3 is used to derive at least one downsampled adjacent top luminance sample, where pY[SubWidthC*x-1][-1], pY[SubWidthC*x-1][-2], pY[SubWidthC*x][-1], pY[SubWidthC*x][-2], pY[SubWidthC*x+1][-1] and pY[SubWidthC*x+1][-2] represent luma samples from adjacent lines of the collocated luma block, and Wherein, x is an integer and SubWidthC is equal to 2 when the chroma block has a 4:2:0 color format.

12. The device according to claim 11, wherein In response to the chroma block having a 4:2:2 color format, a second upper neighboring line of the co-located luma block is excluded from deriving the downsampled adjacent top luma sample.

13. The device according to claim 11 or 12, wherein: In response to the chroma block having a 4:2:2 color format, the downsampled adjacent top luma samples are derived using the same downsampling filter regardless of whether the chroma block is at a top codec tree unit boundary.

14. The device according to claim 13, wherein In response to the chroma block being at the top codec tree unit boundary, the downsampled neighboring top luma samples are derived based on the nearest upper neighboring line of the collocated luma block.

15. The device according to claim 14, wherein In response to the chroma block being at the top codec tree unit boundary or the chroma block having a 4:2:2 color format, pDsY[x]=(pY[2*x–1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2, where pDsY[x] represents the adjacent top luminance sample of the downsampled point, and Where pY[2*x][-1], pY[2*x–1][-1] and pY[2*x+1][-1] represent the luma samples from the nearest upper neighboring line of the co-located luma block.

16. The device according to claim 11, wherein further deriving the parameters of the prediction mode based on downsampled adjacent left luma samples of the co-located luma block; In response to the chroma block having a 4:2:2 color format, the downsampled adjacent top luma samples and the downsampled adjacent left luma samples are derived using a downsampling filter having the same filter coefficient.

17. The device according to claim 16, wherein The same filter coefficients are [1 21].

18. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For a first conversion between a chroma block of a video and a bitstream of the video, determine a prediction mode to apply to the chroma block, wherein in the prediction mode, predicted samples of the chroma block are derived based on reconstructed luma samples of a collocated luma block of the chroma block; deriving parameters of the prediction mode based on adjacent chroma samples of the chroma block and downsampled adjacent top luma samples of the collocated luma block; as well as performing a first conversion based on the parameters, wherein, in the case where the chroma block has a 4:2:2 color format, the downsampled adjacent top luma sample is derived based on the nearest upper adjacent line of the co-located luma block, Wherein, when the chroma co-location flag is equal to 0, the chroma block is not at the top codec tree unit boundary and the chroma block has a 4:2:0 color format, (pY[SubWidthC*x-1][-1]+pY[SubWidthC*x-1][-2]+2*pY[SubWidthC*x][-1]+2*pY[SubWidthC*x][-2]+pY[SubWidthC*x+1][-1]+pY[SubWidthC*x+1][-2]+4)>>3 is used to derive at least one downsampled adjacent top luminance sample, where pY[SubWidthC*x-1][-1], pY[SubWidthC*x-1][-2], pY[SubWidthC*x][-1], pY[SubWidthC*x][-2], pY[SubWidthC*x+1][-1] and pY[SubWidthC*x+1][-2] represent luma samples from adjacent lines of the collocated luma block, and Wherein, x is an integer and SubWidthC is equal to 2 when the chroma block has a 4:2:0 color format.

19. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by a video processing apparatus, wherein the method comprises: Determining a prediction mode to be applied to a chroma block, wherein in the prediction mode, predicted samples of the chroma block are derived based on reconstructed luma samples of a collocated luma block with the chroma block; deriving parameters of the prediction mode based on adjacent chroma samples of the chroma block and downsampled adjacent top luma samples of the collocated luma block; as well as generating the bitstream based on the parameters, wherein, in the case where the chroma block has a 4:2:2 color format, the downsampled adjacent top luma sample is derived based on the nearest upper adjacent line of the co-located luma block, Wherein, when the chroma co-location flag is equal to 0, the chroma block is not at the top codec tree unit boundary and the chroma block has a 4:2:0 color format, (pY[SubWidthC*x-1][-1]+pY[SubWidthC*x-1][-2]+2*pY[SubWidthC*x][-1]+2*pY[SubWidthC*x][-2]+pY[SubWidthC*x+1][-1]+pY[SubWidthC*x+1][-2]+4)>>3 is used to derive at least one downsampled adjacent top luminance sample, where pY[SubWidthC*x-1][-1], pY[SubWidthC*x-1][-2], pY[SubWidthC*x][-1], pY[SubWidthC*x][-2], pY[SubWidthC*x+1][-1] and pY[SubWidthC*x+1][-2] represent luma samples from adjacent lines of the collocated luma block, and Wherein, x is an integer and SubWidthC is equal to 2 when the chroma block has a 4:2:0 color format.

20. A method for storing a bitstream of a video, comprising: Determining a prediction mode to be applied to a chroma block, wherein in the prediction mode, predicted samples of the chroma block are derived based on reconstructed luma samples of a collocated luma block of the chroma block; deriving parameters of the prediction mode based on adjacent chroma samples of the chroma block and downsampled adjacent top luma samples of the collocated luma block; generating the bitstream based on the parameters; as well as storing the bitstream in a non-transitory computer-readable recording medium, wherein, in the case where the chroma block has a 4:2:2 color format, the downsampled adjacent top luma sample is derived based on the nearest upper adjacent line of the co-located luma block, Wherein, when the chroma co-location flag is equal to 0, the chroma block is not at the top codec tree unit boundary and the chroma block has a 4:2:0 color format, (pY[SubWidthC*x-1][-1]+pY[SubWidthC*x-1][-2]+2*pY[SubWidthC*x][-1]+2*pY[SubWidthC*x][-2]+pY[SubWidthC*x+1][-1]+pY[SubWidthC*x+1][-2]+4)>>3 is used to derive at least one downsampled adjacent top luminance sample, where pY[SubWidthC*x-1][-1], pY[SubWidthC*x-1][-2], pY[SubWidthC*x][-1], pY[SubWidthC*x][-2], pY[SubWidthC*x+1][-1] and pY[SubWidthC*x+1][-2] represent luma samples from adjacent lines of the collocated luma block, and Wherein, x is an integer and SubWidthC is equal to 2 when the chroma block has a 4:2:0 color format.

Images