Method and apparatus for cross-component prediction for video coding

Abstract

The present disclosure provides a method for decoding video data, comprising: obtaining a video block from a bitstream; obtaining an internal brightness sample point value of the video block, an external brightness sample point value of an external region of the video block and an external chrominance sample point value of the external region; determining a set of weighting coefficients corresponding to a filter shape based on the external luma sample value and the external chroma sample value, where the filter shape and the set of weighting coefficients are configured for predicting a chroma sample value based on a plurality of corresponding luma sample values, and wherein the set of weighting coefficients comprises weighting coefficients for non-linear values derived by scaling a product of two of a plurality of corresponding luminance sample values or a square of one of the plurality of corresponding luminance sample values to a range of luminance sample values for the video block; predicting the internal chroma sample value based on the internal luma sample value using the filter shape and the set of weighting coefficients; and, using the predicted internal chroma sample values to obtain a decoded video block.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 397,731, filed on August 12, 2022. The entire contents of the U.S. Provisional Application are incorporated herein by reference in their entirety. Technical Field

[0002] Aspects of the present disclosure relate generally to image / video coding and compression, and more particularly, to methods and apparatus for cross-component prediction techniques. Background Art

[0003] Various video codec techniques can be used to compress video data. Video coding is performed according to one or more video codec standards. For example, video codec standards include generic video codec (VVC), high efficiency video codec (H.265 / HEVC), advanced video codec (H.264 / AVC), moving picture experts group (MPEG) codec, etc. Video codecs generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize redundancy present in video images or sequences. An important goal of video coding techniques is to compress video data into a form that uses a lower bit rate while avoiding or minimizing degradation of video quality. Summary of the invention

[0004] A simplified overview of one or more aspects of the present disclosure is presented below to provide a basic understanding of these aspects. The present disclosure is not a broad overview of all expected aspects, is not intended to identify the key or key elements of all aspects, and is not intended to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description presented later.

[0005] According to one aspect of the present disclosure, a method for decoding video data is provided, comprising: obtaining a video block from a bitstream; obtaining an internal luma sample value of the video block, an external luma sample value of an external region of the video block, and an external chroma sample value of the external region; determining a set of weighting coefficients corresponding to a filter shape based on the external luma sample value and the external chroma sample value, wherein the filter shape and the set of weighting coefficients are configured to predict chroma sample values ​​based on a plurality of corresponding luma sample values, and wherein the set of weighting coefficients includes a weighting coefficient for a nonlinear value, the nonlinear value being derived by scaling a product of two of the plurality of corresponding luma sample values ​​or a square of one of the plurality of corresponding luma sample values ​​to a luma sample value range of the video block; predicting the internal chroma sample value based on the internal luma sample value using the filter shape and the set of weighting coefficients; and obtaining a decoded video block using the predicted internal chroma sample value.

[0006] According to another aspect of the present invention, a method for encoding video data is provided, comprising: obtaining a video block; obtaining an internal luma sample value of the video block, an external luma sample value of an external area of ​​the video block, and an external chroma sample value of the external area; determining a set of weighting coefficients corresponding to a filter shape based on the external luma sample value and the external chroma sample value, wherein the filter shape and the set of weighting coefficients are configured to predict chroma sample values ​​based on a plurality of corresponding luma sample values, and wherein the set of weighting coefficients includes a weighting coefficient for a nonlinear value, the nonlinear value being derived by scaling a product of two of the plurality of corresponding luma sample values ​​or a square of one of the plurality of corresponding luma sample values ​​to a luma sample value range of the video block; predicting the internal chroma sample value based on the internal luma sample value using the filter shape and the set of weighting coefficients; and generating a bitstream including an encoded video block by using the predicted internal chroma sample value.

[0007] According to another aspect of the present disclosure, a computer system is provided, comprising: one or more processors; and one or more storage devices storing computer executable instructions, which, when executed, cause the one or more processors to perform the operations of the method of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product storing computer executable instructions is provided, which when executed cause one or more processors to perform the operations of the method of the present disclosure.

[0009] According to another aspect of the present disclosure, a computer-readable medium storing computer-executable instructions is provided, which, when executed, cause one or more processors to perform the operations of the method of the present disclosure based on a bitstream.

[0010] According to another aspect of the present disclosure, a computer-readable medium storing computer-executable instructions is provided, which when executed cause one or more processors to perform the operations of the method of the present disclosure.

[0011] According to another aspect of the present disclosure, a computer-readable medium storing a bit stream is provided, wherein the bit stream is to be decoded by performing the operations of the method of the present disclosure.

[0012] According to another aspect of the present disclosure, a computer-readable medium storing a bitstream is provided, wherein the bitstream is obtained by executing the operation of the method of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The disclosed aspects will hereinafter be described in conjunction with the accompanying drawings, which are provided for the purpose of illustrating rather than limiting the disclosed aspects.

[0014] Figure 1 A block diagram of a general block-based hybrid video coding system is shown.

[0015] FIG. 2A to FIG. 2E Five split types are shown, including a four-pronged split, a horizontal two-pronged split, a vertical two-pronged split, a horizontal three-pronged split, and a vertical three-pronged split.

[0016] Figure 3 An overall block diagram of a block-based video decoder is shown.

[0017] Figure 4 An example of the positions of left and upper samples of the current block and samples of the current block involved in the CCLM mode is shown.

[0018] FIG. 5A to FIG. 5C An example of deriving CCLM parameters is shown.

[0019] Figure 6 An example of classifying adjacent samples into two groups based on the value threshold Threshold is shown.

[0020] Figure 7 An example of classifying adjacent sample points into two groups based on inflection points is shown.

[0021] Fig. 8A and Figure 8B The effect of the scaling adjustment parameter "u" is shown.

[0022] Figure 8C The co-located reconstructed luminance samples are shown.

[0023] Fig.8D Neighboring reconstructed samples are shown.

[0024] FIG. 8E to FIG. 8H The steps of intra mode derivation on the decoder side are shown.

[0025] Fig. 9 An example of four reference lines adjacent to a prediction block is shown.

[0026] Fig. 10A An exemplary pattern for a convolutional cross-component model (CCCM) is shown.

[0027] Fig. 10B An exemplary reference region consisting of 6 rows of chroma samples above and to the left of the PU is shown.

[0028] Fig. 10C and Fig. 10D A schematic diagram showing the correlation between chrominance samples and one or more luma samples.

[0029] Fig.11 An example of using 6 taps in a multiple linear regression (MLR) model according to one or more aspects of the present disclosure is shown.

[0030] Fig.12 Exemplary different filter shapes and / or numbers of taps according to one or more aspects of the present disclosure are shown.

[0031] Fig.13 An example is shown where the FLM may use only the top or left luma and / or chroma samples (extension) for parameter derivation.

[0032] Fig.14 An example is shown where FLM can use different lines for parameter derivation.

[0033] FIG. 15A to FIG. 15D Some examples of 1-tap / 2-tap pre-operation are shown.

[0034] Fig.16 Examples of different shapes / numbers of filter taps are shown.

[0035] Fig.17 Examples of different shapes / numbers of filter taps are shown.

[0036] Fig.18A and Fig.18B Examples of different shapes / numbers of filter taps are shown.

[0037] FIG. 19A to FIG. 19GExamples of different sets of filter taps are shown.

[0038] Fig. 20A and Fig. 20B An example of 2-fold training for implicit filter shape derivation is shown.

[0039] Fig.21 A workflow of a method for decoding video data according to one or more aspects of the present disclosure is shown.

[0040] Fig. 22 A workflow of a method for encoding video data according to one or more aspects of the present disclosure is shown.

[0041] Fig.23 An exemplary computing system in accordance with one or more aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0042] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0043] It should be noted that the terms "first", "second", etc. used in the specification, claims, and drawings of the present disclosure are used to distinguish objects, rather than to describe any specific order or sequence. It should be understood that the terms used in this manner can be interchanged under appropriate conditions, so that the embodiments of the present disclosure described herein can be implemented in an order other than those shown in the drawings or described in the present disclosure.

[0044] The first version of the VVC standard was finalized in July 2020, providing approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video codec standard HEVC. Although the VVC standard provides significant codec improvements over its predecessor, there is evidence that higher codec efficiency can be achieved using additional codec tools. Recently, the Joint Video Exploration Team (JVET), in collaboration with ITU-T VCEG and ISO / IEC MPEG, began exploring advanced technologies that can significantly improve the codec efficiency of VVC. In April 2021, a software code base called the Enhanced Compression Model (ECM) was established for future video codec exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, in which several existing modules (e.g., intra / inter prediction, transform, loop filter, etc.) have been further extended and / or improved. In the future, any new codec tools beyond the VVC standard will need to be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC).

[0045] Similar to all previous video codec standards, ECM is built on a block-based hybrid video codec framework. Figure 1 A block diagram of a general block-based hybrid video coding system is shown. The input video signal is processed block by block, called coding unit (CU). In ECM-1.0, a CU can be up to 128×128 pixels. However, like VVC, a coding tree unit (CTU) is split into CUs based on quadtree / binarytree / ternarytree to adapt to changing local characteristics. In the multi-type tree structure, a CTU is first split according to the quadtree structure. Then, each quadtree leaf node can be further split according to the binary tree and ternary tree structure. As Figure 2A , Figure 2B , Figure 2C , Figure 2D and Figure 2E As shown, there are five types of partitioning, quad split, vertical binary split, horizontal binary split, vertical extended quad split, and horizontal extended quad split.

[0046] exist Figure 1In the video code, spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples (called reference samples) of neighboring blocks that have been coded in the same video picture / slice to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion compensated prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is usually signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference. In addition, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture storage the temporal prediction signal comes from. After spatial prediction and / or temporal prediction, the mode decision block in the encoder selects the best prediction mode, for example, based on a rate-distortion optimization method. The prediction block is then subtracted from the current video block; and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are dequantized and inverse transformed to form a reconstructed residual, which is then added back to the prediction block to form the reconstructed signal of the CU. Further loop filtering, such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF), may be applied to the reconstructed CU before it is placed into reference picture storage and used for encoding and decoding future video blocks. In order to form an output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit for further compression and packaging to form a bitstream. It should be noted that the term "block" or "video block" used herein may be a portion of a frame or picture, particularly a rectangular (square or non-square) portion. For example, with reference to HEVC and VVC, a block or a video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU), and / or may be or correspond to a corresponding block, for example, a codec tree block (CTB), a codec block (CB), a prediction block (PB), or a transform block (TB) and / or correspond to a sub-block.

[0047] Figure 3An overall block diagram of a block-based video decoder is shown. The video bitstream is first entropy decoded at the entropy decoding unit. The codec mode and prediction information are sent to the spatial prediction unit (if it is intra-frame codec) or the temporal prediction unit (if it is inter-frame codec) to form a prediction block. The residual transform coefficients are sent to the dequantization unit and the inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then added. The reconstructed block can be further loop filtered before being stored in the reference picture storage. The reconstructed video in the reference picture storage is then sent out to drive the display device and for predicting future video blocks.

[0048] The main focus of this disclosure is to further enhance the codec efficiency of codec tools applied to cross-component prediction, cross-component linear model (CCLM) in ECM. In the following, some related codec tools in ECM are briefly reviewed. Thereafter, some defects in the existing design of CCLM are discussed. Finally, solutions are provided to improve the existing CCLM prediction design.

[0049] Cross-component linear model prediction

[0050] In order to reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VVC, for which chrominance samples are predicted based on the reconstructed luma samples of the same CU using the following linear model: pred C (i,j)=α·rec L ′(i,j)+ β (1) Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L '(i,j) represents the reconstructed brightness sample point rec L The downsampled reconstructed luma samples of the same CU obtained by downsampling (i, j). The above α and β are linear model parameters derived from up to four adjacent chroma samples and their corresponding downsampled luma samples (which can be called adjacent luma-chroma sample pairs). Assuming that the size of the current chroma block is W×H, W' and H' are obtained as follows: - When LM mode is applied, W'=W, H'=H; - When LM-A mode is applied, W'=W+H; - When LM-L mode is applied, H'=H+W; Among them, in LM mode, the upper and left samples of the CU are used together to calculate the linear model coefficients; in LM_A mode, only the upper samples of the CU are used to calculate the linear model coefficients; and in LM_L mode, only the left samples of the CU are used to calculate the linear model coefficients.

[0051] If the positions of the upper neighboring samples of the chroma block are represented as S[0, -1] ... S[W'-1, -1], and the positions of the left neighboring samples of the chroma block are represented as S[-1, 0] ... S[-1, H'-1], the positions of the four neighboring chroma samples are selected as follows: – When LM mode is applied and both upper and left adjacent samples are available, select S[W' / 4, -1], S[3*W' / 4, -1], S[-1, H' / 4], S[-1, 3*H' / 4] as the positions of the four adjacent chroma samples; – When LM-A mode is applied or only upper adjacent samples are available, select S[W' / 8, -1], S[3*W' / 8, -1], S[5*W' / 8, -1], S[7*W' / 8, -1] as the positions of four adjacent chrominance samples; – When LM-L mode is applied or only left adjacent samples are available, select S[-1, H' / 8], S[-1, 3*H' / 8], S[-1, 5*H' / 8], S[-1, 7*H' / 8] as the positions of four adjacent chroma samples;

[0052] Four adjacent brightness samples corresponding to the selected position are obtained through a downsampling operation, and the obtained four adjacent brightness samples are compared four times to find two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B The chrominance sample values ​​corresponding to the two larger values ​​and the two smaller values ​​are represented as y 0 A ,y 1 A ,y 0 B and 1 B Then X a , X b , Y a and Y b is derived as: X a =(x 0 A +x 1 A +1)>>1; X b =(x 0 B+x 1 B +1)>>1; Y a =(y 0 A +y 1 A +1)>>1; Y b =(y 0 B +y 1 B +1)>>1 (2)

[0053] Finally, the linear model parameters α and β are obtained according to the following formula. β=Y b -α·X b (4)

[0054] Figure 4 Examples of the left sample and top sample of the current block involved in the CCLM mode and the positions of the samples of the current block are shown, including the positions of the left sample and top sample of the N×N chroma block in the CU, and the positions of the left sample and top sample of the 2N×2N luminance block in the CU.

[0055] The division operation to calculate the parameter α is implemented using a lookup table. In order to reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated using a 4-bit significant part and an exponent. Therefore, the table of 1 / diff is reduced to 16 elements for 16 significant bit values, as shown below: DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0056] This will have the following benefits: it will reduce both the computational complexity and the memory size required to store the required tables.

[0057] In addition to the upper template and the left template being used together to calculate the linear model coefficients, they can also be used alternatively in two other LM modes, called LM_A mode and LM_L mode.

[0058] In LM_T mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0059] In LM_LT mode, the left template and the upper template are used to calculate the linear model coefficients.

[0060] To match the chroma sample positions for 4:2:0 video sequences, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "type 0" and "type 2" content, respectively.

[0061] Note that when the upper reference line is at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to make the downsampled luma samples.

[0062] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to convey the α and β values ​​to the decoder.

[0063] For chroma intra mode coding and decoding, a total of 8 intra modes are allowed for chroma intra mode coding and decoding. These modes include five traditional intra modes and three cross-component linear modes (CCLM, LM_A, and LM_L). The chroma mode signaling and derivation process are shown in Table 1. Chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited. Table 1 - Derivation of chrominance prediction mode from luma mode when CCLM is enabled

[0064] As shown in Table 2, regardless of the value of sps_cclm_enabled_flag, a single binarization table is used. Table 2 — Unified binarization table for chroma prediction mode Value of intra_chroma_pred_mode Binary String 4 00 0 0100 1 0101 2 0110 3 0111 5 10 6 110 7 111

[0065] In Table 2, the first binary number indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next binary number indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, the next binary number indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first binary number of the binarization table corresponding to intra_chroma_pred_mode can be discarded before entropy coding. Or, in other words, the first binary number is inferred to be 0 and is therefore not coded. This single binarization table is used for both cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two binary numbers in Table 22 are context coded and decoded with their own context model, while the remaining binary numbers are bypass coded and decoded.

[0066] In addition, to reduce the luma-chroma latency in the binary tree, when the 64×64 luma codec tree node is split with unsplit (and ISP is not used for 64×64 CU) or QT, the chroma CU in the 32×32 / 32×16 chroma codec tree node is allowed to use CCLM in the following way: If a 32×32 chroma node is not split or is split with QT splitting, all chroma CUs in the 32×32 node can use CCLM. If a 32×32 chroma node is split with horizontal BT and the 32×16 child node is not split or uses vertical BT split, all chroma CUs in the 32×16 chroma node can use CCLM.

[0067] In all other luma and chroma codec tree split conditions, CCLM is not allowed for chroma CUs.

[0068] During the ECM development, the simplified derivation (min-max approximation) of α and β was removed. Instead, a linear least squares solution between the causal reconstructed data of the downsampled luma samples and the causal chroma samples was used to derive the model parameters α and β. Among them, Rec C (i) and Rec' L (i) indicates the reconstructed chroma samples and downsampled luminance samples around the target block, and I indicates the total number of samples of neighboring data.

[0069] The LM_A and LM_L modes are also called multi-directional linear models (MDLM). Figure 5A An example of the operation of the MDLM is shown when the block content cannot be predicted from the L-shaped reconstruction region. Figure 5BMDLM_L is shown using only left reconstructed samples to derive CCLM parameters. Figure 5C MDLM_T is shown using only the top reconstructed samples to derive CCLM parameters.

[0070] An integerization of the least mean square (LMS) discussed above (see equations (8)-(9)) has been proposed as an improvement to CCLM. The initial integerization design of the LMS CCLM was first proposed in JCTVC-C206. The method was then improved through a series of simplifications, including reducing the precision n a JCTVC-F0233 / I0178 which reduced the maximum multiplier bit width from 13 to 7, JCTVC-I0151 which reduced the maximum multiplier bit width, and JCTVC-H0490 / I0166 which reduced the division LUT entries from 64 to 32, eventually leading to the ECM LMS version.

[0071] As discussed in equation (1), the integerized design utilizes a linear relationship to model the correlation of the luma and chroma signals. The chroma values ​​are predicted from the reconstructed luma values ​​of the co-located blocks.

[0072] Luma and chroma components have different sampling rates in YUV420 sampling. The sampling rate of the chroma component is half of the sampling rate of the luma component, and has a 0.5 pixel phase difference in the vertical direction. The reconstructed luma needs to be downsampled in the vertical direction and subsampled in the horizontal direction to match the size of the chroma signal. For example, downsampling can be achieved by the following formula: Rec L ′(i,j)=(rec L (2i,2j)+rec L (2i,2j+1))>>1 (10)

[0073] Floating point operations are required in formula (8) to calculate the linear model parameter α to maintain high data accuracy. And when α is represented by a floating point value, floating point multiplication involves formula (1). In this section, an integer implementation of the algorithm is designed. Specifically, the fractional part of the parameter α is represented by n α The parameter value α is quantized by the enlarged and rounded integer value α' and a'=a×(1<<n α ). Then the linear model of formula (1) becomes: pred C [x,y]=(α'·Rec L '[x,y]>>n α )+β' (11) Where β′ is the rounded value of the floating point β and α′ can be calculated as follows.

[0074] It is proposed to replace the division operation of formula (12) with table lookup and multiplication. A2 is first scaled down to reduce the table size. A1 is also scaled down to avoid product overflow. Then, in A2, only the The most significant bit of the value is defined, and the other bits are set to zero. The approximate value A2' can be calculated as: where [...] means rounding, and It can be calculated as: Here, bdepth(A2) represents the bit depth of value A2.

[0075] Perform the same operation on A1 as follows:

[0076] Consider the quantized representation of A1 and A2, and formula (12) can be rewritten as follows. in, Represented as having a length of to avoid the lookup table division.

[0077] In the simulation, the constant parameters are set as: ·n α Equal to 13, this value is a compromise between data accuracy and computational cost. · Equal to 6, resulting in a lookup table size of 64, which can be further reduced to 32 by scaling up A2 when bdepth(A2) < 6 (eg, A2 < 32). ·n table Equal to 15, resulting in a 16-bit data representation of the table elements. · Set to 15 to avoid product overflow and maintain 16-bit multiplication.

[0078] Finally, α' is clipped to [-2 -15 ,2 15 -1] to maintain the 16-bit multiplication in equation (11). With this clipping, when n α When it is equal to 13, the actual value a is limited to [-4,4), which helps prevent error amplification.

[0079] Using the calculated parameter α', the parameter β' is calculated as follows: Wherein, since the value I is a power of 2, the division in the above formula can be simply replaced by a shift.

[0080] Similar to what was discussed above with respect to equation (1), in HM6.0, an intra prediction mode called LM is applied to predict the chroma PU based on a linear model using reconstruction of the co-located luma PU. The parameters of the linear model include the slope (a>>k) and the y-intercept (b), which are derived from neighboring luma and chroma pixels using a least mean square solution. The values ​​of the prediction samples predSamples[x, y] (where x, y = 0...nS-1, where nS specifies the block size of the current chroma PU) are derived as follows: predSamples[x,y]=Clip1 C (((p Y '[x, y]*a)>>k)+b), where, x, y=0..nS-1 (17) Among them, P Y '[x, y] is the reconstructed pixel from the corresponding luminance component. When the coordinates x and y are equal to or greater than 0, P Y ' is the reconstructed pixel from the same luma PU. When x or y is less than 0, P Y ' is the reconstructed neighboring pixel of the same-position brightness PU.

[0081] Some intermediate variables L, C, LL, LC, k2 and k3 in the derivation process are derived as follows: k2=Log2((2*nS)>>k3) (18-5) k3=Max(0,BitDepth C +Log2(nS)-14) (18-6)

[0082] Therefore, the variables a, b, and k can be derived as: a1=(LC< <k2)–L*C (19-1) a2=(LL< <k2)–L*L (19-2) k1 =Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))-14)+2 (19-3) a1s=a1>>Max(0,Log2(abs(a1))-14) (19-4) a2s=abs(a2>>Max(0,Log2(abs(a2))-5)) (19-5) a3 = a2s<1?0: Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1) (19-6) a=a3>>Max(0,Log2(abs(a3))-6) (19-7) k=13–Max(0,Log2(abs(a))-6) (19-8) b = (L–((a*C)>>k1)+(1<<(k2-1)))>>k2, (19-9) Where lmDiv is specified in a 63-entry lookup table (ie, Table 3), which is generated online by the following formula: lmDiv(a2s)=((1<<15)+a2s / 2) / a2s. (20) Table 3—lmDiv Description a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 lmDiv 32768 16384 10923 8192 6554 5461 4681 4096 3641 3277 2979 2731 2521 a2s 14 15 16 17 18 19 20 21 22 23 24 25 26 lmDiv 2341 2185 2048 1928 1820 1725 1638 1560 1489 1425 1365 1311 1260 a2s 27 28 29 30 31 32 33 34 35 36 37 38 39 lmDiv 1214 1170 1130 1092 1057 1024 993 964 936 910 886 862 840 a2s 40 41 42 43 44 45 46 47 48 49 50 51 52 lmDiv 819 799 780 762 745 728 712 697 683 669 655 643 630 a2s 53 54 55 56 57 58 59 60 61 62 63 64 lmDiv 618 607 596 585 575 565 555 546 537 529 520 512

[0083] In formula (19-6), a1s is a 16-bit signed integer, and lmDiv is a 16-bit unsigned integer. Therefore, a 16-bit multiplier and 16-bit storage are required. It is proposed to reduce the bit depth of the multiplier to the internal bit depth and the size of the lookup table, as detailed below.

[0084] The bit depth of a1s is reduced to the internal bit depth by changing formula (19-4) to the following: a1s=a1>>Max(0,Log2(abs(a1))–(BitDepth C –2)) (21) The value of lmDiv with internal bit depth is implemented by the following formula (22) and stored in the lookup table: lmDiv(a2s)=((1<<(BitDepth C -1))+a2s / 2) / a2s. (22)

[0085] Table 4 shows an example of an internal bit depth of 10. Table 4 - Description of lmDiv with internal bit depth equal to 10 a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 lmDiv 512 256 171 128 102 85 73 64 57 51 47 43 39 37 34 32 a2s 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 lmDiv 30 28 27 26 24 23 22 21 20 20 19 18 18 17 17 16 a2s 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 lmDiv 16 15 15 14 14 13 13 13 12 12 12 12 11 11 11 11 a2s 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 10 10 10 10 10 9 9 9 9 9 9 9 8 8 8

[0086] Formulas (19-3) and (19-8) are also modified as follows: k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))–(BitDepth C–2)) (23-1) and k = BitDepth C –1–Max(0,Log2(abs(a))-6). (23-2)

[0087] It is also proposed to reduce the entries from 63 to 32 and the bits per entry from 16 to 10, as shown in Table 5. By doing so, almost 70% memory savings can be achieved. The corresponding changes to formula (19-6), formula (20) and formula (19-8) are as follows: a3=a2s<32?0:Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1) (24-1) lmDiv(a2s)=((1<<(BitDepth C +4))+a2s / 2) / a2s (24-2) k = BitDepth C +4–Max(0,Log2(abs(a))-6). (24-3) Table 5 - Description of lmDiv with internal bit depth equal to 10 a2s 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 lmDiv 512 496 482 468 455 443 431 420 410 400 390 381 372 364 356 349 a2s 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 341 334 328 321 315 309 303 298 293 287 282 278 273 269 264 260

[0088] Multi-model linear model prediction

[0089] In ECM-1.0, a multi-model LM (MMLM) prediction mode is proposed. For this MMLM prediction mode, the chrominance samples are predicted based on the reconstructed luminance samples of the same CU by using the following two linear models: where pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. The threshold Threshold is calculated as the average of adjacent reconstructed luma samples. Figure 6 An example of classifying adjacent samples into two groups based on the value Threshold is given. For each group, the parameter α i and β i (where i is 1 and 2 respectively) is derived from the linear relationship between the luminance and chrominance values ​​from two samples, which are the minimum luminance samples A(X A , Y A ) and the maximum brightness sample B(X B , YB ). Here, X A , Y A is the x-coordinate (i.e., luminance value) and y-coordinate (i.e., chrominance value) value of sample point A, and X B , Y B are the x-coordinate and y-coordinate values ​​of sample point B. The linear model parameters α and β are obtained according to the following formula. β=y A -αx A (26)

[0090] This method is also called the min-max method. The division in the above formula can be avoided and replaced with multiplication and shift.

[0091] For square codec blocks, the above two formulas are directly applied. For non-square codec blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0092] In addition to the scenario in which the upper template and the left template are used together to calculate the linear model coefficients, these two templates can also be used alternately in two other MMLM modes (referred to as MMLM_A and MMLM_L modes).

[0093] In MMLM_A mode, only pixel samples in the upper template are used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to size (W+W). In MMLM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to size (H+H).

[0094] Note that when the upper reference line is located at a CTU boundary, only one luma line (which is stored in the line buffer for intra prediction) is used to make the downsampled luma samples.

[0095] For chroma intra mode coding and decoding, a total of 11 intra modes are allowed for chroma intra mode coding and decoding. Those modes include five traditional intra modes and six cross-component linear model modes (CCLM, LM_A, LM_L, MMLM, MMLM_A, and MMLM_L). The chroma mode signaling and derivation process are shown in Table 6. The chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited. Table 6 - Derivation of chroma prediction mode from luma mode when MMLM is enabled

[0096] MMLM and LM modes can also be used together in an adaptive manner. For MMLM, the two linear models are as follows: Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. Threshold can be simply determined based on the luma and chroma average values ​​and their minimum and maximum values. Figure 7 An example of classifying adjacent samples into two groups based on the inflection point T indicated by the arrow is shown. The linear model parameters α1 and β1 are derived from the linear relationship between the luminance value and the chrominance value of the two samples, which are the minimum luminance sample A(X A , Y A ) and Threshold(X T , Y T The linear model parameters α2 and β2 are derived from the linear relationship between the brightness and chromaticity values ​​of two sample points, which are the maximum brightness sample points B(X B , Y B ) and Threshold(X T , Y T ). Here, X A , Y A is the x-coordinate (i.e., luminance value) and y-coordinate (i.e., chrominance value) value of sample point A, and X B , Y B is the x-coordinate and y-coordinate value of sample point B. The linear model parameter α of each group i and β i (where i is equal to 1 and 2 respectively) is obtained according to the following formula.

[0097] For square codec blocks, the above formula is directly applied. For non-square codec blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0098] In addition to the scenario where the upper template and the left template are used together to determine the linear model coefficients, these two templates can also be used interchangeably in the other two MMLM modes (referred to as MMLM_A and MMLM_L modes, respectively).

[0099] In MMLM_A mode, only pixel samples in the upper template are used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to size (W+W). In MMLM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to size (H+H).

[0100] Note that when the upper reference line is located at a CTU boundary, only one luma line (which is stored in the line buffer for intra prediction) is used to make the downsampled luma samples.

[0101] For chroma intra mode codec, there is a conditional check for selecting LM mode (CCLM, LM_A and LM_L) or multi-model LM mode (MMLM, MMLM_A and MMLM_L). The conditional check is as follows: Among them, BlkSizeThres LM Indicates the minimum block size for LM mode, and BlkSizeThres MM Indicates the minimum block size of the MMLM mode. The symbol d represents a predetermined threshold. In one example, d may be 0. In another example, d may be 8.

[0102] For chroma intra mode coding and decoding, a total of 8 intra modes are allowed for chroma intra mode coding and decoding. Those modes include five traditional intra modes and three cross-component linear modes. The chroma mode signaling and derivation process are shown in Table 1. It is worth noting that for a given CU, if it is coded and decoded in the linear model mode, whether it is the traditional single model LM mode or the MMLM mode is determined based on the above conditional check. Unlike the case shown in Table 3, no separate MMLM mode is signaled. Chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0103] During the development of the ECM, scaling (slope) adjustment of the CCLM was proposed as a further improvement, for example as described in JVET-Y0055 / Z0049.

[0104] As discussed above, CCLM uses a 2-parameter model to map luma values ​​to chroma values. The scaling parameter "a" and the bias parameter "b" define the following mapping: chromaVal=a*lumaVal+b (30)

[0105] It is proposed to signal an adjustment "u" to the scaling parameter to update the model to the following form: chromaVal=a'*lumaVal+b' (31) Where, a'=a+u, and b'=bu*y r .

[0106] With this choice, the mapping function is centered around the brightness value y r It is recommended to use the average value of the reference brightness samples used in model creation as the y r , in order to provide meaningful modifications to the model. FIG. 8A to FIG. 8B shows the effect of scaling adjustment parameter "u", where Fig. 8A shows a model created without the scaling tuning parameter "u", and Figure 8B A model created with a scaling adjustment parameter "u" is shown.

[0107] In one example, the scaling adjustment parameter is provided as an integer between -4 and 4 (inclusive) and signaled in the bitstream. The unit of the scaling adjustment parameter is 1 / 8 of the chroma sample value per luma sample value (for 10-bit content).

[0108] In one example, CCLM models ("LM_CHROMA_IDX" and "MMLM_CHROMA_IDX") that use reference samples above and to the left of a block are adjusted, but not used for "one-sided" mode. This choice is based on a coding efficiency versus complexity tradeoff.

[0109] When scaling adjustments are applied for a multi-mode CCLM model, both models may be adjusted and thus up to two scaling updates may be signaled for a single chroma block.

[0110] To implement scaling at the encoder, the encoder may perform a SATD-based search for the best value of scaling update for Cr and a similar SATD-based search for Cb. If either result is a non-zero scaling adjustment parameter, the combined scaling adjustment pair (SATD-based update for Cr, SATD-based update for Cb) is included in the list of RD checks for TU.

[0111] Fusion of Chroma Intra Prediction Modes

[0112] During the development of ECM, JVET-Y0092 / Z0051 proposed the fusion of chroma intra mode.

[0113] The intra prediction modes enabled for chroma components in ECM-4.0 are six cross-component linear model (LM) modes, including CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L and MMLM_T modes, direct mode (DM) and four default chroma intra prediction modes. The four default modes are given by the list {0,50,18,1}, and if the DM mode already belongs to the list, the mode in the list will be replaced by mode 66.

[0114] The decoder-side intra mode derivation (DIMD) method for luma intra prediction is included in ECM-4.0. First, the horizontal gradient and vertical gradient are calculated for each reconstructed luma sample of the L-shaped template of the second adjacent row and column of the current block to construct a gradient histogram (Histogram of Gradients, HoG). Then, the two intra prediction modes with the largest histogram amplitude value and the second largest histogram amplitude value are mixed with the planar mode to generate the final prediction value of the current luma block.

[0115] In order to improve the coding efficiency of chroma intra prediction, two methods are proposed, including decoder-side derived chroma intra prediction mode (DIMD chroma) and the fusion of non-LM mode and MMLM_LT mode.

[0116] In the first embodiment, a DIMD chroma mode is proposed. The proposed DIMD chroma mode uses the DIMD derivation method to derive the chroma intra prediction mode of the current block based on the co-located reconstructed luma samples. Specifically, the horizontal gradient and the vertical gradient are calculated for each co-located reconstructed luma sample of the current chroma block to construct the HoG, such as Figure 8C Then, the intra prediction mode with the maximum histogram magnitude value is used to perform chroma intra prediction of the current chroma block.

[0117] When the intra prediction mode derived according to the DIMD chroma mode is the same as the intra prediction mode derived according to the DM mode, the intra prediction mode having the second largest histogram magnitude value is used as the DIMD chroma mode.

[0118] A CU level flag is signaled to indicate whether the proposed DIMD chroma mode is applied, as shown in Table 7. Table 7. Binarization process for intra_chroma_pred_mode in the proposed method intra_chroma_pred_mode Binary String Chroma Intra Mode 0 1100 List[0] 1 1101 List[1] 2 1110 List [2] 3 1111 List[3] 4 10 DIMD Chroma 5 0 DM

[0119] In a second embodiment, a fusion of chroma intra prediction modes is proposed, where the DM mode and four default modes can be fused with the MMLM_LT mode as follows: pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift Where pred0 is the prediction value obtained by applying the non-LM mode, pred1 is the prediction value obtained by applying the MMLM_LT mode, and pred is the final prediction value of the current chroma block. The two weights w0 and w1 are determined by the intra prediction mode of the adjacent chroma block and are set equal to 2. Specifically, when the adjacent blocks above and to the left are both encoded in the LM mode, {w0, w1} = {1, 3}; when the adjacent blocks above and to the left are both encoded in the non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}.

[0120] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied. And the proposed fusion is only applied to I slices.

[0121] In the third embodiment, the fusion of the DIMD chroma mode and the chroma intra prediction mode is combined. Specifically, the DIMD chroma mode described in the first embodiment is applied, and for I slices, the DM mode, the four default modes, and the DIMD chroma mode can be fused with the MMLM_LT mode using the weights described in the second embodiment, while for non-I slices, only the DIMD chroma mode can be fused with the MMLM_LT mode using equal weights.

[0122] In a fourth embodiment, a fusion of the DIMD chroma mode with reduced processing and the chroma intra prediction mode is combined. Specifically, the DIMD chroma mode with reduced processing derives the intra mode based on the adjacent reconstructed Y, Cb and Cr samples in the second adjacent rows and columns, such as Fig.8D The other components are the same as those of the third embodiment.

[0123] In one embodiment, when DIMD is applied, two intra modes are derived from reconstructed neighboring samples and these two predictions are combined with the planar mode prediction, where the weights are derived from the gradients as described in JVET-00449. The division operations in the weight derivation are performed using the same lookup table (LUT) based integerization scheme used by CCLM. For example, the division operations in the direction calculation Orient=G y / G x It is calculated by the following LUT-based scheme: x=Floor(Log2(Gx)) normDiff=((Gx<<4)>>x)&15 x+=(3+(normDiff!=0)?1:0) Orient=(Gy*(DivSigTable[normDiff]|8)+(1<<(x-1)))>>x in DivSigTable

[16] ={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}.

[0124] The derived intra modes are included into a master list of intra most probable modes (MPMs), so the DIMD process is performed before building the MPM list. The master derived intra modes of a DIMD block are stored with the block and used for MPM list construction of neighboring blocks.

[0125] FIG. 8E to FIG. 8H The steps of intra mode derivation at the decoder side are shown, where the intra prediction direction is estimated without intra mode signaling. Fig. 8E The first step shown consists in estimating for each sample point (for example Fig. 8E The gradient of the light grey sample points shown in Figure 2 is shown in Figure 2. Fig.8F The second step shown consists of mapping the gradient value to the closest prediction direction within [2, 66]. Figure 8G The third step shown includes: selecting 2 prediction directions, wherein for each prediction direction, all absolute gradients Gx and Gy of neighboring pixels having this direction are summed, and the first 2 directions are selected. Figure 8H The fourth step shown includes enabling weighted intra prediction with the selected direction.

[0126] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Fig. 9 In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not extracted from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra-picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0127] The index of the selected reference line (mrl_idx) is signaled and used to generate intra prediction values. For reference line idx greater than 0, only additional reference line modes in the MPM list are included and only the mpm index is signaled if there are no remaining modes. The reference line index is signaled before the intra prediction mode and the planar mode is excluded from the intra prediction mode if a non-zero reference line index is signaled.

[0128] MRL is disabled for blocks of the first line inside a CTU to prevent the use of extended reference samples outside the current CTU line. In addition, PDPC is disabled when additional lines are used. For MRL mode, the derivation of DC values ​​for non-zero reference line indices in DC intra prediction mode is aligned with the derivation of reference line index 0. MRL requires 3 adjacent luma reference lines to be stored with the CTU to generate the prediction. The Cross Component Linear Model (CCLM) tool also requires 3 adjacent luma reference lines for its downsampling filter. The definition of MRL using the same 3 lines is aligned with CCLM to reduce the storage requirements for the decoder.

[0129] During the development of ECM, the Convolutional Cross-Component Model (CCCM) for Chroma Intra Mode was proposed.

[0130] It is proposed to apply a convolutional cross-component model (CCCM) to predict chroma samples from reconstructed luma samples in a similar spirit to what the current CCLM mode does. As with CCLM, when chroma subsampling is used, the reconstructed luma samples are downsampled to match the lower resolution chroma grid.

[0131] Also, similar to CCLM, there is an option to use a single model or a multi-model variant of CCCM. The multi-model variant uses two models, one derived for samples above the average luminance reference value and the other for the remaining samples (following the spirit of the CCLM design). The multi-model CCCM mode can be selected for PUs with at least 128 available reference samples.

[0132] The proposed convolutional 7-tap filter includes a 5-tap + sign shape spatial component, a nonlinear term, and a bias term. The input of the spatial 5-tap component of the filter includes the center (C) luminance sample, which is co-located with the chrominance sample to be predicted and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, as shown in Fig. 10A as shown in .

[0133] The nonlinear term P is expressed as a power of two in the center luma sample C and scaled to the sample value range of the content:

[0134] P=(C*C+midVal)>>bitDepth

[0135] That is, for 10 bits of content, it is calculated as:

[0136] P=(c*c+512)>>10

[0137] The bias term B represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the intermediate chrominance value (512 for 10-bit content).

[0138] The output of the filter is calculated as: The filter coefficient c i Convolution with the input value and clipped to the range of valid chroma samples:

[0139] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0140] The filter coefficients c are calculated by minimizing the MSE between the predicted chroma samples and the reconstructed chroma samples in the reference region i . Fig. 10B A reference region consisting of 6 lines of chroma samples above and to the left of the PU is shown. The reference region extends one PU width to the right and one PU height below the PU boundary. The region is adjusted to include only available samples. The extension of the region shown in blue is needed to support the "side samples" of the +shape spatial filter and to fill them in when they are in an unavailable region.

[0141] MSE minimization is performed by computing the autocorrelation matrix for the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​LDL decomposed and the final filter coefficients are calculated using back substitution. The process roughly follows the calculation of the ALF filter coefficients in ECM, however LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations. The proposed method uses only integer arithmetic.

[0142] The use of the mode is signaled with the CABAC codec PU-level flag. A new CABAC context is included to support this. CCCM is considered a sub-mode of CCLM when it comes to signaling. That is, the CCCM flag is signaled only when the intra prediction mode is LM_CHROMA_IDX (to enable single-mode CCCM) or MMLM_CHROMA_IDX (to enable multi-model CCCM).

[0143] The encoder performs two new RD checks in the chroma prediction mode loop, one for checking single-model CCCM mode and one for checking multi-model CCCM mode.

[0144] In the existing CCLM or MMLM design, adjacent reconstructed luma-chroma sample pairs are classified into one or more sample groups based on the value Threshold, which only considers the luma DC value. That is, the luma-chroma sample pairs are classified by considering only the intensity of the luma sample. However, the luma component usually retains rich textures, and the current luma sample may be highly correlated with the adjacent luma samples. This inter-sample correlation (AC correlation) can benefit the classification of luma-chroma sample pairs and can bring additional coding efficiency.

[0145] In addition, if Fig. 10C As shown in , CCLM assumes that a given chrominance sample is only related to the corresponding luma sample (L0.5, which can be regarded as the fractional luma sample position), and uses simple linear regression (SLR) with ordinary least squares (OLS) estimation to predict the given chrominance sample. Fig. 10D As shown in , in some video contents, a chrominance sample may be simultaneously correlated with multiple luminance samples (AC or DC correlated), so a multiple linear regression (MLR) model can further improve the prediction accuracy.

[0146] Although the CCCM mode can improve intra-frame prediction efficiency, there is room for further improvement in its performance. At the same time, some parts of the existing CCCM mode also need to be simplified for efficient codec hardware implementation, or improved to achieve better codec efficiency. In addition, it is necessary to further improve the trade-off between its implementation complexity and its codec efficiency benefits.

[0147] Marginal Linear Model (ELM)

[0148] In order to improve the coding efficiency of the luma component and the chroma component, a classifier that considers luma edge or AC information is introduced, which is in contrast to the above-mentioned embodiment in which only the luma DC value is considered. In addition to the existing band classification MMLM, the present disclosure also provides an exemplary classifier. The process of generating a linear prediction model for different sample groups can be similar to CCLM or MMLM (e.g., via least squares, or a simplified minimum-maximum method, etc.), but with different metrics for classification. Different classifiers can be used to classify adjacent luma samples (e.g., adjacent luma samples of adjacent luma-chroma sample pairs) and / or luma samples corresponding to the chroma samples to be predicted. Luma samples corresponding to chroma samples can be obtained by downsampling operations to match the positions of corresponding chroma samples for 4:2:0 video sequences. For example, luma samples corresponding to chroma samples can be obtained by performing downsampling operations on more than one (e.g., 4) reconstructed luma samples corresponding to chroma samples (e.g., located around chroma samples). Alternatively, for example, in the case of a 4:4:4 video sequence, the luma sample may be obtained directly from the reconstructed luma sample. Alternatively, the luma sample may be obtained from a corresponding luma sample in the reconstructed luma sample located at a corresponding co-located position of the corresponding chroma sample. For example, the luma sample to be classified may be obtained from one of the four reconstructed luma samples corresponding to the chroma sample, the reconstructed luma sample being located at an upper left position of the four reconstructed luma samples, and the upper left position may be considered as a co-located position of the chroma sample.

[0149] The first classifier can classify the luma samples according to the edge strength of the luma samples. For example, a direction (e.g., 0 degrees, 45 degrees, or 90 degrees, etc.) can be selected to calculate the edge strength. The direction can be formed by the current sample and the adjacent sample along the direction (e.g., the adjacent sample is located 45 degrees to the upper right of the current sample). The edge strength can be calculated by subtracting the adjacent sample from the current sample. The edge strength can be quantized into one of the M segments by M-1 thresholds, and the first classifier can use M classes to classify the current sample. Alternatively or additionally, N directions can be formed by the current sample and N adjacent sample along N directions. N edge strengths can be calculated by subtracting N adjacent sample from the current sample respectively. Similarly, if each of the N edge strengths can be quantized into one of the M segments by M-1 thresholds, the first classifier can use MN classes to classify the current sample.

[0150] The second classifier can be used to classify based on local patterns. For example, the current luma sample Y0 can be compared with its N neighboring luma samples Yi. If the value of Y0 is greater than the value of Yi, the score can be increased by one point, otherwise, the score can be reduced by one point. The score can be quantized to form K classes. The second classifier can classify the current sample into one of the K classes. For example, the neighboring luma samples can be obtained from four neighbors located above, to the left, to the right, and below the current luma sample, i.e., there are no diagonal neighbors.

[0151] It is contemplated that multiple first classifiers, second classifiers, or different instances of the first classifier or the second classifier or other classifiers described herein may be combined. For example, the first classifier may be combined with an existing MMLM threshold-based classifier. For another example, an instance A of the first classifier may be combined with another instance B of the first classifier, wherein instance A and instance B are in different directions (e.g., vertical and horizontal directions, respectively).

[0152] Those skilled in the art will appreciate that, although the existing CCLM design in the VVC standard is used as the basic CCLM method in the specification, the proposed cross-component method described in the present disclosure can also be applied to other predictive codecs with similar design spirits. For example, for chroma from luma (CfL) in the AV1 standard, the proposed method can also be applied by dividing luma / chroma sample pairs into multiple sample groups.

[0153] The skilled person will understand that Y / Cb / Cr can also be represented as Y / U / V in the field of video coding. If the video data is in RGB format, for example, the proposed method can also be applied by simply mapping the YUV symbols to GBR.

[0154] Filter-based Linear Model (FLM)

[0155] A filter-based linear model (FLM) utilizing the MLR model is introduced as follows to take into account the possibility that one chrominance sample may be simultaneously correlated with multiple luma samples.

[0156] For the chroma samples to be predicted, the reconstructed co-located and adjacent luma samples can be used to predict the chroma samples to capture the inter-sample correlation between co-located luma samples, adjacent luma samples, and chroma samples. The reconstructed luma samples are linearly weighted and combined with an "offset" to generate predicted chroma samples (C: predicted chroma samples, L i : the i-th reconstructed co-located or adjacent brightness sample, α i: filter coefficient, β: offset, N: filter tap), as shown in the following formula (32-1). Note that the linear weighted + offset value directly forms the predicted chroma sample (which can be adaptively low-passed or high-passed according to the video content), and is then added to the residual to form the reconstructed chroma sample.

[0157] In some implementations similar to CCCM, the offset term may also be implemented as the intermediate chrominance value B (512 for 10-bit content) multiplied by another coefficient, as shown in equation (32-2) below.

[0158] For a given CU, the top and left reconstructed luma and chroma samples can be used to derive or train FLM parameters (α2, β). Similar to CCLM, α i and β can be derived via OLS. The top and left training samples are collected and a pseudo-inverse matrix is ​​computed on both the encoder and decoder sides to derive the parameters, which are then used to predict the chroma samples in a given CU. Let N denote the number of filter taps applied to the luma samples, M denote the total top and left reconstructed luma-chroma sample pairs used for training the parameters, represents the luma sample with the i-th sample pair and the j-th filter tap, C i Denotes the chrominance sample with the i-th sample pair, the following formula shows the pseudo-inverse matrix A + and the derivation of parameters. Fig.11 An example is shown where N is 6 (6 taps), M is 8, the top 2 rows and left 3 columns of luma samples, and the top 1 row and left 1 column of chroma samples are used to derive or train parameters.

[0159] Note that it is possible to pass only α without the bias β i To predict the chrominance samples, the offset β can be a subset of the proposed method.

[0160] The proposed ELM / FLM / GLM (described below) can be directly extended to the CfL design in the AV1 standard, which explicitly transmits the model parameters (α, β). For example, (1-tap case) α and / or β are derived at the encoder at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / Subblock / sample level and signaled to the decoder for CfL mode.

[0161] To further improve the encoding and decoding performance, additional designs can be used in FLM prediction. Fig.11As shown and discussed above, a 6-tap luma filter is used for FLM prediction. However, although the multi-tap filter can fit the training data (e.g., the top and left adjacent reconstructed luma samples and chroma samples) well, in some cases, the training data does not capture all the characteristics of the test data, which may lead to overfitting and may not predict the test data (i.e., the chroma block samples to be predicted) well. In addition, different filter shapes can adapt well to different video block contents, resulting in more accurate predictions.

[0162] To address this issue, the filter shape and number of filter taps may be predefined or signaled or switched at the sequence parameter set (SPS), adaptive parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, CTU, CU, sub-block, or sample level. A set of filter shape candidates may be predefined, and the selection of the set of filter shape candidates may be signaled or switched at the SPS, APS, PPS, PH, SH, region, CTU, CU, sub-block, or sample level. Different components (e.g., U and V) may have different filter switching controls. For example, a set of filter shape candidates may be predefined (e.g., indicated by indexes 0 to 5), and as Fig.11 As shown, filter shape (1, 2) can represent a 2-tap luminance filter, filter shape (1, 2, 4) can represent a 3-tap luminance filter, etc. The filter shape selection for U and V components can be switched in the PH or in the CU or CTU level. Note that N taps can represent N taps with or without the offset β described herein. An example is given in Table 8 below. Table 8 - Example signaling and switching for different filter shapes

[0163] The FLM or CCCM filter shape may include nonlinear terms. For example, for a CCCM filter, the chroma sample values ​​may be predicted using the following formula: predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B,P=(C*C+midVal)>>bitDepth The filter corresponds to the weighted coefficients c0, c1, ... c6 of the center (C) luma sample value co-located with the chroma sample to be predicted, its upper / north (N), lower / south (S), right / east (E) and left / west (W) neighbors, the nonlinear term P and the bias term B. The value used to derive the nonlinear term P can be a combination of the current luma sample and the neighboring luma samples, but is not limited to C*C. For example, P can be derived as follows: P=(Q*R+midVal)>>bitDepth Where Q and R represent the values ​​used to derive the nonlinear term P.

[0164] Q and R can be linear combinations of the current luma sample and adjacent luma samples in the downsampled domain (eg, Q and R are pre-computed luma samples obtained by weighted average operation) or without any downsampling process.

[0165] For example, each of Q and R can be selected from one of N, S, E, W and C brightness sample values, such as Q*R=C*N, C*S, C*E, C*W, S*N or N*N, etc.; or Q and R can both be equal to the average of N, S, E and W brightness sample values, that is, Q=R=(N+S+E+W) / 4; or, Q is equal to the C brightness sample value, and R is equal to the average of N, S, E and W brightness sample values, that is, Q=C, and R=(N+S+E+W) / 4.

[0166] Different values ​​(Q and R) used to derive the non-linear terms are considered different filter shapes and can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of filter shape candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0167] Different chroma types and / or color formats may have different predefined filter shapes and / or taps. Fig.12 As shown, the predefined filter shape (1, 2, 4, 5) can be used for 4:2:0 type-0, the predefined filter shape (0, 1, 2, 4, 7) can be used for 4:2:0 type-2, and the predefined filter shape (1, 4) can be used for 4:2:2, and the predefined filter shape (0, 1, 2, 3, 4, 5) can be used for 4:4:4.

[0168] In another aspect of the present disclosure, unavailable luma and chroma samples used to derive the MLR model may be filled from available reconstructed samples. Fig.12 6-tap (0, 1, 2, 3, 4, 5) filter shown, then for the CU located at the left picture boundary, the left column including sample (0, 3) is unavailable (beyond the picture boundary), so sample (0, 3) is a repeated padding from sample (1, 4) to apply the 6-tap filter. Note that the padding process can be applied to both training data (top and left adjacent reconstructed luma and chroma samples) and test data (luma and chroma samples in the CU).

[0169] One or more shapes / numbers of filter taps can be used for FLM prediction, examples are given in Fig.16 , Fig.17 and FIG. 18A to FIG. 18B One or more sets of filter taps may be used for FLM prediction, an example being FIG. 19A to FIG. 19G Shown in.

[0170] The filter shape candidate may be derived implicitly without explicitly signaling bits. For example, the filter shape candidate may be a filter shape candidate for FLM or GLM (as discussed below). In another example, the filter shape candidate may be a cross-shape filter for CCCM, Fig.16 , Fig.17 , Fig.18A , Fig.18B and FIG. 19A to FIG. 19G Any filter shown in , or other filters mentioned in this disclosure. Since longer filter taps always theoretically fit the training data (template region) better, but may overfit, the well-known "N-fold cross validation" technique in the field of machine learning can be used to train the filter coefficients. This technique divides the available training data into N groups and uses some of the groups for training and the other groups for validation.

[0171] The following example involves implicit filter shape derivation for FLM prediction: Step 1: Determine M filter shape candidates for predicting the chroma sample values ​​of the current CU; Step 2: Divide the available L-shaped template area outside the CU into N regions, marked as R0, R1, ... R N-1 , that is, the training data is divided into N groups for N-fold training, where the luminance sample values ​​and chrominance sample values ​​of the available template area are known values; Step 3: Apply each of the M filter shape candidates to a portion of the available template area, i.e., all N regions R0, R1, ... R N-1 one or more areas in Step 4: Draw M sets of filter coefficients corresponding to the M filter shape candidates, labeled F0, F1, ... F M-1 ; Step 5: Substitute the derived F0, F1, …F M-1 Applying the filter coefficient set to another portion of the available template area to predict chrominance sample values ​​based on corresponding luminance sample values, wherein the other portion of the available template area is different from the portion of the available template area mentioned in step 3; Step 6: For each of the M filter shapes, accumulate the error between the predicted chrominance sample values ​​and the known chrominance sample values ​​in another part of the available template area by sum of absolute difference (SAD), sum of squared difference (SSD) or sum of absolute transformed difference (SATD), denoted as E0, E1, ... E M-1 ; Step 7: Sort and select the K smallest errors, labeled E'0, E'1, ... E' K-1 , which corresponds to K filter shapes and K sets of filter coefficients; and Step 8: Select a filter shape candidate from the K filter shape candidates to apply to the current CU for chroma prediction. If K is greater than 1, the decoder can still receive a signal from the encoder indicating the filter to be applied. However, if K is 1, the signaling can be omitted and the filter with the smallest cumulative error is determined as the filter to be applied.

[0172] Fig. 20A and Fig. 20B An example of 2-fold training for implicit filter shape derivation is shown. For the current chroma CU prediction (blue area), Fig. 20A It shows that in the template area, the even-numbered row area R0 (yellow) is used for training / deriving 4 sets of filter coefficients, and the odd-numbered row area R1 (red) is used for verification / comparison and sorting costs of the 4 sets of filter coefficients; Fig. 20B It is shown that in the template area, R0 (yellow) and R1 (red) are interleaved. It should be understood that R0 and R1 can be swapped in these examples.

[0173] In one example, one of the four filter shape candidates is selected as the applied filter, and the L-shaped template area is divided into even and odd rows or columns. The steps include: Predefine 4 filter shape candidates for the current CU (e.g., from Fig.16 4 filter shape candidates of ); The available L-shaped template area (e.g., 6 chroma rows and columns for CCCM, note that in CCCM design, each chroma sample involves 6 luma samples for downsampling) is divided into 2 regions labeled R0, R1, where, for example, R0 consists of even-numbered rows or columns, and R1 consists of odd-numbered rows or columns ( Fig. 20A The following example is shown: in the template area, the even-numbered row area R0 is used for training / deriving 4 sets of filter coefficients, and the odd-numbered row area R1 is used for verification / comparison and sorting costs of the 4 sets of filter coefficients); Apply the 4 filter shape candidates independently to a portion of the available template area (e.g., area R0); Four sets of filter coefficients for the four filter shapes are derived respectively, labeled F0, F1, ... F3; Apply the derived F0, F1, ... F3 filter coefficient sets to another part of the available template area (e.g., area R1) to predict the corresponding chrominance sample values; For each of the four filters, respectively, the errors between the predicted chrominance sample values ​​and the known chrominance sample values ​​in another part of the available template area (e.g., area R1) are accumulated by SAD, SSD or SATD, marked as E0, E1, ... E3; A filter shape candidate having the minimum accumulated error (labeled as E'0) among the 4 filter shape candidates is selected, which corresponds to a filter shape and a set of filter coefficients. In this example, only the filter shape candidate having the minimum accumulated error will be determined as the applied filter, and then the decoder does not need to receive a signal indicating the applied filter.

[0174] In one example, the L-shaped template area can be divided into staggered parts, where K=2. The steps include: Determine 4 filter shape candidates for the current CU; The available L-shaped template area (e.g., 6 chroma rows or columns for CCCM) is divided into 2 regions labeled R0, R1, where the luma samples in R0 and R1 are interlaced, for example, as shown in the following table: <![CDATA[R0]]> <![CDATA[R1]]> <![CDATA[R1]]> <![CDATA[R0]]> or <![CDATA[R1]]> <![CDATA[R0]]> <![CDATA[R0]]> <![CDATA[R1]]> and Fig. 20B An example of interleaving of R0 and R1 in the template region is shown; Apply the 4 filter shape candidates independently to a portion of the available template area (e.g., area R0); Four sets of filter coefficients for the four filter shapes are derived respectively, denoted as F0, F1, ... F3; Apply the derived F0, F1, ... F3 filter coefficient sets to another part of the available template area (e.g., area R1) to predict the corresponding chrominance sample values; For each of the 4 filter shapes, the errors between the predicted chroma sample values ​​and the known chroma sample values ​​in another part of the available template area (e.g., area R1) are accumulated by SAD, SSD, or SATD, respectively, marked as E0, E1, ... E3; Sort and select the two smallest accumulated errors among E0, E1, ... E3, labeled as E'0, E'1, which correspond to two filter shapes and two sets of filter coefficients; and Based on the signal received from the encoder, one of the two filter shape candidates is selected to be applied to the current CU for chroma prediction.

[0175] Note that the method of implicit filter shape derivation can also be used to determine whether to introduce nonlinear terms in the CCCM filter coefficients (the cases with / without nonlinear terms are regarded as different filter shapes).

[0176] Although examples are shown above for CCCM filters, it should be understood that the nonlinear term P may also be included in FLM filters (e.g., Fig.11 ) and is derived in a manner similar to that discussed above.

[0177] In one example, the step of dividing the available L-shaped template region may be omitted. In this example, M sets of filter coefficients may be derived based on sample values ​​from the available template region and then applied back to the available template region to predict corresponding chrominance sample values ​​for accumulating errors.

[0178] As mentioned above, the MLR model (linear equation) must be derived at both the encoder and the decoder. According to one or more aspects of the present disclosure, several methods are proposed to derive the pseudo-inverse matrix A + , or solve the linear equation directly. Other known methods can be applied, such as Newton's method, Cayley-Hamilton method and Eigendecomposition mentioned at https: / / en.wikipedia.org / wiki / Invertible_matrix.

[0179] In this disclosure, for simplicity, A + Indicated as A -1 . The linear equation can be solved as follows: 1. Solve A through the decision matrix (adjA) -1 , closed form analytical solution: Below is shown an nxn general form, a 2x2 and a 3x3 case. If FLM uses 3x3, then 2 scalers need to be solved plus an offset. b=Ax,x=(A T A) -1 A T b=A + b, marked as A -1 b By removing the (n - 1) x (n - 1) submatrix in the j-th row and i-th column 2. Gauss-Jordan Elimination Gauss-Jordan elimination can be used to solve linear equations by augmenting the matrix [A I n and performing a series of elementary row operations to obtain the reduced row echelon form [I|X]. 2x2 and 3x3 examples are shown below. 3. Cholesky Decomposition To solve Ax = b, A can first be decomposed by the Cholesky-Crout algorithm, resulting in an upper triangular matrix and a lower triangular matrix, and a forward substitution followed by a back substitution can be applied in sequence to obtain the solution. A 3x3 example is shown below.

[0180] In addition to the above examples, some conditions require special handling. For example, if some conditions result in the inability to solve linear equations, default values can be used to fill the chrominance prediction values. The default values can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, when predefined 1 << (bit Deph - 1), meanC, meanL, or meanC - meanL (average current chrominance or other chrominance, from available luminance values, or a subset of the FLM reconstructed adjacent regions).

[0181] The following examples represent cases where matrix A cannot be solved, where default prediction values can be assigned to the entire current block: 1. Solving by closed form (analytical, conjugate matrix), but A is singular (i.e., detA = 0); 2. Solving by Cholesky decomposition, but A cannot be Cholesky decomposed, g jj <REG_SQR, where REG_SQR is a small value that can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0182] Fig.11A typical case of using top 2 and / or left 3 luma lines and top 1 and / or left 1 chroma line to derive FLM parameters is shown. However, as mentioned above, using different regions for parameter derivation can bring codec benefits due to different block contents and reconstruction quality of different neighboring samples. Several ways of selecting the application region for parameter derivation are proposed below: 1. Similar to MDLM, FLM derivation can use only the top or left luminance and / or chrominance samples to derive parameters. FLM, FLM_L or FLM_T can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Assuming the size of the current chrominance block is W×H, W' and H' are obtained as follows: —When FLM mode is applied, W'=W, H'=H; - When FLM_T mode is applied, W'=W+We; where We represents the extended top luma / chroma sample; - When FLM_L mode is applied, H'=H+He; where He represents the extended left luma / chroma samples. The number of extended luma / chroma samples (We, He) may be predefined, or signaled or switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, (We, He)=(H, W) is predefined as a VVC CCLM, or (W, H) is predefined as an ECM CCLM. Unavailable (We, He) luma / chroma samples can be repeatedly filled from the nearest (horizontal, vertical) luma / chroma samples. Fig.13 A diagram of FLM_L and FLM_T (eg, at 4 taps) is shown. When FLM_L or FLM_T is applied, only H' or W' luma / chroma samples, respectively, are used for parameter derivation. 2. Similar to MRL, different line indices can be predefined or signaled or switched in SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to indicate the selected luma-chroma sample pairs. This can benefit from different reconstruction qualities of different line samples. Fig.14 It is shown that similar to MRL, FLM can use different lines for parameter derivation (e.g., at 4 taps). For example, FLM can use the light blue / yellow luma and / or chroma samples in index 1. 3. The expanded CCLM area and the full top N and / or left M lines are used for parameter derivation. Fig.14It is shown that all dark blue and light blue and yellow areas can be used simultaneously. Training using larger areas (data) can lead to more robust MLR models.

[0183] It should be understood that in the present disclosure, luma sample values ​​of an external region of a video block to be decoded may be referred to as “external luma sample values”, and chroma sample values ​​of an external region may be referred to as “external chroma sample values”.

[0184] The corresponding syntax may be defined for FLM prediction as shown in Table 9 below. Wherein, FLC represents a fixed length code, TU represents a truncated unary code, EGk represents an exponential-golomb code with order k, where k may be fixed or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level, SVLC represents signed EG0, and UVLC represents unsigned EG0. Table 9 — Example of FLM syntax

[0185] NOTE: The binarization of each syntax element can be changed.

[0186] A new method for cross-component prediction is proposed based on the existing linear model design to further improve the encoding and decoding accuracy and efficiency. The main aspects of the proposed method are detailed as follows.

[0187] Although the FLM discussed above provides the best flexibility (leading to the best performance), if the number of filter taps increases, many unknown parameters need to be solved. When the inverse matrix is ​​larger than 3x3, closed form derivation is not suitable (too many multipliers) and iterative methods such as Cholesky are required, which burdens the decoder processing cycle. In this section, pre-operation before applying the linear model is proposed, including the use of sample gradients to exploit the correlation between the luminance AC information and the chrominance intensity. With the help of gradients, the number of filter taps can be effectively reduced.

[0188] Note that the methods / examples in this section can be combined / reused from any of the designs discussed above, including but not limited to classification, filter shapes, matrix derivation (with special handling), application areas, syntax. In addition, the methods / examples listed in this section can also be applied to any of the designs discussed above to have better performance with certain complexity tradeoffs.

[0189] Please note that reference samples / training templates / reconstructed neighboring regions as used herein generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in one CU to predict chroma samples in the CU.

[0190] According to the proposed method, instead of directly using the luma sample intensity values ​​as input to the linear model, pre-operations (e.g., pre-linear weighting, sign, scaling / abs, thresholding, ReLU) can be applied to reduce the dimensionality of the unknown parameters. In one example, the pre-operations can include: calculating sample differences based on the luma sample values. As understood by those skilled in the art, sample differences can be characterized as gradients, and therefore the new method is also referred to as a gradient linear model (GLM) in some embodiments.

[0191] Note that the following detailed description discusses scenarios in which the proposed pre-operation can be reused for the SLR model (also known as the 1-tap case) / combined with the SLR model, and can be reused for the MLR model (also known as the multi-tap case, such as 2-tap) / combined with the MLR model.

[0192] For example, instead of applying a 2-tap to 2 luma samples, the 2 luma samples may be pre-computed and then a simpler 1-tap may be applied to reduce complexity. FIG. 15A to FIG. 15D Some examples of 1-tap / 2-tap (with offset) pre-operation are shown, where the 2-tap coefficients are represented as (a, b). Note that FIG. 15A to FIG. 15D Each circle shown represents a schematic chrominance position in the YUV4:2:0 format. As discussed above, in the YUV 4:2:0 format, a luma sample corresponding to a chrominance sample can be obtained by performing a downsampling operation on more than one (e.g., 4) reconstructed luma samples corresponding to (e.g., located around) the chrominance sample. In other words, a chrominance position can correspond to one or more luma samples including a co-located luma sample. Different 1-tap modes are designed for different gradient directions, and different "interpolated" luma samples (weighting different luma positions) are used for gradient calculations. For example, a typical filter [1, 0, -1; 1, 0, -1] in Fig.15A , Fig. 15C and Fig.15D It is shown in , which represents the following operations: Among them, rec L Represents the reconstructed luminance sample value, and Rec L ″(i,j) represents the pre-calculated brightness sample value. Also note that Fig.15A , Fig. 15C and Fig.15DThe 1-tap filter shown in can be understood as an alternative to the downsampling filter with changed filter coefficients used in CCLM (see equations (6)-(7)).

[0193] Pre-operation can be based on gradient, edge direction (detection), pixel intensity, pixel change, pixel variance, Roberts / Prewitt / compass / Sobel / Laplacian operator, high pass filter (by calculating gradient or other related operators), low pass filter (by performing weighted average operation), etc. The edge direction detector listed in the example can be extended to different edge directions. For example, 1 tap (1, -1) or 2 taps (a, b) are applied along different directions to detect different edge gradients. The filter shape / coefficient can be symmetric with respect to the chrominance position, such as FIG. 15A to FIG. 15D Example (420 Type 0 scenario).

[0194] Pre-operation parameters (coefficients, signs, scaling / abs, thresholding, ReLU) can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Note that in the example, if multiple coefficients apply to one sample (e.g., -1, 4), they can be merged (e.g., 3) to reduce operations.

[0195] In one example, pre-operation may involve calculating sample differences of luma sample values. Optionally, pre-operation may include performing downsampling by a weighted average operation. In some cases, pre-operation may be applied repeatedly. For example, a template filter may be applied to the template using a low-pass smoothing FIR filter [1, 2, 1] / 4 or [1, 2, 1; 1, 2, 1] / 8 (i.e., downsampling) to remove outliers, and then a 1-tap GLM filter may be applied to calculate sample differences to derive a linear model. It is contemplated that sample differences may also be calculated and then downsampling may be enabled.

[0196] In one example, pre-operation coefficients (either ultimately applied (eg, 3) or intermediately applied (eg, -1, 4) to each luma sample) may be limited to powers of 2 to save multipliers.

[0197] In one aspect of the present disclosure, the proposed new method can be reused / combined with the CCLM discussed above, which uses a simple linear regression (SLR) model and uses one corresponding luma sample value to predict the chroma sample value. This is also called the 1-tap case. In this case, deriving the linear model also includes: deriving the scaling parameter α and the offset parameter β by using the pre-calculated adjacent luma sample values ​​and adjacent chroma sample values. Alternatively, the linear model can be rewritten as: C=α·L+β (35) Where L here represents the "pre-computed" luma sample. The parameter derivation of the 1-tap GLM can reuse the CCLM design, but taking into account the directional gradient (possibly with a high-pass filter). In one example, the scaling parameter α can be derived by using a division lookup table as detailed below for simplification.

[0198] In one example, when the GLM is combined with the SLR model, the scaling parameter α and the offset parameter β can be derived by utilizing the minimum-maximum method discussed above. Specifically, the scaling parameter α and the offset parameter β can be derived by comparing the adjacent pre-calculated luminance sample values ​​to determine the minimum luminance sample value Y A and the maximum brightness sample value Y B ; are the minimum brightness sample value Y A and the maximum brightness sample value Y B To determine the corresponding chrominance sample value X A and X B ; and, based on the minimum brightness sample value Y A , maximum brightness sample value Y B and the corresponding chrominance sample value X A and X B , the scaling parameter α and the offset parameter β are derived according to the following formula:

[0199] In one example, when the GLM is combined with the SLR model, the scaling adjustment discussed above can be reused. In this case, the encoder can determine the scaling adjustment value (e.g., "u") signaled in the bitstream and add the scaling adjustment value to the derived scaling parameter α. The decoder can determine the scaling adjustment value (e.g., "u") from the bitstream and add the scaling adjustment value to the derived scaling parameter α. The added value is ultimately used to predict the intra chroma sample value.

[0200] In one aspect of the present disclosure, the proposed new method can be reused / combined with FLM, which utilizes a multiple linear regression (MLR) model and uses multiple luma sample values ​​to predict chroma sample values. This is also called a multi-tap case, such as 2 taps. In this case, the linear model can be rewritten as:

[0201] In this case, multiple scaling parameters α and offset parameters β may be derived by using adjacent luma sample values ​​and adjacent chroma sample values ​​that have been pre-operated. In one example, the offset parameter β is optional. In one example, at least one of the multiple scaling parameters α may be derived by using sample differences. In addition, another scaling parameter of the multiple scaling parameters α may be derived by using downsampled luma sample values. In one example, at least one of the multiple scaling parameters α may be derived by using horizontal or vertical sample differences calculated based on downsampled adjacent luma sample values. In other words, the linear model may combine multiple scaling parameters α associated with different pre-operations.

[0202] Implicit filter shape derivation

[0203] In one example, instead of explicitly signaling the selected filter shape index, the used directional filter shape can be derived at the decoder to save bit overhead. For example, at the decoder, multiple directional gradient filters can be applied to each reconstructed luminance sample of the L-shaped template of the i-th adjacent row and column of the current block. Then, the filter values ​​(gradients) can be accumulated for each direction in the multiple directional gradient filters respectively. In the example, the accumulated value is the accumulated value of the absolute value of the corresponding filter value. After accumulation, the direction of the directional gradient filter with the largest accumulated value can be determined as the derived (luminance) gradient direction. For example, a gradient histogram (HoG) can be constructed to determine the maximum value. The derived direction can be further applied as a direction for predicting the chrominance sample in the current block.

[0204] The following example involves reusing the decoder-side intra mode derivation (DIMD) method for luma intra prediction included in ECM-4.0: Step 1: Apply 2 directional gradient filters (3x3 hor / ver Sobel) to each reconstructed luminance sample of the L-shaped template of the second adjacent row and column of the current block; Step 2: Accumulate the filter value (gradient) of each directional gradient filter through SAD (sum of absolute difference); Step 3: constructing a Histogram of Gradients (HoG) based on the accumulated filtered values; and Step 4: The maximum value in HoG is determined as the derived (brightness) gradient direction, based on which the GLM filter can be determined.

[0205] In one example, if the shape candidates are [-1, 0, 1; -1, 0, 1] (horizontal) and [1, 2, 1; -1, -2, -1] (vertical), when the maximum value is associated with the horizontal shape, the shape [-1, 0, 1; -1, 0, 1] is used for GLM-based chrominance prediction.

[0206] The gradient filter used to derive the gradient direction may be the same or different in shape from the GLM filter. For example, the two filters may be horizontal [-1, 0, 1; -1, 0, 1], or the two filters may have different shapes, and the GLM filter may be determined based on the gradient filter.

[0207] The proposed GLM can be combined with the MMLM or ELM discussed above. When combined with classification, each group can share or have its own filter shape, where the syntax indicates the shape for each group. For example, as an exemplary classifier, the horizontal gradient grad_hor can be classified into the first group, which corresponds to the first linear model, and the vertical gradient grad_ver can be classified into the second group, which corresponds to the second linear model. In one example, the horizontal brightness pattern can be generated only once.

[0208] Further possible classifiers are also provided as follows. Using the classifier, adjacent and internal luma-chroma sample pairs of the current video block can be classified into a plurality of groups based on one or more thresholds. Note that, as discussed above, each adjacent / internal chroma sample and its corresponding luma sample can be referred to as a luma-chroma sample pair. One or more thresholds are associated with the intensity of the adjacent / internal luma samples. In this case, each of the plurality of groups corresponds to a corresponding one of the plurality of linear models.

[0209] When combined with the MMLM classifier, the following operations may be performed: adjacent reconstructed luminance-chrominance sample pairs of the current video block are classified into two groups based on a threshold Threshold; different linear models are derived for different groups, wherein the derivation process may be GLM simplified, i.e., the number of taps is reduced by using the above-mentioned pre-operation; similarly, the luminance-chrominance sample pairs (internal luminance-chrominance sample pairs, wherein each internal luminance-chrominance sample pair in the internal luminance-chrominance sample pair includes: an internal chrominance sample value predicted by the derived linear model) within the CU are classified into two groups based on the threshold Threshold; different linear models are applied to the reconstructed luminance samples in different groups; and, the chrominance samples in the CU are predicted based on different classified linear models. Wherein, Threshold may be the average value of adjacent reconstructed luminance samples. Note that the number of categories (2) can be extended to multiple categories by increasing the number of Thresholds (e.g., equal division based on the min / max values ​​of adjacent reconstructed (downsampled) luma samples, fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level).

[0210] In one example, instead of the MMLM luma DC intensity, the filter value of the FLM / GLM applied to the adjacent luma samples is used for classification. For example, if a 1-tap (1, -1) GLM is applied, the average AC value (physical meaning) is used. The processing may be: classifying adjacent reconstructed luma-chroma sample pairs into K groups based on one or more filter shapes, one or more filter values, and K-1 ThresholdTi; deriving different MLR models for different groups, wherein the derivation process may be GLM simplified, i.e., using the above-mentioned pre-operation to reduce the number of taps; similarly classifying the luma-chroma sample pairs (internal luma-chroma sample pairs, wherein each internal luma-chroma sample pair in the internal luma-chroma sample pairs includes an internal chroma sample value predicted using a derived linear model) within the CU into K groups based on one or more filter shapes, one or more filter values, and K-1ThresholdTi; applying different linear models to the reconstructed luma samples in different groups; predicting the chroma samples in the CU based on different classified linear models. Among them, Threshold can be predefined (e.g., 0, or can be a table) or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, Threshold can be the average AC value (filter value) of adjacent reconstructed (can be downsampled) luminance samples (2 groups), or divided equally based on minimum / maximum AC (K groups).

[0211] It is also proposed to combine GLM with ELM classifiers. FIG. 15A to FIG. 15D As shown, a filter shape (e.g., 1 tap) can be selected to calculate the edge strength. The direction is determined as the direction of calculating the sample difference between the current sample and N adjacent samples (e.g., all 6 brightness samples). For example, Fig.15AThe filter in the upper middle of (shape [1, 0, 1; 1, 0, 1]) indicates the horizontal direction, because the sample difference between the samples in the horizontal direction can be calculated, and the filter below it (shape [1, 2, 1; -1, -2, -1]) indicates the vertical direction, because the sample difference between the samples in the vertical direction can be calculated. The positive coefficients and negative coefficients in each filter can calculate the sample difference. Then, the processing may include: calculating an edge strength through the filtered value (e.g., equivalent value); quantizing the edge strength into M segments through the M-1 threshold Ti; using K categories to classify the current sample (e.g., where K == M); deriving different MLR models for different groups, where the derivation process can be GLM simplified, that is, using the above-mentioned pre-operation to reduce the number of taps; classifying the luminance-chrominance sample pairs inside the CU into K groups; applying different MLR models to the reconstructed luminance samples in different groups; and, predicting the chrominance samples in the CU based on different classified MLR models. Note that the filter shape used for classification can be the same or different from the filter shape used for MLR prediction. The number of both thresholds M-1 and threshold values ​​Ti can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. In addition, other classifiers / combined classifiers discussed in ELM can also be used for GLM.

[0212] If the number of classification samples in a group is less than (e.g., predefined as 4), then the default values ​​mentioned when discussing the matrix derivation for the MLR model can be applied to the group parameters (α i , β). If the corresponding adjacent reconstructed samples are not available for the selected LM mode, a default value may be applied, for example, when the MMLM_L mode is selected but the left samples are invalid.

[0213] The following introduces several methods involving simplification of GLM to further improve the encoding and decoding efficiency.

[0214] Matrix / parameter derivation in FLM requires floating-point operations (e.g., closed-form division), which is expensive for decoder hardware, thus requiring a fixed-point design. For the 1-tap GLM case, which can be viewed as a modified luminance reconstruction sample generation of CCLM (e.g., horizontal gradient direction, from CCLM [1, 2, 1; 1, 2, 1] / 8 to GLM [-1, 0, 1; -1, 0, 1]), the original CCLM process can be reused for GLM, including fixed-point operations, MDLM downsampling, division table, applied size restrictions, minimum-maximum approximation, and scaling adjustments. The 1-tap GLM can have its own configuration or share the same design as CCLM for all entries. For example, a simplified minimum-maximum method is used to derive parameters (instead of LMS) and is combined with scaling adjustments after the GLM model is derived. In this case, the center point for rotating the slope (luminance value y r ) becomes the average value of the reference luma sample "gradient". Another example, when GLM is on for the CU, CCLM slope adjustment is inferred to be off, and no slope adjustment related syntax needs to be signaled.

[0215] This section uses typical reference points (1 row above and 1 column on the left) as an example. Fig.14 In , the extended reconstructed region can also use simplifications with the same spirit and can have a syntax (such as MDLM, MRL) to indicate a specific region.

[0216] Please note that the following aspects can be combined and applied jointly, for example, combining reference sample downsampling with a division table for division processing.

[0217] When applying classification (MMLM / ELM), each group can apply the same or different simplification operations. For example, pad each group's samples to the target number of samples before applying the right shift, and then apply the same derivation process and the same division table.

[0218] Fixed-point implementation

[0219] The CCLM design can be reused for the 1-tap case. Division by n can be implemented by right shifting, and division by A2 can be implemented by LUT. α , n tableand the intermediate parameters used to derive the linear model (Equations (19)-(20)) can be the same as CCLM or have different values ​​to have higher accuracy. The integerization parameters can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and can be conditioned on the sequence bit depth. For example, n table =bit depth + 4.

[0220] MDLM downsampling

[0221] When GLM is combined with MDLM, the total existing samples used for parameter derivation may not be a power of 2 value and need to be padded to a power of 2 to replace the division with a right shift operation. For example, for an 8x4 chroma CU, MDLM requires W+H=12 samples, of which only 8 samples are available (reconstructed) for MDLM_T, and then the downsampled 4 samples (0, 2, 4, 6) can be equally filled. The code for implementing this operation is as follows: int targetSampNum=1<<(floorLog2(existSampNum-1)+1); if(targetSampNum!=existSampNum) / / if existSampNum not a value ofpower of 2 { xPadMdlmTemplateSample; } int step=(int)(existSampNum / sampNumToBeAdd); for(int i=0; i <sampNumToBeAdd;i++) { pTempSrc[i] = pSrc[i*step]; pTempCur[i]=pCur[i*step]; }

[0222] Other filling methods may also be applied, such as repeated / mirror filling relative to the last labeled sample point (rightmost / bottommost).

[0223] The filling method of GLM can be the same as or different from the filling method of CCLM.

[0224] Note that in the ECM version, the 8x4 chroma CU MDLM_T / MDLM_L requires 2T / 2L=16 / 8 samples respectively, in which case the same padding method can be applied to meet the target power-of-2 sample number.

[0225] Division LUT

[0226] The division LUT proposed for CCLM / LIC (local illumination compensation) in known standard developments (such as AVC / HEVC / AV1 / VVC / AVS) can be used for GLM division. For example, for the case of bit depth = 10 (Table 4), the LUT in JCTVC-I0166 is reused. The division LUT may be different from CCLM. For example, CCLM uses the min-max with DivTable as in Formula 5, but GLM uses a 32-entry LMS division LUT as in Table 5.

[0227] When GLM is combined with MMLM, meanL values ​​may not always be positive (e.g., using filter / gradient values ​​to classify groups), so sgn(mean L) needs to be extracted and abs(mean L) used to look up the division LUT. Note that the division LUTs used for MMLM classification and parameter derivation can be different. For example, use a lower precision LUT (as in min-max) for mean classification and a higher precision LUT (as in LMS) for parameter derivation.

[0228] Size Limits and Latency Constraints

[0229] Similar to CCLM design, some size restrictions may apply to ELM / FLM / GLM. For example, the same constraints on luma-chroma delay in dual trees may apply.

[0230] Size limit can be based on CU area / width / height / depth. Thresholds can be predefined or signaled at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, for chroma CU region, the predefined threshold can be 128.

[0231] In one example, at least one pre-operation is performed in response to determining that a video block satisfies an enablement threshold, wherein the enablement threshold is associated with an area, width, height, or segmentation depth of the video block. Specifically, the enablement threshold may define a minimum or maximum area, width, height, or segmentation depth of the video block. As will be appreciated by those skilled in the art, a video block may include a current chroma block and its co-located luma block. It is also proposed that the above-mentioned enablement threshold be applied jointly to the current chroma block and its co-located luma block. For example, at least one pre-operation is performed in response to determining that both the current chroma block and its co-located luma block satisfy an enablement threshold.

[0232] Line buffer reduction

[0233] Similar to the CCLM design, if the co-located luma region of the current chroma CU contains the first line within one CTU, the top template sample generation can be limited to 1 line to reduce the CTU line buffer storage. Note that when the top reference line is at the CTU boundary, only one luma line (common line buffer in intra prediction) is used for down-sampled luma samples.

[0234] For example, in Fig.13 In , if the co-located luma region of the current chroma CU contains the first row within a CTU, the top template can be limited to using only 1 row (but not 2 rows) for parameter derivation (other CUs can still use 2 rows). This saves luma sample line buffer storage when processing CTUs line by line at the decoder hardware. Several methods can be used to achieve line buffer reduction. Note that the example of limited "1" rows can be extended to N rows with similar operations. Similarly, 2 taps or multiple taps can also apply such operations. When multiple taps are applied, chroma samples may also need to apply operations.

[0235] For example, Fig.15A The 1-tap filter [1, 0, 1; 1, 0, 1] shown is used as an example. This filter can be reduced to [0, 0, 0; 1, 0, -1], that is, only the next row of coefficients is used. Alternatively, the limited previous row of luma samples can be filled from the next row of luma samples (repeated, mirrored, 0, meanL, meanC, etc.).

[0236] Take N=4 as an example, that is, the video block is located at the top boundary of the current CTU, and the first 4 rows of adjacent luma sample values ​​and corresponding chroma sample values ​​are used to derive the linear model. Note that the corresponding chroma sample values ​​may refer to the corresponding top 4 rows of adjacent chroma sample values ​​(e.g., for YUV 4:4:4 format). Alternatively, the corresponding chroma sample values ​​may refer to the corresponding top 2 rows of adjacent chroma sample values ​​(e.g., for YUV 4:2:0 format). In this case, the top 4 rows of adjacent luma sample values ​​and corresponding chroma sample values ​​may be divided into two regions—a first region including valid sample values ​​(e.g., one nearest row of luma sample values ​​and corresponding chroma sample values) and a second region including invalid sample values ​​(e.g., the other three rows of luma sample values ​​and corresponding chroma sample values). Then, the coefficients of the filter corresponding to the sample positions that do not belong to the first region may be set to zero, so that the sample differences are calculated using only the sample values ​​from the first region. For example, as discussed above, in this case, the filter [1, 0, -1; 1, 0, -1] can be reduced to [0, 0, 0; 1, 0, -1]. Alternatively, the nearest sample values ​​in the first region can be padded to the second region so that the padded sample values ​​can be used to calculate the sample differences.

[0237] Fusion of Chroma Intra Prediction Modes

[0238] In one example, since GLM can be considered as a special CCLM mode, the fusion design can be reused or have its own way. Multiple (two or more) weights can be applied to generate the final prediction value. For example, pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift where pred0 is the prediction based on the non-LM model and pred1 is the prediction based on the GLM, or pred0 is the prediction based on one of the CCLMs (including all MDLMs / MMLMs), and pred1 is the prediction based on the GLM, or pred0 is the predicted value based on GLM, and pred1 is the predicted value based on GLM.

[0239] Different I / P / B slices can have different weight designs, w0 and w1, depending on whether the neighboring blocks are coded with CCLM / GLM / other codec modes or block size / width / height.

[0240] For example, the design of the weight can be determined by the intra prediction mode of the adjacent chroma block and the shift is set to be equal to 2. Specifically, when the upper adjacent block and the left adjacent block are both encoded in LM mode, {w0, w1} = {1, 3}; when the upper adjacent block and the left adjacent block are both encoded in non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}. For non-I slices, both w0 and w1 can be set to be equal to 2.

[0241] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied.

[0242] As described above, the GLM has a good gain-complexity tradeoff because it can reuse the existing CCLM module without introducing additional derivations. According to one or more aspects of the present disclosure, such a 1-tap design can be further extended or generalized.

[0243] In one aspect of the present disclosure, for a chroma sample to be predicted, a single corresponding luma sample L may be generated by combining a co-located luma sample with an adjacent luma sample. For example, the combination may be a combination of different linear filters, such as a high-pass gradient filter (GLM) and a low-pass smoothing filter (e.g., a combination of [1, 2, 1; 1, 2, 1] / 8 FIR downsampling filters, which may be commonly used in CCLM; and / or a combination of a linear filter and a nonlinear filter (e.g., with a power of n, such as L n , n can be positive, negative, or +- fractional (e.g., +1 / 2 square root, or +3 cubed, which can be rounded and rescaled to the bit depth dynamic range)).

[0244] In one aspect of the present disclosure, the combination can be applied repeatedly. For example, a combination of GLM and [1, 2, 1; 1, 2, 1] / 8 FIR can be applied to the reconstructed luma samples, and then a power of 1 / 2 of nonlinearity can be applied. For example, the nonlinear filter can be implemented as a LUT (lookup table), for example, for bit depth = 10, power of n, n = 1 / 2, LUT [i] = (int) (sqrt (i) + 0.5) << 5, i = 0 ~ 1023, where 5 is scaled to the bit depth = 10 dynamic range. When the linear filter cannot effectively handle the luma-chroma relationship, the nonlinear filter can provide an option. Whether to use nonlinear items can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0245] In one or more aspects of the present disclosure, GLM may refer to a generalized linear model (which may be used to linearly or nonlinearly generate a single luminance sample, and the generated single luminance sample may be fed into a CCLM linear model to derive parameters of the CCLM linear model), and the linear / nonlinear generation may be referred to as a general pattern. Different gradients or general patterns may be combined to form another pattern. For example, a gradient pattern may be combined with a CCLM downsampled value; a gradient pattern may be combined with a nonlinear L 2 values; a gradient mode can be combined with another gradient mode, and the two gradient modes to be combined can have different directions or the same direction. For example, [1, 1, 1; -1, -1, -1] and [1, 2, 1; -1, -2, -1] (they all have vertical directions) can be combined, and [1, 1, 1; -1, -1, -1] and [1, 0, -1; 1, 0, -1] (they all have vertical and horizontal directions) can be combined, such as FIG. 15A to FIG. 15D The combination may include additive, subtractive, or linear weighting.

[0246] GLM applied to the downsampled domain

[0247] As described above, pre-operation may be repeatedly applied, and GLM may be applied to the pre-linear weighted / pre-operated samples. For example, as CCLM, a template filter may be applied to the luma samples to remove outliers using a low-pass smoothing FIR filter [1, 2, 1; 1, 2, 1] / 8 (i.e., CCLM downsampling smoothing filter) and generate downsampled luma samples (one downsampled luma sample corresponds to one chroma sample). Thereafter, a 1-tap GLM may be applied to the smoothed downsampled luma samples to derive the MLR model.

[0248] Some gradient filter modes (e.g., 3x3 Sobel or Prewitt operator) can be applied to the downsampled luma samples. The following table shows some gradient filter modes.

[0249] The gradient filter pattern can be combined with other gradient / generic filter patterns in the downsampled luma domain. In one example, the combined filter pattern can be applied to the downsampled luma samples. For example, the combined filter pattern can be derived by performing addition or subtraction operations on the corresponding coefficients of the gradient filter pattern and a DC / lowpass based filter pattern (e.g., filter patterns [0, 0, 0; 0, 1, 0; 0, 0] or [1, 2, 1; 2, 4, 1; 1, 2, 1]). In another example, the combined filter pattern can be derived by performing addition or subtraction operations on the coefficients of the gradient filter pattern and nonlinear values ​​(e.g., L 2) to derive the combined filter pattern. In another example, the combined filter pattern is derived by performing an addition or subtraction operation on the corresponding coefficients of the gradient filter pattern and another gradient filter pattern with different or the same direction. In another example, the combined filter pattern is derived by performing a linear weighted operation on the coefficients of the gradient filter pattern.

[0250] A GLM applied in the downsampled domain can fit into the CCCM framework, but may sacrifice high frequency accuracy because low-pass smoothing is applied before applying the GLM.

[0251] GLM used as input to CCCM

[0252] As shown above, CCCM applies luma downsampling before convolution like CCLM. When chroma subsampling is used, the reconstructed luma samples are downsampled to match the lower resolution chroma grid. Since a 1-tap GLM can also be viewed as changing the CCLM downsampling filter coefficients (e.g., from [1, 2, 1; 1, 2, 1] / 8 to [1, 2, 1, -1, -2, -1], i.e., from low pass to high pass), the GLM can be used as input to the CCCM. Specifically, the gradient filter of the GLM replaces the luma downsampling filter ([1, 2, 1; 1, 2, 1] / 8) with gradient-based coefficients (e.g., [1, 2, 1, -1, -2, -1, -1). In this case, the CCCM operation becomes a "linear / nonlinear combination of gradients", as shown in the following formula:

[0253] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B Where C, N, S, E, W, P are the gradients of the current or neighboring samples (compared to the original downsampled values ​​for CCCM). The related GLM methods described in this disclosure can be applied in the same way before entering the CCCM convolution, e.g., classification, separate Cb / Cr control, syntax, mode combination, PU size restrictions, etc.

[0254] Gradient-based coefficient replacement can be applied to specific CCCM taps. Moreover, not only high-pass but also low-pass / band-pass / all-pass coefficient replacement can be used. Replacement can be combined with the FLM / CCCM shape switching discussed above (resulting in a different number of taps). For example, FIG. 15A to FIG. 15DThe gradient mode in can be used for replacement. In one example, the operation for applying GLM as an input to CCCM includes: predefining one or more coefficient candidates for CCCM / FLM downsampling; determining the CCCM / FLM filter shape and the number of filter taps for the CU; applying different CCLM downsampling coefficients to different filter taps, where the coefficients can be a high-pass filter (GLM), or a low-pass / band-pass / all-pass filter; generating downsampled luma samples for CCCM input samples (using the applied coefficients); and, feeding the generated downsampled luma samples into the CCCM process.

[0255] Some examples for changing the CCLM downsampling filter coefficients are shown below:

[0256] Example 1:

[0257] The candidate filters are [1, 2, 1; 1, 2, 1] / 8 and [1, 0, -1; 1, 0, -1]; predChromaVal = c0C+c1N+c2S+c3E+c4W+c5P+c6B, using the typical CCCM cross shape, 7 taps; N, S, W, E use the filter [1, 2, 1; 1, 2, 1] / 8, keeping the original CCLM downsampling filter; and, C, P use the filter [1, 0, -1; 1, 0, -1], i.e., a horizontal gradient filter, and then P physically represents gradient^2.

[0258] Example 2:

[0259] The candidate filters are [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [2, 1, -1; 1, -1, -2] and [-1, 1, 2; -2, -1, 1]; predChromaVal = c0C0+c1C1+c2C2+c3C3+c4C4+c5P+c6B; C0 uses the filter [1, 2, 1; 1, 2, 1] / 8, keeping the original CCLM downsampling filter; C1 uses the filter [1, 0, - 1; 1, 0, -1]; C2 uses filter [1, 2, 1; -1, -2, -1]; C3 uses filter [2, 1, -1; 1, -1, -2]; C4 uses filter [-1, 1, 2; -2, -1, 1]; C5 uses filter [1, 2, 1; 1, 2, 1] / 8, keeping the original CCLM downsampling filter; C0 to C5 and P have the same downsampled brightness position (in the typical CCCM cross shape, == C); and C1 to C4 are generated by different Sobel-based gradient filters (such as FIG. 15A to FIG. 15D shown).

[0260] Example 3:

[0261] The candidate filters are [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [0, 1, 1; 0, 1, 1] and [1, 1, 0; 1, 1, 0]; predChromaVal = c0C0+c1C1+c2C2+c3C3+c4C4+c5P+c6B; C0 uses the filter [1, 2, 1; 1, 2, 1] / 8, keeping the original CCLM downsampling filter; C1 uses the filter [1, 0, -1 ; 1, 0, -1]; C2 uses filter [1, 2, 1; -1, -2, -1]; C3 uses filter [0, 1, 1; 0, 1, 1]; C4 uses filter [1, 1, 0; 1, 1, 0]; C5 uses filter [1, 2, 1; 1, 2, 1] / 8, keeping the original CCLM downsampling filter; C0 to C5 and P have the same downsampled brightness position (in the typical CCCM cross shape, == C); C1 to C2 are generated by Sobel-based gradient filters in different directions (such as Figures 15A to 15D as shown); and C3 to C4 are generated by a low-pass filter.

[0262] Which CCCM / FLM taps to apply coefficient replacement may be predefined (as in the above examples) or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0263] For each CCCM / FLM tap, coefficient candidates for CCCM / FLM downsampling may be predefined or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0264] Example 4:

[0265] The candidate filters are [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [2, 1, -1; 1, -1, -2] and [-1, 1, 2; -2, -1, 1]; predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B, using the typical CCCM crossover shape, 7 taps; C uses a switching downsampling filter between the 5 candidate filters; N, S, W, E use the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter; and, P uses a switching downsampling filter between the 5 candidate filters.

[0266] Example 5:

[0267] The candidate filters are: [1, 2, 1; 1, 2, 1] / 8, [1, 0, -1; 1, 0, -1], [1, 2, 1; -1, -2, -1], [0, 1, 1; 0, 1, 1] and [1, 1, 0; 1, 1, 0]; predChromaVal = c0C + c1W + c2E + c3P + c4B, that is, horizontal minus shape, 5 taps; C uses the downsampling filter switched between 5 candidate filters; W and E use the downsampling filter switched between 3 candidate filters: [1, 2, 1; 1, 2, 1] / 8, [0, 1, 1; 0, 1, 1] and [1, 1, 0; 1, 1, 0]; and, P uses the filter [1, 2, 1; 1, 2, 1] / 8, maintaining the original CCLM downsampling filter.

[0268] In one or more aspects of the present disclosure, one or more syntaxes may be introduced to indicate information about GLM. An example of GLM syntax is shown in Table 10 below. Table 10 FLC: Fixed Length Code TU: Truncated Unary Code EGk: exponential-golomb code with order k, where k may be fixed or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. SVLC: Signed EG0 UVLC: Unsigned EG0

[0269] Note that the binarization of each syntax element can be changed.

[0270] In one aspect of the present disclosure, GLM on / off control for Cb / Cr components can be performed jointly or individually. For example, at the CU level, 1 flag can be used to indicate whether GLM is active for this CU. If it is active, 1 flag can be used to indicate whether both Cb / Cr are active. If not both are active, 1 flag indicates that Cb or Cr is active. When Cb and / or Cr are active, the filter index / gradient (general) mode can be signaled separately. All flags can have their own context model or be bypassed.

[0271] In another aspect of the present disclosure, whether to signal the GLM on / off flag may depend on the luminance / chrominance coding mode and / or the CU size. For example, in the ECM5 chrominance intra mode syntax, when MMLM or MMLM_L or MMLM_T is applied, it can be inferred that GLM is off; when the CU area < A, where A can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level; if combined with CCCM, then when CCCM is on, it can be inferred that GLM is off.

[0272] Note that when GLM is combined with MMLM, different models can share the same or have their own gradients / general modes.

[0273] When GLM is combined with CCCM / FLM, if the current CU is enabled for CCCM / FLM, the CU-level GLM enable flag can be inferred to be off. hasGlmFlag &=!pu.cccmFlag;

[0274] CCCM without downsampling

[0275] CCCM needs to process the downsampled luminance reference value before calculating the model parameters and applying the CCCM model, which burdens the decoder processing cycle. In this part, a CCCM without downsampling processing is proposed, including using different selections of non-downsampled luminance reference values and / or non-downsampled luminance references. One or more filter shapes can be used for the purposes described below.

[0276] Note that the methods / examples in this part can be combined / reused with the above methods, including but not limited to the methods related to classification, filter shapes, matrix derivation (with special processing), application areas, and syntax. In addition, the methods / examples listed in this part can also be applied to the above methods / examples (more taps) to have better performance under certain complexity trade-offs.

[0277] In the present disclosure, the reference sample / training template / reconstructed adjacent region generally refers to the luminance samples used to derive the MLR model parameters, which are then applied to the internal luminance samples in a CU to predict the chrominance samples in the CU.

[0278] Filter shape

[0279] One or more shapes / numbers of filter taps can be used for CCCM prediction, as Fig.16 、 Fig.17 和 FIG. 18A to FIG. 18B One or more sets of filter taps can be used for FLM prediction, for example in FIG. 19A to FIG. 19G The selected luma reference value is non-downsampled. One or more predefined shapes / numbers of filter taps may be used for CCCM prediction based on previously decoded information on TB / CB / slice / picture / sequence level.

[0280] Although multi-tap filters can be well suited to training data (i.e., top / left adjacent reconstructed luminance / chrominance samples), in some cases, the training data does not capture the complete characteristics of the test data and may result in overfitting and may not predict the test data well (i.e., chrominance block samples to be predicted). Moreover, different filter shapes can be well adapted to different video block contents, resulting in more accurate predictions. To address this problem, the filter shape / number of filter taps can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of filter shape candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) can have different filter switching controls. For example, as shown in the following table, a set of filter shape candidates (idx=0-5) is predefined, and the filter shape (1, 2) represents a 2-tap brightness filter, and the filter shape (1, 2, 4) represents a 3-tap brightness filter, such as Fig.11 As shown in FIG. 1 , the filter shape selection for the U / V component can be switched in the PH or CU / CTU level. Note that N-tap may represent N-tap with or without offset β as described above.

[0281] Different chroma types / color formats may have different predefined filter shapes / taps. For example, Fig.12 As shown in , predefined filter shapes are used for 420 type 0: (1, 2, 4, 5), 420 type 2: (0, 1, 2, 4, 7), 422: (1, 4), 444: (0, 1, 2, 3, 4, 5).

[0282] The unavailable luminance / chrominance samples used to derive the MLR model can be filled in from the available reconstructed samples. Fig.12, then for the CU located at the left picture boundary, the left column including (0, 3) is unavailable (outside the picture boundary), so (0, 3) is repeated padding from (1, 4) to apply the 6-tap filter. Note that the padding process is applied to both the training data (top / left adjacent reconstructed luma / chroma samples) and the test data (luma / chroma samples in the CU).

[0283] According to one or more embodiments of the present disclosure, unavailable luma / chroma samples for deriving the MLR model may be skipped and not used. Then, the unavailable luma / chroma samples do not require padding processing.

[0284] CCLM / MMLM with LDL decomposition

[0285] CCCM needs to process LDL decomposition to calculate the model parameters of CCCM model, avoiding the use of square root operations and only requiring integer operations. In this section, CCLM / MMLM with LDL decomposition is proposed. As mentioned above, LDL decomposition can also be used in ELM / FLM / GLM.

[0286] Please note that the methods / examples in this section can be combined / reused with the above methods, including but not limited to methods related to classification, filter shape, matrix derivation (with special treatments), application area, and syntax. In addition, the methods / examples listed in this section can also be applied together with the above methods / examples to have better performance with certain complexity tradeoffs.

[0287] In this disclosure, reference samples / training templates / reconstructed neighboring regions generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0288] CCLM / MMLM with extended range

[0289] One or more reference samples can be used for CCLM / MMLM prediction, i.e. Fig. 10B As shown, the reference region can be the same as the reference region in CCCM. Based on the previous decoding information on TB / CB / slice / picture / sequence level, different reference regions can be used for CCLM / MMLM prediction.

[0290] Although training data with multiple reference areas can be well suited for the calculation of model parameters, in some cases, the training data does not capture the complete characteristics of the test data, which may lead to overfitting and may not predict the test data (i.e., the chroma block samples to be predicted) well. In addition, different reference areas can be well adapted to different video block contents, resulting in more accurate predictions. To address this problem, the reference shape / number of reference areas can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of reference area candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) can have different reference area switching controls. For example, a set of reference area candidates (idx=0~4) is predefined, as shown in the following table. The reference area selection for the U / V component can be switched at the PH or CU / CTU level. Different colorimetric types / color formats may have different predefined reference areas.

[0291] Unavailable luma / chroma samples used to derive the MLR model can be filled from available reconstructed samples, and the filling process is applied to both the training data (top / left adjacent reconstructed luma / chroma samples) and the test data (luma / chroma samples in the CU).

[0292] According to one or more embodiments of the present disclosure, unavailable luma / chroma samples for deriving the MLR model may be skipped and not used. Then, the unavailable luma / chroma samples do not require padding processing.

[0293] FLM / GLM / ELM / CCCM with minimum sample point restriction

[0294] FLM needs to process the downsampled luma reference and calculate the model parameters, which places a burden on the decoder processing cycle, especially for small blocks. In this section, FLM with minimum sample restriction is proposed, for example, FLM is only used for samples greater than a predetermined number, such as 64, 128. One or more different restrictions can be used for this purpose, for example, FLM is only used for samples greater than a predefined number (such as 256) in a single model, and FLM is only used for samples greater than a predefined number (such as 128) in a multi-model.

[0295] According to one or more embodiments of the present disclosure, the number of predefined minimum sample points for a single model may be greater than or equal to the number of predefined minimum sample points for a multi-model. For example, FLM / GLM / ELM / CCCM is only used for sample points greater than or equal to a predefined number (such as 128) in a single model, and FLM / GLM / ELM / CCCM is only used for sample points greater than or equal to a predefined number (such as 256) in a multi-model.

[0296] According to one or more embodiments of the present invention, the number of predefined minimum sample points for FLM / GLM / ELM may be greater than or equal to the number of predefined minimum sample points for CCCM. For example, CCCM is only used in a single model for sample points greater than or equal to a predefined number (such as 0), and CCCM is only used in a multi-model for sample points greater than or equal to a predefined number (such as 128). FLM is only used in a single model for sample points greater than or equal to a predefined number (such as 128), and FLM is only used in a multi-model for sample points greater than or equal to a predefined number (such as 256).

[0297] Please note that the methods / examples in this section can be combined / reused with the above methods, including but not limited to methods related to classification, filter shape, matrix derivation (with special treatments), application area, and syntax. In addition, the methods / examples listed in this section can also be applied to the above methods / examples (with more taps) to have better performance with certain complexity tradeoffs.

[0298] Fig.21 A workflow of a method 2100 for decoding video data according to one or more aspects of the present disclosure is shown.

[0299] At step 2110 , method 2100 includes obtaining a video block from a bitstream.

[0300] At step 2120 , method 2100 includes obtaining an internal luma sample value of the video block, an external luma sample value of an external region of the video block, and an external chroma sample value of an external region.

[0301] At step 2130, method 2100 includes determining a set of weighting coefficients corresponding to a filter shape based on external luma sample values ​​and external chroma sample values, wherein the filter shape and the set of weighting coefficients are configured to predict chroma sample values ​​based on multiple corresponding luma sample values, and wherein the set of weighting coefficients includes a weighting coefficient for a non-linear value derived by scaling a product of two of the multiple corresponding luma sample values ​​or a product of the squares of one of the multiple corresponding luma sample values ​​to a luma sample value range of the video block.

[0302] At step 2140, method 2100 includes predicting intra chroma sample values ​​based on intra luma sample values ​​using a filter shape and a set of weighting coefficients.

[0303] At step 2150 , method 2100 includes obtaining a decoded video block using the predicted intra chroma sample values.

[0304] In an example, the set of weighting coefficients further includes: one or more weighting coefficients respectively for one or more luma sample values ​​among the plurality of corresponding luma sample values.

[0305] In one example, the set of weighting coefficients further includes: a weighting coefficient for a deviation value.

[0306] In an example, the plurality of corresponding luma sample values ​​include: a plurality of pre-calculated luma sample values ​​obtained by weighted average calculation.

[0307] In one example, the plurality of corresponding luma sample values ​​include: a first pre-operated luma sample value co-located with the chroma sample value to be predicted, a second pre-operated luma sample value to the left of the first pre-operated luma sample value, a third pre-operated luma sample value to the right of the first pre-operated luma sample value, a fourth pre-operated luma sample value above the first pre-operated luma sample value, and a fifth pre-operated luma sample value below the first pre-operated luma sample value.

[0308] In one example, the product of two of the plurality of corresponding luma sample values ​​comprises: the product of any two of the first pre-operated luma sample value, the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value; and the square of one of the plurality of corresponding luma sample values ​​comprises: the square of one of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

[0309] In one example, the plurality of corresponding luma sample values ​​further include: an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the square of one of the plurality of corresponding luma sample values ​​includes: the square of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

[0310] In one example, the plurality of corresponding luma sample values ​​further include: an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes: a product of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value and the first pre-operated luma sample value.

[0311] In an example, the plurality of corresponding luma sample values ​​include: a first luma sample value co-located with the chroma sample value to be predicted, a second luma sample value to the left of the first luma sample value, a third luma sample value to the right of the first luma sample value, a fourth luma sample value below the first luma sample value, a fifth luma sample value below the second luma sample value, and a sixth luma sample value below the third luma sample value.

[0312] In one example, the product of two of the multiple corresponding luma sample values ​​includes: the product of any two of the first luma sample value, the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value; and the square of one of the multiple corresponding luma sample values ​​includes: the square of one of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, or the sixth luma sample value.

[0313] In an example, the multiple corresponding luma sample values ​​further include: an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value, and the square of one of the multiple corresponding luma sample values ​​includes: the square of the average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value.

[0314] In an example, the plurality of corresponding luma sample values ​​further include: an average of a second luma sample value, a third luma sample value, a fourth luma sample value, a fifth luma sample value, and a sixth luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes: a product of an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value and the first luma sample value.

[0315] Fig. 22 A workflow of a method 2200 for encoding video data according to one or more aspects of the present disclosure is shown.

[0316] At step 2210 , method 2200 includes obtaining a video block.

[0317] At step 2220 , the method 2200 includes obtaining an internal luma sample value of the video block, an external luma sample value of an external region of the video block, and an external chroma sample value of an external region.

[0318] At step 2230, method 2200 includes determining a set of weighting coefficients corresponding to a filter shape based on external luma sample values ​​and external chroma sample values, wherein the filter shape and the set of weighting coefficients are configured to predict chroma sample values ​​based on multiple corresponding luma sample values, and wherein the set of weighting coefficients includes a weighting coefficient for a non-linear value derived by scaling a product of two of the multiple corresponding luma sample values ​​or a product of the squares of one of the multiple corresponding luma sample values ​​to a luma sample value range of the video block.

[0319] At step 2240, method 2200 includes predicting intra chroma sample values ​​based on intra luma sample values ​​using a filter shape and a set of weighting coefficients.

[0320] At step 2250, the method 2200 includes generating a bitstream including the encoded video block by using the predicted intra chroma sample values.

[0321] In one example, a set of weighting coefficients further includes: one or more weighting coefficients respectively for one or more corresponding luma sample values ​​among the plurality of corresponding luma sample values.

[0322] In one example, the set of weighting coefficients further includes: a weighting coefficient for a deviation value.

[0323] In one example, the plurality of corresponding luma sample values ​​include: a plurality of pre-calculated luma sample values ​​obtained by weighted average calculation.

[0324] In one example, the plurality of corresponding luma sample values ​​include: a first pre-operated luma sample value co-located with the chroma sample value to be predicted, a second pre-operated luma sample value to the left of the first pre-operated luma sample value, a third pre-operated luma sample value to the right of the first pre-operated luma sample value, a fourth pre-operated luma sample value above the first pre-operated luma sample value, and a fifth pre-operated luma sample value below the first pre-operated luma sample value.

[0325] In one example, the product of two of the plurality of corresponding luma sample values ​​comprises: the product of any two of the first pre-operated luma sample value, the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value; and the square of one of the plurality of corresponding luma sample values ​​comprises: the square of one of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

[0326] In one example, the plurality of corresponding luma sample values ​​further include: an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the square of one of the plurality of corresponding luma sample values ​​includes: the square of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

[0327] In one example, the plurality of corresponding luma sample values ​​further include: an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes: a product of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value and the first pre-operated luma sample value.

[0328] In an example, the multiple corresponding luma sample values ​​include: a first luma sample value co-located with the chroma sample value to be predicted, a second luma sample value to the left of the first luma sample value, a third luma sample value to the right of the first luma sample value, a fourth luma sample value below the first luma sample value, a fifth luma sample value below the second luma sample value, and a sixth luma sample value below the third luma sample value.

[0329] In one example, the product of two of the multiple corresponding luma sample values ​​includes: the product of any two of the first luma sample value, the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value; and the square of one of the multiple corresponding luma sample values ​​includes: the square of one of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, or the sixth luma sample value.

[0330] In an example, the multiple corresponding luma sample values ​​further include: an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value, and the square of one of the multiple corresponding luma sample values ​​includes: the square of the average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value.

[0331] In an example, the plurality of corresponding luma sample values ​​further include: an average of a second luma sample value, a third luma sample value, a fourth luma sample value, a fifth luma sample value, and a sixth luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes: a product of an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value and the first luma sample value.

[0332] Fig.23 An exemplary computing system 2300 according to one or more aspects of the present disclosure is shown. The computing system 2300 may include at least one processor 2310. The computing system 2300 may also include at least one storage device 2320. The storage device 2320 may store computer executable instructions that, when executed, cause the processor 2310 to perform the steps of the above method. The processor 2310 may be a general-purpose processor, or may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. The storage device 2320 may store input data, output data, data generated by the processor 2310, and / or instructions executed by the processor 2310.

[0333] It should be understood that the storage device 2320 may store computer-executable instructions that, when executed, cause the processor 2310 to perform any operations according to the embodiments of the present disclosure.

[0334] Embodiments of the present disclosure may be embodied in a computer-readable medium such as a non-transitory computer-readable medium. The non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform any operation according to an embodiment of the present disclosure. For example, the instructions, when executed, may cause one or more processors to receive a bitstream and perform a decoding operation as described above. As another example, the instructions, when executed, may cause one or more processors to perform an encoding operation and send a bitstream including encoded video information associated with predicted chroma samples as described above.

[0335] It should be understood that all operations in the above methods are merely exemplary, and the present disclosure is not limited to any operations in the methods or the sequential order of these operations, and should cover all other equivalents under the same or similar concepts.

[0336] It should also be understood that all modules in the above method can be implemented in various ways. These modules can be implemented as hardware, software or a combination thereof. In addition, any one of these modules can be further divided into submodules or combined together in function.

[0337] The foregoing description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents of the elements of the various aspects described in this disclosure that are known or will be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims.

Claims

1. A method for decoding video data, comprising: Get video chunks from the bitstream; Obtaining an internal luminance sample value of the video block, an external luminance sample value of an external area of ​​the video block, and an external chrominance sample value of the external area; determining a set of weighting coefficients corresponding to a filter shape based on the outer luma sample values ​​and the outer chroma sample values, wherein the filter shape and the set of weighting coefficients are configured to predict chroma sample values ​​based on a plurality of corresponding luma sample values, and wherein the set of weighting coefficients includes a weighting coefficient for a non-linear value derived by scaling a product of two of the plurality of corresponding luma sample values ​​or a square of one of the plurality of corresponding luma sample values ​​to a range of luma sample values ​​of the video block; predicting the intra chroma sample values ​​based on the intra luma sample values ​​using the filter shape and the set of weighting coefficients; and The predicted intra chroma sample values ​​are used to obtain a decoded video block.

2. The method according to claim 1, wherein: The set of weighting coefficients also includes: one or more weighting coefficients respectively for one or more luma sample values ​​of the plurality of corresponding luma sample values.

3. The method according to claim 1, wherein: The set of weighting coefficients also includes a weighting coefficient for a deviation value.

4. The method according to claim 1, wherein: The plurality of corresponding brightness sample values ​​include: a plurality of pre-calculated brightness sample values ​​obtained by weighted average calculation.

5. The method according to claim 4, wherein: The plurality of corresponding luma sample values ​​include: a first pre-operated luma sample value co-located with the chroma sample value to be predicted, a second pre-operated luma sample value on the left side of the first pre-operated luma sample value, a third pre-operated luma sample value on the right side of the first pre-operated luma sample value, a fourth pre-operated luma sample value above the first pre-operated luma sample value, and a fifth pre-operated luma sample value below the first pre-operated luma sample value.

6. The method according to claim 5, wherein: The product of two of the plurality of corresponding luma sample values ​​comprises: a product of any two of the first pre-operated luma sample value, the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value; and the square of one of the plurality of corresponding luma sample values ​​comprises: a square of one of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

7. The method according to claim 5, wherein: The plurality of corresponding luma sample values ​​further include an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the square of one of the plurality of corresponding luma sample values ​​includes a square of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

8. The method according to claim 5, wherein: The plurality of corresponding luma sample values ​​further include an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes a product of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value and the first pre-operated luma sample value.

9. The method according to claim 1, wherein: The multiple corresponding luma sample values ​​include: a first luma sample value co-located with the chroma sample value to be predicted, a second luma sample value to the left of the first luma sample value, a third luma sample value to the right of the first luma sample value, a fourth luma sample value below the first luma sample value, a fifth luma sample value below the second luma sample value, and a sixth luma sample value below the third luma sample value.

10. The method according to claim 9, wherein: The product of two of the plurality of corresponding luma sample values ​​comprises: a product of any two of the first luma sample value, the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value; and the square of one of the plurality of corresponding luma sample values ​​comprises: a square of one of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, or the sixth luma sample value.

11. The method according to claim 9, wherein: The plurality of corresponding luma sample values ​​further include an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value, and the square of one of the plurality of corresponding luma sample values ​​includes the square of the average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value.

12. The method according to claim 9, wherein: The plurality of corresponding luma sample values ​​further include an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes a product of the average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value and the first luma sample value.

13. A method for encoding video data, comprising: Get the video block; Obtaining an internal luminance sample value of the video block, an external luminance sample value of an external area of ​​the video block, and an external chrominance sample value of the external area; determining a set of weighting coefficients corresponding to a filter shape based on the outer luma sample values ​​and the outer chroma sample values, wherein the filter shape and the set of weighting coefficients are configured to predict chroma sample values ​​based on a plurality of corresponding luma sample values, and wherein the set of weighting coefficients includes a weighting coefficient for a non-linear value derived by scaling a product of two of the plurality of corresponding luma sample values ​​or a square of one of the plurality of corresponding luma sample values ​​to a range of luma sample values ​​of the video block; predicting the intra chroma sample values ​​based on the intra luma sample values ​​using a filter shape and the set of weighting coefficients; and A bitstream including an encoded video block is generated by using the predicted intra chroma sample values.

14. The method according to claim 13, wherein: The set of weighting coefficients also includes: one or more weighting coefficients respectively for one or more luma sample values ​​of the plurality of corresponding luma sample values.

15. The method according to claim 13, wherein: The set of weighting coefficients also includes a weighting coefficient for a deviation value.

16. The method according to claim 13, wherein: The plurality of corresponding luminance sample values ​​include a plurality of pre-calculated luminance sample values ​​obtained by weighted average calculation.

17. The method according to claim 16, wherein: The plurality of corresponding luma sample values ​​include: a first pre-operated luma sample value co-located with the chroma sample value to be predicted, a second pre-operated luma sample value on the left side of the first pre-operated luma sample value, a third pre-operated luma sample value on the right side of the first pre-operated luma sample value, a fourth pre-operated luma sample value above the first pre-operated luma sample value, and a fifth pre-operated luma sample value below the first pre-operated luma sample value.

18. The method according to claim 17, wherein: The product of two of the plurality of corresponding luma sample values ​​comprises: a product of any two of the first pre-operated luma sample value, the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value; and the square of one of the plurality of corresponding luma sample values ​​comprises: a square of one of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

19. The method according to claim 17, wherein: The plurality of corresponding luma sample values ​​further include an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the square of one of the plurality of corresponding luma sample values ​​includes a square of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value.

20. The method according to claim 17, wherein: The plurality of corresponding luma sample values ​​further include an average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes a product of the average of the second pre-operated luma sample value, the third pre-operated luma sample value, the fourth pre-operated luma sample value, and the fifth pre-operated luma sample value and the first pre-operated luma sample value.

21. The method according to claim 13, wherein: The multiple corresponding luma sample values ​​include: a first luma sample value co-located with the chroma sample value to be predicted, a second luma sample value to the left of the first luma sample value, a third luma sample value to the right of the first luma sample value, a fourth luma sample value below the first luma sample value, a fifth luma sample value below the second luma sample value, and a sixth luma sample value below the third luma sample value.

22. The method according to claim 21, wherein: The product of two of the plurality of corresponding luma sample values ​​comprises: a product of any two of the first luma sample value, the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value; and the square of one of the plurality of corresponding luma sample values ​​comprises: a square of one of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, or the sixth luma sample value.

23. The method according to claim 21, wherein: The plurality of corresponding luma sample values ​​further include an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value, and the square of one of the plurality of corresponding luma sample values ​​includes the square of the average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value.

24. The method according to claim 21, wherein: The plurality of corresponding luma sample values ​​further include an average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value, and the product of two of the plurality of corresponding luma sample values ​​includes a product of the average of the second luma sample value, the third luma sample value, the fourth luma sample value, the fifth luma sample value, and the sixth luma sample value and the first luma sample value.

25. A computer system comprising: one or more processors; as well as One or more storage devices storing computer executable instructions which, when executed, cause the one or more processors to perform the operations of the method according to any one of claims 1 to 24.

26. A computer program product storing computer executable instructions which, when executed, cause one or more processors to perform the operations of the method according to any one of claims 1 to 24.

27. A computer-readable medium storing computer-executable instructions which, when executed, cause one or more processors to perform the operations of the method of any one of claims 1 to 12 based on a bitstream.

28. A computer readable medium storing computer executable instructions which, when executed, cause one or more processors to perform the operations of the method of any one of claims 13 to 24.

29. A computer-readable medium storing a bit stream, wherein: The bit stream is to be decoded by performing the operations of the method according to any one of claims 1-12.

30. A computer-readable medium storing a bit stream, wherein: The bit stream is obtained by performing the operations of the method according to any one of claims 13-24.

Images