Method and apparatus for cross-component prediction for video coding

Abstract

The present disclosure provides a method for decoding video data, comprising: obtaining a video block from a bitstream; obtaining an internal brightness sample point value of the video block, an external brightness sample point value of an external region of the video block and an external chrominance sample point value of the external region; selecting one filter shape candidate for predicting an internal chroma sample value of the video block from a plurality of filter shape candidates based on the external luma sample value and the external chroma sample value, where the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting a chroma sample value based on the corresponding luma sample value; predicting an internal chroma sample value based on the internal luma sample value using the selected filter shape candidate; and obtaining a decoded video block using the predicted internal chroma sample values.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 393,177, filed on July 28, 2022, which is hereby incorporated by reference in its entirety. Technical Field

[0003] Generally speaking, aspects of the present disclosure relate to image / video coding and compression, and more particularly, aspects of the present disclosure relate to methods and apparatus for cross-component prediction techniques. Background Art

[0004] Various video coding and decoding technologies can be used to compress video data. Video coding and decoding is performed according to one or more video coding and decoding standards. For example, video coding and decoding standards include Versatile Video Codec (VVC), High Efficiency Video Codec (H.265 / HEVC), Advanced Video Codec (H.264 / AVC), Moving Picture Experts Group (MPEG) codec, etc. Video coding and decoding generally use prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize redundancy in video images or sequences. An important goal of video coding and decoding technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality. Summary of the Invention

[0005] The following is a simplified summary of one or more aspects of the present disclosure to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all contemplated aspects and is neither intended to identify key or critical elements of all aspects nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be presented later.

[0006] According to one aspect of the present disclosure, a method for decoding video data is provided, including: obtaining a video block from a bitstream; obtaining an internal luma sample value of the video block, an external luma sample value of an outer region of the video block, and an external chroma sample value of the outer region; selecting a filter shape candidate from a plurality of filter shape candidates for predicting an internal chroma sample value of the video block based on the outer luma sample value and the outer chroma sample value, wherein the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting chroma sample values ​​based on corresponding luma sample values; predicting internal chroma sample values ​​based on the internal luma sample value using the selected filter shape candidate; and obtaining a decoded video block using the predicted internal chroma sample values.

[0007] According to another aspect of the present disclosure, a method for encoding video data is provided, including: obtaining a video block; obtaining an internal luma sample value of the video block, an external luma sample value of an external region of the video block, and an external chroma sample value of the external region; selecting a filter shape candidate from a plurality of filter shape candidates for predicting an internal chroma sample value of the video block based on the external luma sample value and the external chroma sample value, wherein the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting chroma sample values ​​based on corresponding luma sample values; predicting the internal chroma sample values ​​based on the internal luma sample value using the selected filter shape candidate; and generating a bitstream including the encoded video block by using the predicted internal chroma sample values.

[0008] According to another aspect of the present disclosure, a computer system is provided, comprising: one or more processors; and one or more storage devices storing computer-executable instructions, which, when executed, cause the one or more processors to perform the operations of the method of the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, which stores computer-executable instructions that, when executed, cause the one or more processors to perform the operations of the method of the present disclosure.

[0010] According to another aspect of the present disclosure, a computer-readable medium is provided, which stores computer-executable instructions. When the instructions are executed, one or more processors receive a bitstream and perform the operations of the method of the present disclosure based on the bitstream.

[0011] According to another aspect of the present disclosure, a computer-readable medium is provided that stores computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method of the present disclosure and send a bitstream including encoded video information associated with the predicted chroma samples.

[0012] According to another aspect of the present disclosure, a computer-readable medium storing a bitstream is provided, wherein the bitstream is to be decoded by performing the operations of the method of the present disclosure.

[0013] According to another aspect of the present disclosure, a computer-readable medium storing a bitstream is provided, wherein the bitstream is obtained by executing the operation of the method of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The disclosed aspects will be described hereinafter in conjunction with the accompanying drawings, which are provided for the purpose of illustrating rather than limiting the disclosed aspects.

[0015] Figure 1A block diagram of a general block-based hybrid video coding system is shown.

[0016] Figures 2A to 2E Five partition types are shown, including quaternary partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning.

[0017] Figure 3 An overall block diagram of a block-based video decoder is shown.

[0018] Figure 4 An example of the positions of the left and upper samples and the samples of the current block involved in CCLM mode is shown.

[0019] Figures 5A to 5C An example of deriving CCLM parameters is shown.

[0020] Figure 6 An example of classifying neighboring samples into two groups based on the value Threshold is shown.

[0021] Figure 7 An example of classifying neighboring samples into two groups based on inflection points is shown.

[0022] Figure 8A and Figure 8B The effect of the scaling adjustment parameter "u" is shown.

[0023] Figure 8C Collocated reconstructed luminance samples are shown.

[0024] Figure 8D Neighboring reconstructed samples are shown.

[0025] Figures 8E to 8H The steps of intra mode derivation at the decoder side are shown.

[0026] Figure 9 An example of four reference rows of neighboring prediction blocks is shown.

[0027] Figure 10A An exemplary mode of a convolutional cross-component model (CCCM) is shown.

[0028] Figure 10B An exemplary reference region is shown, consisting of 6 rows of chroma samples above and to the left of the PU.

[0029] Figure 10C and Figure 10D Schematic diagram showing the correlation between chrominance samples and one or more luma samples.

[0030] Figure 11An example of using 6 taps in a multiple linear regression (MLR) model according to one or more aspects of the present disclosure is shown.

[0031] Figure 12 Exemplary different filter shapes and / or numbers of taps according to one or more aspects of the present disclosure are shown.

[0032] Figure 13 An example is shown where the FLM may use only the top or left luma and / or chroma samples (extended) for parameter derivation.

[0033] Figure 14 An example is shown where FLM can use different rows for parameter derivation.

[0034] 15A to 15D Some examples of 1-tap / 2-tap pre-operation are shown.

[0035] Figure 16 Examples of different shapes / numbers of filter taps are shown.

[0036] Figure 17 Examples of different shapes / numbers of filter taps are shown.

[0037] Figure 18A and Figure 18B Examples of different shapes / numbers of filter taps are shown.

[0038] Figures 19A to 19G Examples of different sets of filter taps are shown.

[0039] Figure 20 A workflow of a method for decoding video data according to one or more aspects of the present disclosure is shown.

[0040] Figure 21 A workflow of a method for encoding video data according to one or more aspects of the present disclosure is shown.

[0041] Figure 22 An exemplary computing system in accordance with one or more aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0042] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices having digital video capabilities.

[0043] It should be noted that the terms "first," "second," and the like used in the specification, claims, and drawings of the present disclosure are used to distinguish between objects, and are not used to describe any specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be implemented in an order other than that shown in the drawings or described in the present disclosure.

[0044] The first version of the VVC standard was finalized in July 2020, offering approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video codec standard, HEVC. While the VVC standard offers significant codec improvements over its predecessor, there is evidence that superior codec efficiency can be achieved by employing additional codec tools. Recently, the Joint Video Exploration Team (JVET), a collaboration between ITU-TVCEG and ISO / IEC MPEG, began exploring advanced technologies that could significantly improve codec efficiency over VVC. In April 2021, a software codebase, the Enhanced Compression Model (ECM), was established for future VVC codec exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC and further extends and / or improves several existing modules (e.g., intra / inter prediction, transforms, and loop filters). Going forward, any new codec tools beyond the VVC standard will need to be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC).

[0045] Like all previous video codec standards, ECM is built on a block-based hybrid video codec framework. Figure 1A block diagram of a general block-based hybrid video coding system is shown. The input video signal is processed block by block, which is called a coding unit (CU). In ECM-1.0, a CU can be up to 128×128 pixels. However, like VVC, a coding tree unit (CTU) is divided into CUs to adapt to different local characteristics based on quadtree / binary / ternary trees. In the multi-type tree structure, a CTU is first divided by a quadtree structure. Then, each quadtree leaf node can be further divided by a binary tree structure and a ternary tree structure. As shown in FIG. Figure 2A 、 2B As shown in , 2C, 2D and 2E, there are five types of segmentation: quadruple segmentation, vertical binary segmentation, horizontal binary segmentation, vertical extended quadruple segmentation and horizontal extended quadruple segmentation.

[0046] exist Figure 1In

[15] , spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples of coded neighboring blocks in the same video picture / slice (which are called reference samples) to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion compensated prediction") uses reconstructed pixels from coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference. In addition, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from. After spatial and / or temporal prediction, the mode decision block in the encoder selects the best prediction mode, for example based on a rate-distortion optimization method. The prediction block is then subtracted from the current video block; and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are inverse quantized and inverse transformed to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. In addition, loop filtering (such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF)) can be applied to the reconstructed CU before the reconstructed CU is placed in the reference picture store and used to encode and decode future video blocks. In order to form the output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit to be further compressed and packed to form a bitstream. It should be noted that the terms "block" or "video block" used herein can be a portion of a frame or picture, specifically a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block, such as a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0047] Figure 3The overall block diagram of a block-based video decoder is shown. The video bitstream is first entropy decoded at the entropy decoding unit. The coding mode and prediction information are sent to the spatial prediction unit (if intra-frame coding) or the temporal prediction unit (if inter-frame coding) to form a prediction block. The residual transform coefficients are sent to the inverse quantization unit and the inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then added together. The reconstructed block can also be loop filtered before being stored in the reference picture store. The reconstructed video in the reference picture store is then sent to drive the display device and used to predict future video blocks.

[0048] The primary focus of this disclosure is to further enhance the codec efficiency of codec tools used for cross-component prediction in ECM, the Cross-Component Linear Model (CCLM). Below, we briefly review some relevant codec tools in ECM. Thereafter, we discuss some deficiencies in existing CCLM designs. Finally, we provide solutions for improving existing CCLM prediction designs.

[0049] Cross-component linear model prediction

[0050] To reduce cross-component redundancy, the Cross-Component Linear Model (CCLM) prediction mode is used in VVC, for which chroma samples are predicted based on the reconstructed luma samples of the same CU using the following linear model:

[0051] pred C (i,j)=α·rec L ′(i,j)+ β (1)

[0052] Among them, pred C (i, j) represents the predicted chroma sample in CU, and rec L '(i,j) represents the reconstructed brightness sample point rec L (i, j) is the downsampled reconstructed luma sample of the same CU obtained by downsampling. The above α and β are linear model parameters derived from up to four adjacent chroma samples and their corresponding downsampled luma samples, which can be called adjacent luma-chroma sample pairs. Assuming that the current chroma block has a size of W×H, W' and H' are obtained as follows:

[0053] - When LM mode is applied, W'=W, H'=H;

[0054] - When LM-A mode is applied, W'=W+H;

[0055] - When LM-L mode is applied, H'=H+W;

[0056] Among them, in LM mode, the upper and left samples of CU are used together to calculate the linear model coefficients; in LM_A mode, only the upper samples of CU are used to calculate the linear model coefficients; and in LM_L mode, only the left samples of CU are used to calculate the linear model coefficients.

[0057] If the positions of the upper neighboring samples of the chroma block are represented as S[0,-1]...S[W'-1,-1], and the positions of the left neighboring samples of the chroma block are represented as S[-1,0]...S[-1,H'-1], then the positions of the four neighboring chroma samples are selected as follows:

[0058] -When LM mode is applied and the upper and left adjacent samples are available, select S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4] as the positions of the four adjacent chroma samples;

[0059] - When LM-A mode is applied or only upper neighboring samples are available, select S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1],

[0060] S[7*W' / 8,-1] is the position of the four adjacent chroma samples;

[0061] - When LM-L mode is applied or only left neighbor samples are available, select S[-1,H' / 8], S[-1,3*H' / 8], S[-1,5*H' / 8],

[0062] S[-1,7*H' / 8] is used as the position of the four adjacent chroma samples.

[0063] The four adjacent brightness samples corresponding to the selected position are obtained by the downsampling operation, and the obtained four adjacent brightness samples are compared four times to find the two larger values ​​x 0 A and x 1 A and two smaller values ​​x 0 B and x 1 B The chrominance sample values ​​corresponding to the two larger and two smaller values ​​are represented as y 0 A 、y 1 A 、y 0 B and y 1 B Then, export X as follows a 、X b 、Y aand Y b :

[0064] X a =(x 0 A +x 1 A +1)>>1;

[0065] X b =(x 0 B +x 1 B +1)>>1;

[0066] Y a =(y 0 A +y 1 A +1)>>1;

[0067] Y b =(y 0 B +y 1 B +1)>>1 (2)

[0069] Finally, the linear model parameters α and β are obtained according to the following equations.

[0070]

[0071] β=Y b -α·X b (4)

[0072] Figure 4 Examples of the positions of the left and top samples and samples of the current block involved in the CCLM mode are shown, including the positions of the left and top samples of the N×N chroma block in the CU and the left and top samples of the 2N×2N luminance block in the CU.

[0073] The division operation used to calculate the parameter α is implemented using a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed using exponential notation. For example, diff is approximated using a 4-bit significant part and an exponent. Therefore, the table for 1 / diff is reduced to 16 elements with 16 significant values, as shown below:

[0074] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0075] This will have the benefit of reducing both the complexity of the calculations and the memory size required to store the required tables.

[0076] In addition to the upper template and the left template being used together to calculate the linear model coefficients, they can also be used alternatively in two other LM modes (referred to as LM_A and LM_L modes).

[0077] In LM_T mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0078] In LM_LT mode, the left template and the upper template are used to calculate the linear model coefficients.

[0079] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both the horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "Type 0" and "Type 2" content, respectively.

[0080]

[0081] Note that when the upper reference line is located at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to make the downsampled luma samples.

[0082] This parameter calculation is performed as part of the decoding process, not just as an encoder retrieval operation. As a result, no syntax is used to convey the α and β values ​​to the decoder.

[0083] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A and LM_L). The chroma mode signaling and derivation process are shown in Table 1. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I slice, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0084] Table 1 - Chroma prediction modes derived from luma mode when CCLM is enabled

[0085]

[0086] Regardless of the value of sps_cclm_enabled_flag, a single binarization table is used, as shown in Table 2.

[0087] Table 2 - Unified binarization table for chroma prediction mode

[0088]

[0089]

[0090] In Table 2, the first binary bit indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next binary bit indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, the next 1 binary bit indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first binary bit of the corresponding intra_chroma_pred_mode binarization table can be discarded before entropy coding. Or, in other words, the first binary bit is inferred to be 0 and therefore not coded. This single binarization table is used for both cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two binary bits in Table 2 perform context coding using their own context model, and the remaining binary bits perform bypass coding.

[0091] Additionally, to reduce luma-chroma latency in dual trees, when a 64x64 luma CT node is split using NotSplit (and ISP is not used for 64x64 CUs) or QT, chroma CUs in 32x32 / 32x16 chroma CT nodes are allowed to use CCLM as follows:

[0092] If a 32x32 chroma node is not split or is split by QT, all chroma CUs in the 32x32 node can use CCLM.

[0093] If a 32x32 chroma node utilizes horizontal BT splitting, and its 32x16 child nodes are not split or use vertical BT splitting, then all chroma CUs in the 32x16 chroma node can use CCLM.

[0094] Under all other luma and chroma coding tree partitioning conditions, CCLM is not allowed for chroma CUs.

[0095] During the ECM development, the simplified derivation of α and β (min-max approximation) was removed. Instead, a linear least squares solution was performed between the causal reconstructed data of the downsampled luma samples and the causal chroma samples to derive the model parameters α and β.

[0096]

[0097]

[0098] Wherein, RecC(i) and Rec'L(i) indicate the reconstructed chroma samples and the down-sampled reconstructed luma samples around the target block, and I indicates the total number of samples of neighboring data.

[0099] The LM_A and LM_L models are also called multidirectional linear models (MDLM). Figure 5A An example of the operation of the MDLM is shown when the block content cannot be predicted from the L-shaped reconstructed region. Figure 5B MDLM_L is shown using only the left reconstructed samples to derive CCLM parameters. Figure 5C MDLM_T is shown using only the top reconstructed samples to derive CCLM parameters.

[0100] The integerization of the least mean square (LMS) discussed above (see equations (8)-(9)) has been proposed as an improvement to the CCLM. The initial integerization design of the LMS CCLM was first proposed in JCTVC-C206. The method was then improved through a series of simplifications, culminating in the ECM LMS version. The improvements include: JCTVC-F0233 / I0178, which reduces the α accuracy n α from 13 to 7; JCTVC-I0151, which reduced the maximum multiplier bit width; and JCTVC-H0490 / I0166, which reduced the divide LUT entries from 64 to 32.

[0101] As discussed in equation (1), the integer design models the correlation of the luma and chroma signals using a linear relationship. The chroma values ​​are predicted from the reconstructed luma values ​​of the co-located blocks.

[0102] In YUV420 sampling, the luma component and the chroma component have different sampling ratios. The sampling ratio of the chroma component is half of the sampling ratio of the luma component, and there is a phase difference of 0.5 pixels in the vertical direction. The reconstructed luma needs to be downsampled in the vertical direction and subsampled in the horizontal direction to match the size of the chroma signal. For example, downsampling can be achieved by:

[0103] Rec L ′(i,j)=(rec L (2i,2j)+rec L (2i,2j+1))>>1 (10)

[0104] In equation (8), in order to maintain high data accuracy, floating-point operations are required when calculating the linear model parameter α. Moreover, when α is represented by a floating-point value, floating-point multiplication is involved in equation (1). In this section, an integer implementation of the algorithm is designed. Specifically, the fractional part of the parameter α is calculated using n α The parameter α is quantized by the enlarged and rounded integer value α' and a'=a×(1<<n α ) is expressed. Then the linear model of equation (1) is changed to:

[0105] pred C [x,y]=(α'·Rec L '[x,y]>>n α )+β' (11)

[0106] Where β' is the rounded value of the floating point β, and α' can be calculated as follows.

[0107]

[0108] It is proposed to replace the division operation of equation (12) by table lookup and multiplication. First, A2 is reduced to reduce the table size. A1 is also reduced to avoid product overflow. Then, in A2, only the The most significant bit of the value is defined, and all other bits are set to zero. The approximate value A2' can be calculated as:

[0109]

[0110] where [...] represents rounding operation, and It can be calculated as:

[0111]

[0112] Here, bdepth(A2) represents the bit depth of the value A2.

[0113] Perform the same operation on A1 as follows:

[0114]

[0115] Considering the quantized representations of A1 and A2, equation (12) can be rewritten as follows.

[0116]

[0117]

[0118] in, Indicates that The length of the lookup table to avoid division.

[0119] In the simulation, the constant parameters are set as:

[0120] ·n α Equal to 13, this value is a compromise between data accuracy and computational cost.

[0121] · Equal to 6, the lookup table size is 64. When bdepth(A2)<6 (for example, A2<32), the table size can be further reduced to 32 by enlarging A2.

[0122] ·n table Equal to 15, produces a 16-bit data representation of the table element.

[0123] · Set to 15 to avoid product overflow and maintain 16-bit multiplication.

[0124] Finally, α' is clipped to [-2 -15 ,2 15 -1] to preserve the 16-bit multiplication in equation (11). By this clipping, when n α When it is equal to 13, the actual a value is limited to [-4, 4), which helps prevent error amplification.

[0125] By calculating the parameter α', the parameter β' is calculated as follows:

[0126]

[0127] The division in the above equation can be simply replaced by a shift since the value I is a power of 2.

[0128] Similar to the discussion above about equation (1), in HM6.0, an intra prediction mode called LM is applied to predict the chroma PU based on a linear model using the reconstruction of the co-located luma PU. The parameters of the linear model consist of the slope (a>>k) and the y-intercept (b), which are derived from the neighboring luma and chroma pixels using the least mean square solution. The values ​​of the predicted samples predSamples[x,y], where x,y=0...nS-1, are derived as follows:

[0129] predSamples[x,y]=Clip1 C (((p Y '[x,y]*a)>>k)+b), where x,y=0..nS-1 (17)

[0130] Among them, P Y'[x,y] is the reconstructed pixel from the corresponding luminance component. When the coordinates x and y are equal to or greater than 0, P Y ' is the reconstructed pixel from the same luma PU. When x or y is less than 0, P Y ' is the reconstructed neighboring pixel of the same-position luma PU.

[0131] Some intermediate variables L, C, LL, LC, k2 and k3 in the derivation process are derived as follows:

[0132]

[0133]

[0134] k2=Log2((2*nS)>>k3) (18-5)

[0135] k3=Max(0,BitDepth C +Log2(nS)-14) (18-6)

[0136] Therefore, the variables a, b, and k can be derived as:

[0137] a1=(LC< <k2)–L*C (19-1)

[0138] a2=(LL< <k2)–L*L (19-2)

[0139] k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))-14)+2 (19-3)

[0140] a1s=a1>>Max(0,Log2(abs(a1))-14) (19-4)

[0141] a2s=abs(a2>>Max(0,Log2(abs(a2))-5)) (19-5)

[0142] a3=a2s<1?0:Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1) (19-6)

[0143] a=a3>>Max(0,Log2(abs(a3))-6) (19-7)

[0144] k=13–Max(0,Log2(abs(a))-6) (19-8)

[0145] b=(L–((a*C)>>k1)+(1<<(k2-1)))>>k2, (19-9)

[0146] Here, lmDiv is specified in a 63-item lookup table (Table 3), which is generated online as follows:

[0147] lmDiv(a2s)=((1<<15)+a2s / 2) / a2s. (20)

[0148] Table 3 - lmDiv Specifications

[0149] a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 lmDiv 32768 16384 10923 8192 6554 5461 4681 4096 3641 3277 2979 2731 2521 a2s 14 15 16 17 18 19 20 21 22 23 24 25 26 lmDiv 2341 2185 2048 1928 1820 1725 1638 1560 1489 1425 1365 1311 1260 a2s 27 28 29 30 31 32 33 34 35 36 37 38 39 lmDiv 1214 1170 1130 1092 1057 1024 993 964 936 910 886 862 840 a2s 40 41 42 43 44 45 46 47 48 49 50 51 52 lmDiv 819 799 780 762 745 728 712 697 683 669 655 643 630 a2s 53 54 55 56 57 58 59 60 61 62 63 64 lmDiv 618 607 596 585 575 565 555 546 537 529 520 512

[0150] In equation (19-6), a1s is a 16-bit signed integer and lmDiv is a 16-bit unsigned integer. Therefore, a 16-bit multiplier and 16-bit storage are required. It is proposed to reduce the bit depth of the multiplier to the internal bit depth and reduce the size of the lookup table, as detailed below.

[0151] The bit depth of a1s is reduced to the internal bit depth by changing equation (19-4) to the following equation:

[0152] a1s=a1>>Max(0,Log2(abs(a1))–(BitDepth C –2)). (21)

[0153] The value of lmDiv with internal bit depth is implemented using the following equation (22) and stored in a lookup table:

[0154] lmDiv(a2s)=((1<<(BitDepth C -1))+a2s / 2) / a2s. (22)

[0155] Table 4 shows an example of an internal bit depth of 10.

[0156] Table 4 - Specification of lmDiv with internal bit depth equal to 10

[0157]

[0158]

[0159] Equation (19-3) and Equation (19-8) are also modified as follows:

[0160] k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))–(BitDepth C –2)), and (23-1)

[0161] k = BitDepth C –1–Max(0,Log2(abs(a))-6) (23-2)

[0162] It is also proposed to reduce the number of entries from 63 to 32 and the bits per entry from 16 to 10, as shown in Table 5. By doing so, a memory saving of nearly 70% can be achieved. The corresponding changes to Equation (19-6), Equation (20), and Equation (19-8) are as follows:

[0163] a3=a2s<32?0:Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1) (24-1)

[0164] lmDiv(a2s)=((1<<(BitDepth C +4))+a2s / 2) / a2s (24-2)

[0165] k = BitDepth C +4–Max(0,Log2(abs(a))-6). (24-3)

[0166] Table 5 - Specification of lmDiv with internal bit depth equal to 10

[0167] a2s 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 lmDiv 512 496 482 468 455 443 431 420 410 400 390 381 372 364 356 349 a2s 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 341 334 328 321 315 309 303 298 293 287 282 278 273 269 264 260

[0168] Multi-model linear model prediction

[0169] In ECM-1.0, the Multi-Model LM (MMLM) prediction mode is proposed. For this mode, chroma samples are predicted based on the reconstructed luma samples of the same CU using two linear models, as shown below:

[0170]

[0171] Among them, pred C (i, j) represents the predicted chroma sample in CU, and rec L '(i,j) represents the downsampled reconstructed luma sample of the same CU. The Threshold is calculated as the average of the adjacent reconstructed luma samples. Figure 6 An example of classifying neighboring samples into two groups based on the value Threshold is shown. For each group, the minimum brightness sample A(X) from two samples (which is the minimum brightness sample A(X) within the group) is obtained. A 、Y A ) and the maximum brightness sample B(XB 、Y B )) is used to derive the parameter α from the linear relationship between the brightness and chromaticity values i and β i (where i is equal to 1 and 2 respectively). Here, X A 、Y A is the x-coordinate (i.e., luminance value) and y-coordinate (i.e., chrominance value) value of sample point A, and X B 、Y B are the x-coordinate and y-coordinate values ​​of sample point B. The linear model parameters α and β are obtained according to the following equations.

[0172]

[0173] β=y A -αx A (26)

[0175] Such a method is also known as the min-max method. The division in the above equation can be avoided and replaced by multiplication and shift.

[0176] For coding blocks with a square shape, the above two equations are applied directly. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0177] In addition to the scenario where the upper template and the left template are used together to calculate the linear model coefficients, these two templates can also be used alternatively in the other two MMLM modes (referred to as MMLM_A mode and MMLM_L mode).

[0178] In MMLM_A mode, only the pixel samples in the upper template are used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to a size of (W + W). In MMLM_L mode, only the pixel samples in the left template are used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to a size of (H + H).

[0179] Note that when the upper reference line is located at a CTU boundary, only one luma line (stored in the line buffer for intra prediction) is used to make the downsampled luma samples.

[0180] For chroma intra mode coding, a total of 11 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and six cross-component linear model modes (CCLM, LM_A, LM_L, MMLM, MMLM_A and MMLM_L). The chroma mode signaling and derivation process are shown in Table 6. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I slice, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0181] Table 6 - Chroma prediction modes derived from luma mode when MMLM_ is enabled

[0182]

[0183] The MMLM mode and the LM mode can also be used together in an adaptive manner. For MMLM, the two linear models are as follows:

[0184]

[0185] Among them, pred C (i, j) represents the predicted chroma sample in CU, and rec L ′(i, j) represents the downsampled reconstructed luma sample of the same CU. Threshold can be simply determined based on the average value of luma and chroma together with their minimum and maximum values. Figure 7 An example of classifying adjacent samples into two groups based on the inflection point T indicated by the arrow is shown. The linear model parameters α1 and β1 are obtained based on the minimum brightness sample A(X A 、Y A ) and Threshold(X T 、Y T The linear relationship between the luminance and chrominance values ​​of the linear model is derived. The linear model parameters α2 and β2 are derived from the two samples (which are the maximum luminance samples B(X B 、Y B ) and Threshold(X T 、Y T )) is derived from the linear relationship between the brightness and chromaticity values. Here, X A 、Y A is the x-coordinate (i.e., luminance value) and y-coordinate (i.e., chrominance value) value of sample point A, and X B 、Y B is the x-coordinate and y-coordinate value of sample point B. The linear model parameter α of each group is obtained according to the following equation iand β i , where i is equal to 1 and 2 respectively.

[0186]

[0187] β1=Y A -α1X A

[0188]

[0189] β2=Y T -α2X T (28)

[0190] For coding blocks with a square shape, the above equation is applied directly. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0191] In addition to the scenario where the upper template and the left template are used together to determine the linear model coefficients, these two templates can also be used alternatively in the other two MMLM modes (referred to as MMLM_A mode and MMLM_L mode, respectively).

[0192] In MMLM_A mode, only the pixel samples in the upper template are used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to a size of (W + W). In MMLM_L mode, only the pixel samples in the left template are used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to a size of (H + H).

[0193] Note that when the upper reference line is located at a CTU boundary, only one luma line (stored in the line buffer for intra prediction) is used to make the downsampled luma samples.

[0194] For chroma intra mode coding, there is a conditional check to select LM mode (CCLM, LM_A and LM_L) or multi-model LM mode (MMLM, MMLM_A and MMLM_L). The conditional check is as follows:

[0195]

[0196] Among them, BlkSizeThres LM Indicates the minimum block size for LM mode, and BlkSizeThres MM Indicates the minimum block size of the MMLM mode. The symbol d represents a predetermined threshold. In one example, d can take the value 0. In another example, d can take the value 8.

[0197] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes. The chroma mode signaling and derivation process are shown in Table 1. It is worth noting that for a given CU, if it is encoded in the linear model mode, it is determined whether it is a conventional single model LM mode or an MMLM mode based on the above conditional check. Unlike the case shown in Table 6, there is no separate MMLM mode that needs to be signaled. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I slice, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0198] During the development of the ECM, scaling (slope) adjustments to the CCLM were proposed as a further improvement, for example as described in JVET-Y0055 / Z0049.

[0199] As discussed above, CCLM uses a 2-parameter model to map luma values ​​to chroma values. The scaling parameter "a" and the bias parameter "b" define the mapping as follows:

[0200] chromaVal=a*lumaVal+b (30)

[0201] We propose to signal the adjustment “u” to the scaling parameter to update the model to the following form:

[0202] chromaVal=a'*lumaVal+b' (31)

[0203] Where, a'=a+u, and b'=bu*y r .

[0204] With this choice, the mapping function will be centered around the value y with the brightness r The point is tilted or rotated. It is proposed to use the average value of the reference brightness samples used in model creation as y r , in order to provide meaningful modifications to the model. Figures 8A to 8B shows the effect of the scaling adjustment parameter "u", where Figure 8A shows the model created without the scaling tuning parameter “u”, and Figure 8B A model created with a scaling adjustment parameter "u" is shown.

[0205] In one example, the scaling adjustment parameter is provided as an integer between -4 and 4 (inclusive) and signaled in the bitstream. The unit of the scaling adjustment parameter is 1 / 8 of the chroma sample value per luma sample value (for 10-bit content).

[0206] In one example, adjustments are made to CCLM models ("LM_CHROMA_IDX" and "MMLM_CHROMA_IDX") that use reference samples above and to the left of the block, but not to "one-sided" mode. This choice is based on a trade-off between codec efficiency and complexity.

[0207] When applying scaling adjustments to a multi-mode CCLM model, both models may be adjusted and thus for a single chroma block, at most two scaling updates are signaled.

[0208] To enable scaling at the encoder, the encoder can perform a SATD-based search for the best value of the scaling update for Cr and a similar SATD-based search for Cb. If either result is a non-zero scaling parameter, the combined scaling pair (SATD-based update for Cr, SATD-based update for Cb) will be included in the list of RD checks for the TU.

[0209] Fusion of chroma intra prediction modes

[0210] During the development of ECM, JVET-Y0092 / Z0051 proposed the fusion of chroma intra mode.

[0211] The intra prediction modes enabled for chroma components in ECM-4.0 are six cross-component linear model (LM) modes (including CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T modes), direct mode (DM), and four default chroma intra prediction modes. The four default modes are given by the list {0, 50, 18, 1}, and if the DM mode already belongs to the list, the mode in the list will be replaced by mode 66.

[0212] The decoder-side intra mode derivation (DIMD) method for luma intra prediction is included in ECM-4.0. First, the horizontal and vertical gradients are calculated for each reconstructed luma sample of the L-shaped template of the second adjacent row and column of the current block to construct a histogram of gradients (HoG). Then, the two intra prediction modes with the largest and second largest histogram magnitude values ​​are blended with the planar mode to generate the final predictor for the current luma block.

[0213] In order to improve the coding and decoding efficiency of chroma intra prediction, two methods are proposed, including the chroma intra prediction mode derived at the decoder side (DIMD chroma) and the fusion of non-LM mode and MMLM_LT mode.

[0214] In the first embodiment, a DIMD chroma mode is proposed. The proposed DIMD chroma mode uses the DIMD derivation method to derive the chroma intra prediction mode of the current block based on the co-located reconstructed luma samples. Specifically, the horizontal gradient and vertical gradient are calculated for each co-located reconstructed luma sample of the current chroma block to construct the HoG, as shown in FIG. Figure 8C Then, the intra prediction mode with the largest histogram magnitude value is used to perform chroma intra prediction of the current chroma block.

[0215] When the intra prediction mode derived from the DIMD chroma mode is the same as the intra prediction mode derived from the DM mode, the intra prediction mode having the second largest histogram magnitude value is used as the DIMD chroma model.

[0216] As shown in Table 7, a CU level flag is signaled to indicate whether the proposed DIMD chroma mode is applied.

[0217] Table 7. Binarization process of intra_chroma_pred_mode in the proposed method

[0218] intra_chroma_pred_mode binary bit string Chroma Intra mode 0 1100 List[0] 1 1101 List[1] 2 1110 List[2] 3 1111 List[3] 4 10 DIMD chromaticity 5 0 DM

[0219] In the second embodiment, the fusion of chroma intra prediction modes is proposed, wherein the DM mode and four default modes can be fused with the MMLM_LT mode as shown below:

[0220] pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift

[0221] Where pred0 is the predictor obtained by applying the non-LM mode, pred1 is the predictor obtained by applying the MMLM_LT mode, and pred is the final predictor of the current chroma block. The two weights (w0 and w1) are determined by the intra prediction mode of the adjacent chroma blocks, and shift is set to be equal to 2. Specifically, when the upper and left adjacent blocks are both coded and decoded using the LM mode, {w0, w1} = {1, 3}; when the upper and left adjacent blocks are both coded and decoded using the non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}.

[0222] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied, and the proposed fusion is only applied to I slices.

[0223] In a third embodiment, the DIMD chroma mode is combined with the fusion of the chroma intra prediction mode. Specifically, the DIMD chroma mode described in the first embodiment is applied, and for I slices, the DM mode, the four default modes, and the DIMD chroma mode can be fused with the MMLM_LT mode using the weights described in the second embodiment, while for non-I slices, only the DIMD chroma mode can be fused with the MMLM_LT mode using equal weights.

[0224] In a fourth embodiment, the DIMD chroma mode with reduced processing is combined with a fusion of chroma intra prediction modes. Specifically, the DIMD chroma mode with reduced processing derives an intra mode based on the adjacent reconstructed Y, Cb, and Cr samples in the second adjacent row and column, such as Figure 8D The other parts are the same as those in the third embodiment.

[0225] In one embodiment, when DIMD is applied, two intra modes are derived from the reconstructed neighboring samples, and these two predictors are combined with a planar mode predictor with weights derived from gradients, as described in JVET-00449. The division operation in the weight derivation is performed using the same lookup table (LUT)-based integration scheme used by CCLM. For example, the division operation in the orientation calculation

[0226] Orient=G y / G x

[0227] is calculated using the following LUT-based scheme:

[0228] x=Floor(Log2(Gx))

[0229] normDiff=((Gx<<4)>>x)&15

[0230] x+=(3+(normDiff!=0)?1:0)

[0231] Orient=(Gy*(DivSigTable[normDiff]|8)+(1<<(x-1)))>>x

[0232] in,

[0233] DivSigTable

[16] ={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}.

[0234] The derived intra modes are included into the main list of intra most probable modes (MPMs), so the DIMD process is performed before constructing the MPM list. The main derived intra modes of a DIMD block are stored with the block and used for MPM list construction of neighboring blocks.

[0235] Figures 8E to 8H The steps of intra mode derivation at the decoder side are shown, where the intra prediction direction is estimated without intra mode signaling. Figure 8E The first step shown involves estimating the gradient at each sample point (for Figure 8E As shown in the light grey sample points). Figure 8F The second step shown consists of mapping the gradient value to the closest prediction direction within [2,66]. Figure 8G The third step shown in consists of selecting two prediction directions, where for each prediction direction, all absolute gradients Gx and Gy of the neighboring pixels in that direction are summed, and the top two directions are selected. Figure 8H The fourth step shown includes enabling weighted intra prediction using the selected direction.

[0236] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Figure 9 In

[15] , an example of 4 reference lines is depicted, where the samples of segments A and F are not taken from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra-picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0237] The index of the selected reference row (mrl_idx) is signaled and used to generate the intra predictor. For reference row idx greater than 0, only additional reference row modes are included in the MPM list, and only the mpm indices for which there are no remaining modes are signaled. The reference row index is signaled before the intra prediction mode, and if a non-zero reference row index is signaled, planar mode is excluded from the intra prediction mode.

[0238] For blocks in the first row within a CTU, MRL is disabled to prevent the use of extended reference samples outside the current CTU row. In addition, PDPC is also disabled when additional rows are used. For MRL mode, the derivation of the DC value in DC intra prediction mode for non-zero reference row index is consistent with the derivation for reference row index 0. MRL requires 3 adjacent luma reference rows stored in the CTU to generate the prediction. The Cross Component Linear Model (CCLM) tool also requires 3 adjacent luma reference rows for its downsampling filter. The definition of MRL using the same 3 rows is consistent with CCLM to reduce the storage requirements of the decoder.

[0239] During the development of ECM, the Convolutional Cross-Component Model (CCCM) for Chroma Intra Mode was proposed.

[0240] We propose to apply a convolutional cross-component model (CCCM) to predict chroma samples from reconstructed luma samples, similar in spirit to the current CCLM mode. Like CCLM, when chroma subsampling is used, the reconstructed luma samples are downsampled to match the lower resolution chroma grid.

[0241] In addition, similar to CCLM, there is an option to use a single model or a multi-model variant of CCCM. The multi-model variant uses two models, one derived for samples above the average luminance reference value and another derived for the remaining samples (following the spirit of CCLM design). For PUs with at least 128 available reference samples, the multi-model CCCM mode can be selected.

[0242] The proposed convolutional 7-tap filter consists of a 5-tap plus sign shape spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of the center (C) luminance sample (which is co-located with the chrominance sample to be predicted) and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighboring samples, as shown in Figure 10A shown.

[0243] The nonlinear term P is expressed as a power of two of the center luma sample C and scaled to the sample value range of the content:

[0244] P=(C*C+midVal)>>bitDepth

[0245] That is, for 10-bit content, it is calculated as:

[0246] P=(C*C+512)>>10

[0247] The bias term B represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the intermediate chrominance value (512 for 10-bit content).

[0248] The output of the filter is calculated as the filter coefficient c i Convolution with the input value and clipped to the range of valid chroma samples:

[0249] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0250] The filter coefficients c are calculated by minimizing the MSE between the predicted chroma samples and the reconstructed chroma samples in the reference region i . Figure 10B A reference region is shown, consisting of six rows of chroma samples above and to the left of the PU. The reference region extends one PU width to the right and one PU height below the PU boundary. The region is adjusted to include only available samples. The extension of the region shown in blue is required to support the "side sampling" of the plus-shaped spatial filter and to fill in when in unavailable areas.

[0251] MSE minimization is performed by computing the autocorrelation matrix for the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​subjected to an LDL decomposition, and back substitution is used to compute the final filter coefficients. This process roughly follows the calculation of the ALF filter coefficients in ECM, however, LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations. The proposed method uses only integer arithmetic.

[0252] The use of this mode is signaled using the PU-level flag of the CABAC codec. A new CABAC context is included to support this. In terms of signaling, CCCM is treated as a sub-mode of CCLM. That is, the CCCM flag is signaled only when the intra prediction mode is LM_CHROMA_IDX (to enable single-mode CCCM) or MMLM_CHROMA_IDX (to enable multi-mode CCCM).

[0253] The encoder performs two new RD checks in the chroma prediction mode loop, one for checking single-model CCCM mode and one for checking multi-model CCCM mode.

[0254] In existing CCLM or MMLM designs, adjacent reconstructed luma-chroma sample pairs are classified into one or more sample groups based on a threshold value, which only considers the luma DC value. That is, luma-chroma sample pairs are classified by considering only the intensity of the luma sample. However, the luma component typically retains rich texture, and the current luma sample may be highly correlated with neighboring luma samples. Such inter-sample correlation (AC correlation) may be beneficial for the classification of luma-chroma sample pairs and can bring additional coding efficiency.

[0255] like Figure 10C As shown in , CCLM assumes that a given chroma sample is only related to the corresponding luma sample (L0.5, which can be taken as the fractional luma sample position), and uses a simple linear regression (SLR) estimate using ordinary least squares (OLS) to predict the given chroma sample. Figure 10DAs shown in , in some video contents, a chrominance sample can be simultaneously correlated with multiple luma prediction samples (AC or DC correlation), so the multiple linear regression (MLR) model can further improve the accuracy.

[0256] Although CCCM can enhance intra-frame prediction efficiency, its performance still needs to be further improved. Furthermore, some parts of the existing CCCM mode need to be simplified to enable efficient codec hardware implementations or improved to achieve better codec efficiency. Furthermore, the trade-off between implementation complexity and codec efficiency benefits needs to be further improved.

[0257] Marginal Linear Model (ELM)

[0258] To improve the coding efficiency of luma and chroma components, a classifier that considers luma edge or AC information is introduced, in contrast to the above implementation method in which only the luma DC value is considered. In addition to the existing band-classified MMLM, the present disclosure also provides an exemplary classifier. The process of generating linear prediction models for different groups of samples may be similar to CCLM or MMLM (e.g., via least squares or a simplified minimum-maximum method), but using different metrics for classification. Different classifiers can be used to classify adjacent luma samples (e.g., adjacent luma-chroma sample pairs) and / or luma samples corresponding to the chroma samples to be predicted. Luma samples corresponding to chroma samples can be obtained by downsampling to match the positions of corresponding chroma samples in a 4:2:0 video sequence. For example, luma samples corresponding to chroma samples can be obtained by performing a downsampling operation on more than one (e.g., four) reconstructed luma samples corresponding to the chroma samples (e.g., located around the chroma samples). Alternatively, for example, in the case of a 4:4:4 video sequence, the luma samples may be obtained directly from the reconstructed luma samples. Alternatively, the luma samples may be obtained from corresponding luma samples in the reconstructed luma samples located at the corresponding co-located positions of the corresponding chroma samples. For example, the luma sample to be classified may be obtained from one of the four reconstructed luma samples corresponding to the chroma sample, the chroma sample being located at the upper left position of the four reconstructed luma samples, which position may be regarded as the co-located position of the chroma samples.

[0259] The first classifier can classify luma samples based on their edge strength. For example, a direction (e.g., 0 degrees, 45 degrees, or 90 degrees, etc.) can be selected to calculate the edge strength. The direction can be formed by the current sample and the neighboring samples along that direction (e.g., the neighboring sample located 45 degrees to the upper right of the current sample). The edge strength can be calculated by subtracting the neighboring samples from the current sample. The edge strength can be quantized into one of M segments using M-1 thresholds, and the first classifier can use M classes to classify the current sample. Alternatively or additionally, N directions can be formed by the current sample and N neighboring samples along N directions. N edge strengths can be calculated by subtracting the N neighboring samples from the current sample, respectively. Similarly, if each of the N edge strengths can be quantized into one of M segments using M-1 thresholds, the first classifier can use MN classes to classify the current sample.

[0260] The second classifier can be used to classify based on local patterns. For example, the current luma sample Y0 can be compared with its N neighboring luma samples Yi. If the value of Y0 is greater than the value of Yi, the score may be increased by one, otherwise the score may be decreased by one. The score can be quantized to form K classes. The second classifier can classify the current sample into one of the K classes. For example, the neighboring luma samples can be obtained from the four neighboring samples located above, to the left, to the right, and below the current luma sample, i.e., there are no diagonal neighboring samples.

[0261] It is contemplated that multiple first classifiers, second classifiers, or different instances of the first classifier or the second classifier or other classifiers described herein can be combined. For example, the first classifier can be combined with an existing MMLM threshold-based classifier. For another example, an instance A of the first classifier can be combined with another instance B of the first classifier, where instances A and B are in different orientations (e.g., vertical and horizontal, respectively).

[0262] Those skilled in the art will recognize that, although the existing CCLM design in the VVC standard is used as the basic CCLM method in the specification, the proposed cross-component method described in this disclosure can also be applied to other predictive codecs with similar design spirit. For example, for Chroma from Luma (CfL) in the AV1 standard, the proposed method can also be applied by dividing the luma-chroma sample pairs into multiple sample groups.

[0263] Those skilled in the art will recognize that Y / Cb / Cr can also be expressed as Y / U / V in the field of video coding and decoding. For example, if the video data is in RGB format, the proposed method can also be applied by simply mapping the YUV symbol to GBR.

[0264] Filter-based linear model (FLM)

[0265] Considering the possibility that one chroma sample may be correlated with multiple luma samples simultaneously, a filter-based linear model (FLM) using an MLR model is introduced, as described below.

[0266] For the chroma samples to be predicted, the reconstructed co-located and adjacent luma samples can be used to predict the chroma samples to capture the inter-sample correlation between the co-located luma samples, adjacent luma samples and chroma samples. The reconstructed luma samples are linearly weighted and combined with an "offset" to generate predicted chroma samples (C: predicted chroma samples, L i : the i-th reconstructed co-located or adjacent brightness sample, α i : filter coefficients, β: offset, N: filter taps), as shown in the following equation (32-1). Note that the linear weighting plus the offset value directly forms the predicted chroma samples (which can be adaptively low-pass or high-pass according to the video content), and they are then added to the residual to form the reconstructed chroma samples.

[0267]

[0268] In some implementations like CCCM, the offset term may also be implemented as the intermediate chrominance value B (512 for 10-bit content) multiplied by another coefficient, as shown in equation (32-2) below.

[0269]

[0270] For a given CU, the top and left reconstructed luma and chroma samples can be used to derive or train FLM parameters (α i, , β). As with CCLM, α i and β can be derived via OLS. The top and left training samples are collected and a pseudo-inverse matrix is ​​computed at both the encoder and decoder sides to derive the parameters, which are then used to predict the chroma samples in a given CU. Let N denote the number of filter taps applied to the luma samples, M denote the total top and left reconstructed luma-chroma sample pairs used for training the parameters, represents the luma sample with the i-th sample pair and the j-th filter tap, C i Denotes the chrominance sample with the i-th sample pair, the following equation shows the pseudo-inverse matrix A + The derivation and parameters of . Figure 11 An example is shown where N is 6 (6 taps), M is 8, and parameters are derived or trained using luma samples from the top 2 rows and 3 columns from the left, and chroma samples from the top 1 row and 1 column from the left.

[0271]

[0272] b=Ax

[0273] x=(A T A) -1 A T b=A + b (33)

[0275] Note that without the bias β, one can only i Predict chroma samples, which may be a subset of the proposed method.

[0276] The proposed ELM / FLM / GLM (discussed below) can be directly extended to the CfL design in the AV1 standard, which explicitly sends the model parameters (α, β). For example, (1-tap case) α and / or β are derived at the encoder at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and signaled to the decoder for use in CfL mode.

[0277] In order to further improve the encoding and decoding performance, additional designs can be used in FLM prediction. Figure 11 As shown and discussed above, a 6-tap luma filter is used for FLM prediction. However, while a multi-tap filter can fit the training data well (e.g., the reconstructed luma and chroma samples adjacent to the top and left), in some cases, the training data does not capture the complete characteristics of the test data, which may lead to overfitting and may not predict the test data well (i.e., the chroma block samples to be predicted). In addition, different filter shapes can adapt well to different video block contents, resulting in more accurate predictions.

[0278] To address this issue, the filter shape and the number of filter taps may be predefined or signaled or switched at the sequence parameter set (SPS), adaptive parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, CTU, CU, sub-block, or sample level. A set of filter shape candidates may be predefined, and the selection of the set of filter shape candidates may be signaled or switched at the SPS, APS, PPS, PH, SH, region, CTU, CU, sub-block, or sample level. Different components (e.g., U and V) may have different filter switching controls. For example, a set of filter shape candidates may be predefined (e.g., indicated by indices 0 to 5), and a filter shape (1,2) may represent a 2-tap luminance filter, a filter shape (1,2,4) may represent a 3-tap luminance filter, and so on. Figure 11As shown. The filter shape selection for U and V components can be switched at the PH or CU or CTU level. Note that N taps can represent N taps with or without the offset β described in this document. Table 8 below gives an example.

[0279] Table 8 - Example signaling and switching for different filter shapes

[0280]

[0281]

[0282] Different chroma types and / or color formats may have different predefined filter shapes and / or taps. Figure 12 As shown, the predefined filter shape (1,2,4,5) can be used for 4:2:0 type-0, the predefined filter shape (0,1,2,4,7) can be used for 4:2:0 type-2, and the predefined filter shape (1,4) can be used for 4:2:2, and the predefined filter shape (0,1,2,3,4,5) can be used for 4:4:4.

[0283] In another aspect of the present disclosure, unavailable luma and chroma samples used to derive the MLR model can be filled in from available reconstructed samples. Figure 12 As shown in the 6-tap (0,1,2,3,4,5) filter, for the CU located at the left picture boundary, the left column including sample (0,3) is unavailable (beyond the picture boundary), so sample (0,3) is a repeated padding from sample (1,4) to apply the 6-tap filter. Note that the padding process can be applied to both training data (reconstructed luma and chroma samples adjacent to the top and left) and test data (luma and chroma samples in the CU).

[0284] One or more shapes / numbers of filter taps can be used for FLM prediction, examples are given in Figure 16 、 Figure 17 and 18A to 18B One or more sets of filter taps can be used for FLM prediction, for example in Figures 19A to 19G Shown in.

[0285] The filter shape candidate may be derived implicitly without explicit signaling bits. For example, the filter shape candidate may be a filter shape candidate for FLM or GLM (as described below). In another example, the filter shape candidate may be a cross filter for CCCM, Figure 16 、 Figure 17 、 Figure 18A 、 Figure 18B and Figures 19A to 19GAny of the filters shown in or other filters mentioned in this disclosure. N-fold training techniques well known in the field of machine learning can be used to train the filter coefficients.

[0286] The following example involves implicit filter shape derivation for FLM prediction:

[0287] Step 1: Determine M filter shape candidates for predicting the chroma sample values ​​of the current CU;

[0288] Step 2: Divide the available L-shaped template area outside the CU into N regions, denoted as R0, R1, ..., R N-1 , that is, the training data is divided into N sets for N-fold training, where the luminance sample values ​​and chrominance sample values ​​of the available template area are known values; Step 3: Each of the M filter shape candidates is applied to a part of the available template area, that is, all N regions R0, R1, ..., R N-1 One or more areas in

[0289] Step 4: Derive M sets of filter coefficients corresponding to the M filter shape candidates, denoted as F0, F1, ..., F M-1 ; Step 5: Export F0, F1, ..., F M-1 Applying the set of filter coefficients to another portion of the available template area to predict chrominance sample values ​​based on corresponding luma sample values, wherein the another portion of the available template area is different from the portion of the available template area mentioned in step 3;

[0290] Step 6: For each of the M filter shapes, accumulate the errors between the predicted chrominance sample values ​​and the known chrominance sample values ​​in another part of the available template area by sum of absolute differences (SAD), sum of squared differences (SSD), or sum of absolute transformed differences (SATD), denoted as E0, E1, ..., E M-1 ;

[0291] Step 7: Sort and select the K smallest errors, denoted as E'0, E'1, ..., E' K-1 , which corresponds to K filter shapes and K sets of filter coefficients; and

[0292] Step 8: Select a filter shape candidate from the K filter shape candidates to apply to the current CU for chroma prediction. If K is greater than 1, the decoder can still receive a signal from the encoder indicating the filter to be applied. However, if K is 1, the signaling can be omitted and the filter with the smallest accumulated error is determined as the filter to be applied.

[0293] In one example, one of the four filter shape candidates is selected as the filter to be applied, while the L-shaped template area is divided into even and odd rows or columns. These steps include:

[0294] Four candidate filter shapes are predefined for the current CU (e.g., from Figure 16 4 filter shape candidates in );

[0295] Divide the available L-shaped template area (e.g., 6 chrominance rows and columns for CCCM) into 2 regions denoted as R0, R1, where, for example, R0 consists of even rows or columns and R1 consists of odd rows or columns;

[0296] Apply the four filter shape candidates independently to a portion of the available template region (e.g., region R0);

[0297] Four sets of filter coefficients are derived for each of the four filter shapes, denoted as F0, F1, ..., F3;

[0298] Applying the derived F0, F1, ..., F3 filter coefficient sets to another part of the available template area (e.g., area R1) to predict corresponding chrominance sample values;

[0299] For each of the four filters, the error (expressed as E0, E1, ..., E3) between the predicted chroma sample value and the known chroma sample value in another part of the available template area (e.g., area R1) is accumulated by SAD, SSD, or SATD; and a filter shape candidate with the smallest accumulated error (expressed as E'0) among E0, E1, ..., E3 among the four filter shape candidates is selected, where E'0 corresponds to a filter shape and a set of filter coefficients. In this example, only the filter shape candidate with the smallest accumulated error will be determined as the filter to be applied, and then the decoder does not need to receive a signal indicating the application of the filter.

[0300] In one example, the L-shaped template area can be divided into staggered parts, and K=2. The steps include:

[0301] Determine four filter shape candidates for the current CU;

[0302] The available L-shaped template area (e.g., 6 chroma rows or columns for CCCM) is divided into two areas denoted as R0 and R1, where the luma samples in R0 and R1 are interleaved as shown in the following table:

[0303] <![CDATA[R0]]> <![CDATA[R1]]> <![CDATA[R1]]> <![CDATA[R0]]>

[0304] or

[0305] <![CDATA[R1]]> <![CDATA[R0]]> <![CDATA[R0]]> <![CDATA[R1]]>

[0306] Apply the four filter shape candidates independently to a portion of the available template region (e.g., region R0);

[0307] deriving four sets of filter coefficients for the four filter shapes, denoted as F0, F1, ..., F3;

[0308] Applying the derived F0, F1, ..., F3 filter coefficient sets to another part of the available template area (e.g., area R1) to predict corresponding chrominance sample values;

[0309] For each of the four filters, the error (denoted as E0, E1, ..., E3) between the predicted chroma sample value and the known chroma sample value in another part of the available template area (for example, area R1) is accumulated by SAD, SSD or SATD respectively;

[0310] Sort by E0, E1, ..., E 3并 Selecting the two smallest accumulated errors among E0, E1, ..., E3, denoted as E'0, E'1, wherein E'0, E'1 correspond to two filter shapes and two filter coefficient sets; and

[0311] Based on a signal received from the encoder, one of the two filter shape candidates is selected to be applied to the current CU for chroma prediction.

[0312] In one example, the step of dividing the available L-shaped template area can be omitted. In this instance, M sets of filter coefficients can be derived based on the sample values ​​from the available template area, and then the M sets of filter coefficients can be applied back to the available template area to predict corresponding chrominance sample values ​​for accumulating errors.

[0313] As mentioned above, the MLR model (linear equation) must be derived at both the encoder and the decoder. According to one or more aspects of the present disclosure, several methods are proposed to derive the pseudo-inverse matrix A. + , or directly solve the linear equation. Other known methods can also be applied, such as Newton's method, Cayley–Hamilton method, and eigendecomposition as mentioned in https: / / en.wikipedia.org / wiki / Invertible_matrix.

[0314] In this disclosure, A + Indicated as A -1 , to simplify. The linear equation can be solved as follows: 1. Solve A by adjoint matrix (adjA), closed form, analytical solution-1 :

[0315] Below is shown an n×n general form, a 2×2 and a 3×3 case. If FLM uses 3×3, then 2 scalers plus an offset need to be solved.

[0316] b=Ax,x=(A T A) -1 A T b=A + b, represented by A -1 b

[0317]

[0318] By removing the (n-1)×(n-1) submatrix with the j-th row and the i-th column.

[0319]

[0320] 2. Gauss-Jordan elimination method

[0321] The linear equation can be solved using Gauss-Jordan elimination method by augmenting the matrix [AI n ] and a series of elementary row operations to obtain the simplified row echelon form [I|X]. 2×2 and 3×3 examples are shown below.

[0322]

[0323] 3. Cholesky decomposition

[0324] To solve Ax = b, we first decompose A using the Cholesky-Crout algorithm to obtain an upper triangular matrix and a lower triangular matrix. We then apply a forward substitution and a backward substitution to obtain the solution. A 3x3 example is shown below.

[0325]

[0326] In addition to the above examples, some cases require special handling. For example, if certain situations result in that the linear equation cannot be solved, a default value can be used to fill in the chroma prediction value. The default value can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level, for example, when 1<<(bitDepth-1), meanC, meanL, or meanC-meanL (average current chroma or other chroma, available luma value, or a subset of FLM reconstructed neighboring regions) is predefined.

[0327] The following examples represent situations where matrix A cannot be solved, where the default predictor can be assigned to the entire current block:

[0328] 1. Solve by closed form (analytical, adjoint matrix), but A is singular (i.e., detA = 0);

[0329] 2. Solve by Cholesky decomposition, but A cannot be Cholesky decomposed, g jj <REG_SQR, where REG_SQR is a small value that can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0330] Figure 11 Illustrates a typical case of deriving FLM parameters using the top 2 and / or left 3 luminance rows and the top 1 and / or left 1 chrominance row. However, as mentioned above, due to different block contents and reconstruction qualities of different neighboring samples, using different regions for parameter derivation may bring coding and decoding benefits. Several methods for selecting the application region for parameter derivation are proposed below:

[0331] 1. Similar to MDLM, FLM derivation can only use top or left luminance and / or chrominance samples to derive parameters. Whether to use FLM, FLM_L or FLM_T can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Assuming the size of the current chrominance block is W×H, then W’ and H’ are obtained as follows:

[0332] - When applying the FLM mode, W’ = W, H’ = H;

[0333] - When applying the FLM_T mode, W’ = W + We; where We represents the extended top luminance / chrominance samples;

[0334] - When applying the FLM_L mode, H’ = H + He; where He represents the extended left luminance / chrominance samples.

[0335] The number of extended luminance / chrominance samples (We, He) can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0336] For example, (We, He) = (H, W) is predefined as VVC CCLM, or (W, H) is predefined as ECM CCLM. Unavailable (We, He) luminance / chrominance samples can be repeatedly filled from the nearest (horizontal, vertical) luminance / chrominance samples.

[0337] Figure 13 A diagram of FLM_L and FLM_T is shown (eg, under 4 taps). When FLM_L or FLM_T is applied, only H' or W' luma / chroma samples, respectively, are used for parameter derivation.

[0338] 2. Similar to MRL, different row indices can be predefined or signaled or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to indicate the selected luminance-chrominance sample pair rows. This may benefit from different reconstruction qualities of different row samples.

[0339] Figure 14 It is shown that, similar to MRL, FLM can use different rows for parameter derivation (e.g., under 4 taps). For example, FLM can use light blue / yellow luma and / or chroma samples in index 1.

[0340] 3. Expand the CCLM region and obtain the complete top N rows and / or left M rows for parameter derivation. Figure 14 It shows that all dark blue, light blue and yellow areas can be used at the same time. Using a larger area (data) for training may produce a more robust MLR model.

[0341] It should be understood that throughout this disclosure, luma sample values ​​of an outer region of a video block to be decoded may be referred to as “outer luma sample values,” and chroma sample values ​​of an outer region may be referred to as “outer luma sample values.”

[0342] The corresponding syntax for FLM prediction may be defined as follows in Table 9, where FLC represents a fixed length code, TU represents a truncated unary code, EGk represents an exponential Golomb code with order k, where k may be fixed or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level, SVLC represents signed EG0, and UVLC represents unsigned EG0.

[0343] Table 9 - Example of FLM syntax

[0344]

[0345]

[0346] Note that the binarization of each syntax element can be changed.

[0347] Based on the existing linear model design, a new method for cross-component prediction is proposed to further improve the encoding and decoding accuracy and efficiency. The main aspects of the proposed method are detailed below.

[0348] While the FLM discussed above offers the best flexibility (resulting in the best performance), it requires solving for many unknown parameters as the number of filter taps increases. When the inverse matrix is ​​larger than 3×3, closed-form derivation is unsuitable (too many multipliers) and requires iterative methods like Cholesky, which increases the burden on the decoder processing loop. In this section, we propose pre-operations before applying the linear model, including the use of sample gradients to exploit the correlation between luma AC information and chroma intensity. With the help of gradients, the number of filter taps can be effectively reduced.

[0349] Please note that the methods / examples in this section can be combined / reused from any of the designs discussed above, including but not limited to classification, filter shapes, matrix derivation (with special handling), application areas, and syntax. In addition, the methods / examples listed in this section can also be applied to any of the designs discussed above to achieve better performance with certain complexity trade-offs.

[0350] Please note that reference samples / training templates / reconstructed neighborhood as used in this article generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0351] According to the proposed method, pre-operations (e.g., pre-linear weighting, sign, scale / absolute value, thresholding, ReLU) can be applied to reduce the dimensionality of unknown parameters instead of directly using luma sample intensity values ​​as input to the linear model. In one example, the pre-operations can include calculating sample differences based on the luma sample values. As understood by those skilled in the art, the sample differences can be characterized as gradients, and therefore this new method is also referred to as a gradient linear model (GLM) in some embodiments.

[0352] Note that the following detailed description discusses scenarios in which the proposed pre-operation can be reused for / combined with the SLR model (also known as the 1-tap case) and reused for / combined with the MLR model (also known as the multi-tap case, e.g., 2-tap).

[0353] For example, instead of applying a 2-tap on the 2 luma samples, the 2 luma samples can be pre-operated and then a simpler 1-tap can be applied to reduce complexity. 15A to 15DSome examples of 1-tap / 2-tap (with offset) pre-operation are shown, where the 2-tap coefficients are represented as (a, b). Note that 15A to 15D Each circle shown in represents an illustrative chroma position in the YUV 4:2:0 format. As discussed above, in the YUV 4:2:0 format, a luma sample corresponding to a chroma sample can be obtained by performing a downsampling operation on more than one (e.g., 4) reconstructed luma samples corresponding to the chroma sample (e.g., located around the chroma sample). In other words, a chroma position can correspond to one or more luma samples including a co-located luma sample. Different 1-tap modes are designed for different gradient directions and use different "interpolated" luma samples (weighted to different luma positions) for gradient calculation. For example, a typical filter [1,0,-1; 1,0,-1] is as follows Figure 15A 、 Figure 15C and Figure 15D As shown, this represents the following operation:

[0354]

[0355] Among them, rec L Represents the reconstructed luminance sample value, and Rec L ″(i,j) represents the pre-calculated luminance sample value. Also note that Figure 15A 、 Figure 15C and Figure 15D The 1-tap filter shown can be understood as an alternative to the downsampling filter as used in CCLM (see equations (6)-(7)), with the filter coefficients changed.

[0356] Pre-operation can be based on gradient, edge direction (detection), pixel intensity, pixel change, pixel variance, Roberts / Prewitt / compass / Sobel / Laplacian operators, high-pass filters (by calculating gradients or other related operators), low-pass filters (by performing weighted average operations) ... etc. The edge direction detector listed in the example can be extended to different edge directions. For example, applying 1 tap (1, -1) or 2 taps (a, b) along different directions to detect different edge gradients. The filter shape / coefficient can be symmetrical about the chrominance position, with 15A to 15D Same as example (420 type 0 case).

[0357] The pre-operation parameters (coefficient, sign, scale / absolute value, thresholding, ReLU) can be fixed or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Note that in the example, if multiple coefficients apply to one sample (e.g., -1, 4), they can be merged (e.g., 3) to reduce operations.

[0358] In one example, the pre-operation may involve calculating the sample difference of the luma sample value. Alternatively, the pre-operation may include performing downsampling by weighted averaging. In some cases, the pre-operation may be applied repeatedly. For example, a template filter may be applied to the template using a low-pass smoothing FIR filter [1,2,1] / 4 or [1,2,1; 1,2,1] / 8 (i.e., downsampling) to remove outliers, and then a 1-tap GLM filter may be applied to calculate the sample difference to derive the linear model. It is contemplated that the sample difference may also be calculated and then downsampling may be enabled.

[0359] In one example, pre-operation coefficients (applied last (eg, 3), or applied intermediately (eg, -1, 4) to each luma sample) may be restricted to power-of-two values ​​to save multipliers.

[0360] In one aspect of the present disclosure, the proposed new method can be reused / combined with the CCLM discussed above, which utilizes a simple linear regression (SLR) model and uses a corresponding luma sample value to predict the chroma sample value. This is also called the 1-tap case. In this case, deriving the linear model further includes deriving the scaling parameter α and the offset parameter β by using the pre-calculated neighboring luma sample values ​​and the neighboring chroma sample values. Alternatively, the linear model can be rewritten as:

[0361] C=α·L+β (35)

[0362] Here, L represents the "pre-calculated" luma sample. The parameter derivation for the 1-tap GLM can reuse the CCLM design, but consider the directional gradient (possibly with a high-pass filter). In one example, the scaling parameter α can be derived by using a division lookup table (as described in detail below) for simplification.

[0363] In one example, when combining the GLM with the SLR model, the scaling parameter α and the offset parameter β can be derived by using the minimum-maximum method discussed above. Specifically, the scaling parameter α and the offset parameter β can be derived by comparing the pre-calculated adjacent luminance sample values ​​to determine the minimum luminance sample value Y A and the maximum brightness sample value Y B ; Determine the minimum brightness sample value YA and the maximum brightness sample value Y B The corresponding chroma sample value X A and X B ; and according to the following equation, based on the minimum brightness sample value Y A , maximum brightness sample value Y B and the corresponding chroma sample value X A and X B Derive the scaling parameter α and the offset parameter β:

[0364]

[0365] β=Y A -αX A (36)

[0366] In one example, when combining the GLM with the SLR model, the scaling adjustment discussed above can be reused. In this case, the encoder can determine a scaling adjustment value (e.g., "u") to be signaled in the bitstream and add the scaling adjustment value to the derived scaling parameter α. The decoder can determine the scaling adjustment value (e.g., "u") from the bitstream and add the scaling adjustment value to the derived scaling parameter α. The added value is ultimately used to predict the intra chroma sample value.

[0367] In one aspect of the present disclosure, the proposed new method can be reused / combined with FLM, which uses a multiple linear regression (MLR) model and uses multiple luma sample values ​​to predict chroma sample values. This is also called a multi-tap case, such as 2 taps. In this case, the linear model can be rewritten as:

[0368]

[0369] In this case, multiple scaling parameters α and offset parameters β can be derived using adjacent luma sample values ​​and adjacent chroma sample values ​​from the pre-operation. In one example, the offset parameter β is optional. In one example, at least one of the multiple scaling parameters α can be derived using sample differences. In addition, another scaling parameter of the multiple scaling parameters α can be derived using downsampled luma sample values. In one example, at least one of the multiple scaling parameters α can be derived using horizontal or vertical sample differences calculated based on downsampled adjacent luma sample values. In other words, the linear model can combine multiple scaling parameters α associated with different pre-operations.

[0370] Implicit filter shape derivation

[0371] In one example, the directional filter shape used can be derived at the decoder to save bit overhead, rather than explicitly signaling the selected filter shape index. For example, at the decoder, multiple directional gradient filters can be applied to each reconstructed luma sample of the L-shaped template of the i-th adjacent row and column of the current block. The filtered values ​​(gradients) can then be accumulated for each direction in the multiple directional gradient filters. In the example, the accumulated value is the accumulated value of the absolute value of the corresponding filtered values. After accumulation, the direction of the directional gradient filter with the largest accumulated value can be determined as the derived (luminance) gradient direction. For example, a histogram of gradients (HoG) can be constructed to determine the maximum value. The derived direction can be further used as the direction for predicting the chroma samples in the current block.

[0372] The following example involves reusing the decoder-side intra mode derivation (DIMD) method included in ECM-4.0 for luma intra prediction:

[0373] Step 1: Apply two directional gradient filters (3×3 horizontal / vertical Sobel) to each reconstructed luminance sample of the L-shaped template of the second adjacent row and column of the current block;

[0374] Step 2: Accumulate the filtered values ​​(gradients) for each directional gradient filter through SAD (sum of absolute differences);

[0375] Step 3: Constructing a Histogram of Gradients (HoG) based on the accumulated filtered values; and

[0376] Step 4: The maximum value in the HoG is determined as the derived (brightness) gradient direction, based on which the GLM filter can be determined.

[0377] In one example, if the shape candidates are [-1, 0, 1; -1, 0, 1] (horizontal) and [1, 2, 1; -1, -2, -1] (vertical), when the maximum value is associated with the horizontal shape, the shape [-1, 0, 1; -1, 0, 1] is used for GLM-based chrominance prediction.

[0378] The shape of the gradient filter used to derive the gradient direction can be the same as or different from the GLM filter. For example, both filters can be horizontal [-1, 0, 1; -1, 0, 1], or the two filters can have different shapes, and the GLM filter can be determined based on the gradient filter.

[0379] The proposed GLM can be combined with the MMLM or ELM discussed above. When combined with classification, each group can share or have its own filter shape, where the syntax indicates the shape for each group. For example, as an exemplary classifier, the horizontal gradient grad_hor can be classified into the first group, which corresponds to the first linear model, and the vertical gradient grad_ver can be classified into the second group, which corresponds to the second linear model. In one example, the horizontal brightness pattern can be generated only once.

[0380] Another possible classifier is provided below. Using the classifier, pairs of adjacent and internal luma samples of the current video block can be classified into a plurality of groups based on one or more thresholds. Note that, as discussed above, each adjacent / internal chroma sample and its corresponding luma sample can be referred to as a luma-chroma sample pair. One or more thresholds are associated with the intensity of the adjacent / internal luma samples. In this case, each of the plurality of groups corresponds to a respective linear model of the plurality of linear models.

[0381] When combined with an MMLM classifier, the following operations may be performed: neighboring reconstructed luma-chroma sample pairs of the current video block are classified into two groups based on a threshold; different linear models are derived for different groups, wherein the derivation process may be simplified by GLM, i.e., the number of taps is reduced by the above-mentioned pre-operation; similarly, luma-chroma sample pairs within the CU (intra-luma-chroma sample pairs, wherein each intra-luma-chroma sample pair includes an intra-chroma sample value predicted using the derived linear model) are classified into two groups based on a threshold; different linear models are applied to the reconstructed luma samples in different groups; and chroma samples in the CU are predicted based on different classified linear models. The threshold may be the average value of the neighboring reconstructed luma samples. Note that the number of classes (2) can be extended to multiple classes by increasing the number of Thresholds (e.g., equally divided based on the min / max values ​​of adjacent reconstructed (downsampled) luma samples, fixed or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level).

[0382] In one example, instead of the MMLM luma DC intensity, the filtered values ​​of the FLM / GLM applied to the adjacent luma samples are used for classification. For example, if a 1-tap (1, -1) GLM is applied, the average AC value (physical meaning) is used. The processing can be: based on one or more filter shapes, one or more filtered values, and K-1Threshold Ti, the adjacent reconstructed luma-chroma sample pairs are classified into K groups; different MLR models are derived for different groups, wherein the derivation process can be simplified by GLM, that is, the number of taps is reduced by the above-mentioned pre-operation; similarly, based on one or more filter shapes, one or more filtered values, and K-1Threshold Ti, the luma-chroma sample pairs within the CU (intra-luma-chroma sample pairs, wherein each intra-luma-chroma sample pair in the intra-luma-chroma sample pairs includes an intra-chroma sample value predicted using the derived linear model) are classified into K groups; different linear models are applied to the reconstructed luma samples in different groups; and the chroma samples in the CU are predicted based on different classified linear models. Among them, the Threshold can be predefined (e.g., 0, or can be a table) or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, the Threshold can be the average AC value (filtered value) of adjacent reconstructed (possibly downsampled) luma samples (2 groups), or based on the minimum / maximum AC equal division (K groups).

[0383] It is also proposed to combine GLM with ELM classifier. 15A to 15D As shown, a filter shape (e.g., 1 tap) can be selected to calculate the edge strength. The direction is determined as the direction along which the sample difference between the current sample and N neighboring samples (e.g., all 6 luma samples) is calculated. For example, Figure 15AThe filter at the upper middle part of (shape [1,0,-1; 1,0,-1]) indicates the horizontal direction, because the sample difference between samples can be calculated in the horizontal direction, and the filter below it (shape [1,2,1; -1,-2,-1]) indicates the vertical direction, because the sample difference between samples can be calculated in the vertical direction. The positive coefficients and negative coefficients in each of the filters enable the calculation of sample differences. The processing may then include: calculating an edge intensity through the filtered value (e.g., equivalent); quantizing the edge intensity into M segments through M-1 thresholds Ti; classifying the current sample using K classes (e.g., K == M); deriving different MLR models for different groups, wherein the derivation process may be GLM-simplified, that is, reducing the number of taps through the above-mentioned pre-operation; classifying the luminance-chrominance sample pairs within the CU into K groups; applying different MLR models to the reconstructed luminance samples in different groups; and predicting the chrominance samples in the CU based on the differently classified MLR models. Note that the filter shape used for classification can be the same as or different from the filter shape used for MLR prediction. Both threshold M-1 and threshold Ti, as well as the number of thresholds M-1 and threshold Ti, can be fixed or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. In addition, other classifiers / combinations of classifiers as discussed in ELM can also be used for GLM.

[0384] If the number of classification samples in a group is less than a number (e.g., a predefined 4), the default values ​​mentioned in the discussion of matrix derivation for the MLR model can be applied to the group parameters (α i, , β). If the corresponding neighboring reconstructed samples are not available for the selected LM mode, a default value may be applied, for example, when the MMLM_L mode is selected but the remaining samples are invalid.

[0385] Several methods related to the simplification of GLM are introduced below to further improve the encoding and decoding efficiency.

[0386] The matrix / parameter derivation in FLM requires floating point operations (e.g. closed form division), which is expensive for the decoder hardware, hence the need for a fixed point design. For the 1-tap GLM case, it can be viewed as a modified luminance reconstructed sample generation of CCLM (e.g. horizontal gradient direction, from CCLM [1, 2, 1; 1, 2, 1] / 8 to GLM [-1, 0, 1; -1, 0, 1]), the original CCLM process can be reused for GLM, including fixed point operations, MDLM downsampling, division table, applying size constraints, min-max approximation and scaling adjustment. The 1-tap GLM can have its own configuration or share the same design with CCLM for all terms. For example, the parameters are derived using a simplified min-max method (instead of LMS) and combined with scaling adjustment after deriving the GLM model. In this case, the center point for rotating the slope (luminance value y r ) becomes the average value of the reference luma sample "gradient". For another example, when GLM is turned on for the CU, CCLM slope adjustment is inferred to be off, and no syntax related to slope adjustment needs to be signaled.

[0387] This section takes the typical case reference sample (top row and left column) as an example. Figure 14 As shown, the extended reconstructed region may also use the same essential simplification and may have a syntax (eg, MDLM, MRL) to indicate a specific region.

[0388] Please note that the following aspects can be combined and applied jointly: For example, combining reference sample downsampling and partitioning tables to perform the partitioning process.

[0389] When applying classification (MMLM / ELM), each group can apply the same or different simplification operations. For example, before applying the right shift, the samples of each group are filled to the target sample number, and then the same derivation process and the same partition table are applied.

[0390] Fixed-point implementation

[0391] The CCLM design can be reused for the 1-tap case. Division by n can be implemented by right shifting, and division by A2 can be implemented by LUT. Integerization parameters, including n involved in the integerization design of LMS CCLM α 、n A1 、n A2 、r A1 、r A2 n tableAs well as the intermediate parameters used to derive the linear model (Equations (19)-(20)), they can be the same as CCLM or have different values ​​to obtain higher accuracy. The integerization parameters can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and can be adjusted according to the sequence bit depth. For example, n table = bit depth + 4.

[0392] MDLM downsampling

[0393] When GLM is combined with MDLM, the total number of samples available for parameter derivation may not be a power of 2 and needs to be padded to a power of 2 to utilize right shift operations instead of division. For example, for an 8×4 chroma CU, MDLM requires W+H=12 samples, while MDLM_T has only 8 samples available (reconstructed), which can then be equivalently padded with the downsampled 4 samples (0, 2, 4, 6). The code to implement such an operation is as follows:

[0394]

[0395]

[0396] Other filling methods can also be applied, such as repeated / mirrored filling relative to the last adjacent sample point (rightmost / bottommost).

[0397] The filling method used for the GLM can be the same as or different from the filling method of the CCLM.

[0398] Note that in the ECM version, the 8×4 chroma CU MDLM_T / MDLM_L requires 2T / 2L=16 / 8 samples respectively, in which case the same padding method can be applied to meet the target power-of-two sample number.

[0399] Division LUT

[0400] The partitioning LUT proposed for CCLM / LIC (local illumination compensation) in known standard developments such as AVC / HEVC / AV1 / VVC / AVS can be used for GLM division. For example, the LUT in JCTVC-I0166 can be reused with a bit depth of 10 (Table 4). The division LUT can be different from CCLM. For example, CCLM uses min-max with DivTable (as in Equation 5), but GLM uses a 32-entry LMS division LUT (as in Table 5).

[0401] When GLM is combined with MMLM, the meanL value may not always be positive (for example, using filtered values / gradient values ​​to classify groups), so it is necessary to extract sgn(meanL) and use abs(meanL) to find the division LUT. Note that the division LUT used for MMLM classification and parameter derivation can be different. For example, use a lower precision LUT (such as the LUT in min-max) for mean classification and a higher precision LUT (such as in LMS) for parameter derivation.

[0402] Size constraints and latency constraints

[0403] Similar to CCLM design, some size constraints can be applied to ELM / FLM / GLM. For example, the same constraints for luma-chroma delay in dual trees can be applied.

[0404] Size constraints can be based on CU area / width / height / depth. Thresholds can be predefined or signaled at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, for chroma CU area, the predefined threshold can be 128.

[0405] In one example, at least one pre-operation is performed in response to determining that a video block meets an enablement threshold, where the enablement threshold is associated with an area, width, height, or segmentation depth of the video block. Specifically, the enablement threshold may define a minimum or maximum area, width, height, or segmentation depth of the video block. As will be appreciated by those skilled in the art, a video block may include a current chroma block and its co-located luma block. It is also proposed that the enablement threshold be applied jointly to the current chroma block and its co-located luma block. For example, in response to determining that both the current chroma block and its co-located luma block meet the enablement threshold, at least one pre-operation is performed.

[0406] Line buffer reduction

[0407] Similar to the CCLM design, if the co-located luma region of the current chroma CU contains the first row inside a CTU, the top template sample generation can be limited to 1 row to reduce the row buffer storage of CTU rows. Note that when the upper reference row is at the CTU boundary, only one luma row (common row buffer in intra prediction) is used to make the downsampled luma samples.

[0408] For example, in Figure 13In

[15] , if the co-located luminance region of the current chroma CU contains the first row inside a CTU, the top template can be restricted to using only 1 row (instead of 2 rows) for parameter derivation (other CUs can still use 2 rows). This can save luminance sample row buffer storage when processing CTUs row by row at the decoder hardware. Several methods can be used to achieve row buffer reduction. Note that the example of a limited "1" row can be extended to N rows using similar operations. Similarly, 2 taps or multiple taps can also apply such operations. When multiple taps are applied, chroma samples may also need to apply operations.

[0409] For example, Figure 15A The 1-tap filter [1, 0, -1; 1, 0, -1] is shown as an example. This filter can be reduced to [0, 0, 0; 1, 0, -1], i.e., only using the lower row coefficients. Alternatively, the limited upper row luma samples can be padded from the lower row luma samples (repeated, mirrored, 0, meanL, meanC, etc.).

[0410] Take N=4 as an example, that is, the video block is located at the top boundary of the current CTU, and the neighboring luminance sample values ​​and corresponding chrominance sample values ​​of the top 4 rows are used to derive the linear model. Note that the corresponding chrominance sample values ​​may refer to the corresponding neighboring chrominance sample values ​​of the top 4 rows (e.g., for YUV 4:4:4 format). Alternatively, the corresponding chrominance sample values ​​may refer to the corresponding top 2 rows of neighboring chrominance sample values ​​(e.g., for YUV 4:2:0 format). In this case, the neighboring luminance sample values ​​and corresponding chrominance sample values ​​of the top 4 rows can be divided into two regions—a first region, which includes valid sample values ​​(e.g., the luminance sample values ​​and corresponding chrominance sample values ​​of the most recent row); and a second region, which includes invalid sample values ​​(e.g., the other three rows of luminance sample values ​​and corresponding chrominance sample values). The coefficients of the filter corresponding to the sample positions that do not belong to the first region can then be set to zero, so that only the sample values ​​from the first region are used to calculate the sample difference. For example, as discussed above, in this case, the filter [1, 0, -1; 1, 0, -1] can be simplified to [0, 0, 0; 1, 0, -1]. Alternatively, the nearest sample values ​​in the first region can be padded to the second region so that the padded sample values ​​can be used to calculate the sample difference.

[0411] Fusion of chroma intra prediction modes

[0412] In one example, since GLM can be used as a special CCLM mode, the fusion design can be reused or have its own way. Multiple (two or more) weights can be applied to generate the final predictor. For example,

[0413] pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift

[0414] where pred0 is a non-LM based predictor and pred1 is a GLM based predictor, or

[0415] pred0 is a predictor based on one of the CCLMs (including all MDLM / MMLMs), and pred1 is a predictor based on the GLM, or

[0416] pred0 is a GLM-based predictor, and pred1 is a GLM-based predictor.

[0417] Different I / P / B slices can have different weight designs (w0 and w1), depending on whether the neighboring blocks are coded using CCLM / GLM / other coding modes or block size / width / height.

[0418] For example, the weight design can be determined by the intra prediction mode of the adjacent chroma block, and the shift is set to 2. Specifically, when the upper and left adjacent blocks are both coded or decoded using the LM mode, {w0, w1} = {1, 3}; when the upper and left adjacent blocks are both coded or decoded using the non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}. For non-I slices, both w0 and w1 can be set to 2.

[0419] For grammar design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied.

[0420] As described above, GLM has a good gain-complexity trade-off because it can reuse the existing CCLM module without introducing additional derivations. Such a 1-tap design can be further extended or generalized according to one or more aspects of the present disclosure.

[0421] In one aspect of the present disclosure, for a chroma sample to be predicted, a single corresponding luma sample L can be generated by combining the co-located luma sample and the adjacent luma samples. For example, the combination can be a combination of different linear filters, such as a high-pass gradient filter (GLM) and a low-pass smoothing filter (e.g., a [1,2,1; 1,2,1] / 8 FIR downsampling filter that can be commonly used in CCLM); and / or a linear filter and a nonlinear filter (e.g., with a power of n, such as L n , n can be a positive number, a negative number, or a combination of +- fractions (e.g., +1 / 2, square root, or +3, cube, which can be rounded and rescaled to the bit depth dynamic range).

[0422] In one aspect of the present disclosure, the combination can be applied repeatedly. For example, a combination of GLM and [1,2,1; 1,2,1] / 8FIR can be applied on the reconstructed luma samples, and then a nonlinear power of 1 / 2 can be applied. For example, the nonlinear filter can be implemented as a LUT (lookup table), for example, for bit depth = 10, power of n, n = 1 / 2, LUT[i] = (int)(sqrt(i) + 0.5) << 5, i = 0-1023, where 5 will be scaled to the bit depth = 10 dynamic range. When the linear filter cannot efficiently handle the luma-chroma relationship, the nonlinear filter can provide an option. Whether to use the nonlinear term can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0423] In one or more aspects of the present disclosure, GLM may refer to a generalized linear model (which may be used to linearly or nonlinearly generate a single luminance sample, and the generated single luminance sample may be fed into a CCLM linear model to derive parameters of the CCLM linear model), and the linear / nonlinear generation may be referred to as a general pattern. Different gradients or general patterns may be combined to form another pattern. For example, a gradient pattern may be combined with a CCLM downsampled value; a gradient pattern may be combined with a nonlinear L 2 Value combination; a gradient pattern can be combined with another gradient pattern. The two gradient patterns to be combined can have different directions or the same direction. For example, [1,1,1; -1, -1, -1] and [1,2,1; -1, -2, -1] can be combined, both of which have vertical directions. You can also combine [1,1,1; -1, -1, -1] and [1,0, -1; 1,0, -1], both of which have vertical and horizontal directions, such as 15A to 15D The combination may include additive, subtractive or linear weighting.

[0424] GLM applied to the downsampled domain

[0425] As described above, pre-operation can be repeatedly applied, and GLM can be applied to the pre-linear weighted / pre-operated samples. For example, as CCLM, a template filter can be applied to the luma sample to remove outliers using a low-pass smoothing FIR filter [1, 2, 1; 1, 2, 1] / 8 (i.e., CCLM downsampling smoothing filter) and generate downsampled luma samples (one downsampled luma sample corresponding to one chroma sample). And thereafter, a 1-tap GLM can be applied to the smoothed downsampled luma samples to derive the MLR model.

[0426] Some gradient filter modes (such as 3×3 Sobel or Prewitt operator) can be applied to the downsampled luma samples. The following table shows some of the gradient filter modes.

[0427]

[0428] The gradient filter pattern can be combined with other gradient / generic filter patterns in the downsampled luma domain. In one example, the combined filter pattern can be applied to the downsampled luma samples. For example, the combined filter pattern can be derived by performing addition or subtraction operations on the corresponding coefficients of the gradient filter pattern and a DC / lowpass based filter pattern such as the filter pattern [0,0,0; 0,1,0; 0,0,0] or [1,2,1; 2,4,1; 1,2,1]. In another example, the combined filter pattern can be derived by performing addition or subtraction operations on the coefficients of the gradient filter pattern and a nonlinear value such as L 2 ) to derive the combined filter pattern. In another example, the combined filter pattern is derived by performing an addition or subtraction operation on the corresponding coefficients of the gradient filter pattern and another gradient filter pattern having a different direction or the same direction. In another example, the combined filter pattern is derived by performing a linear weighting operation on the coefficients of the gradient filter pattern.

[0429] A GLM applied in the downsampled domain can fit into the CCCM framework but may sacrifice high frequency accuracy because low-pass smoothing is applied before applying the GLM.

[0430] GLM as input to CCCM

[0431] As shown above, CCCM, like CCLM, applies luma downsampling before convolution. When chroma subsampling is used, the reconstructed luma samples are downsampled to match the lower resolution chroma grid. Since a 1-tap GLM can also be viewed as changing the CCLM downsampling filter coefficients (e.g., from [1,2,1; 1,2,1] / 8 to [1,2,1,-1,-2,-1], i.e., from low pass to high pass), the GLM can be used as input to the CCCM. Specifically, the GLM's gradient filter replaces the luma downsampling filter ([1,2,1; 1,2,1] / 8) with gradient-based coefficients (e.g., [1,2,1,-1,-2,-1]). In this case, the CCCM operation becomes a "linear / nonlinear combination of gradients" as shown below:

[0432] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0433] Where C, N, S, E, W, and P are the gradients of the current or neighboring sample points (compared to the original downsampled values ​​used for CCCM). The related GLM methods described in this disclosure can be applied in the same way before entering the CCCM convolution, such as classification, separate Cb / Cr control, syntax, mode combination, PU size limit, etc.

[0434] Gradient-based coefficient replacement can be applied to specific CCCM taps. In addition, not only high-pass, but also low-pass / band-pass / all-pass coefficient replacement can be used. Replacement can be combined with the FLM / CCCM shape switching discussed above (resulting in different tap numbers). For example, 15A to 15D The gradient mode in can be used for replacement. In one example, the operation for applying GLM to the input of CCCM includes: predefining one or more coefficient candidates for CCCM / FLM downsampling; determining the CCCM / FLM filter shape and number of filter taps for the CU; applying different CCLM downsampling coefficients to different filter taps, where the coefficients can be a high-pass filter (GLM) or a low-pass / band-pass / all-pass filter; generating downsampled luma samples for CCCM input samples (using the applied coefficients); and inputting the generated downsampled luma samples into the CCCM process.

[0435] Some examples of changing the CCLM downsampling filter coefficients are shown below:

[0436] Example 1:

[0437] The candidate filters are [1,2,1; 1,2,1] / 8 and [1,0,-1; 1,0,-1]; predChromaVal = c0C+c1N+c2S+c3E+c4W+c5P+c6B, using the typical CCCM cross shape, 7 taps; N, S, W, E use the filter [1,2,1; 1,2,1] / 8, retaining the original CCLM downsampling filter; and C, P use the filter [1,0,-1; 1,0,-1], that is, the horizontal gradient filter, and P physically represents the gradient^2.

[0438] Example 2:

[0439] The candidate filters are [1,2,1; 1,2,1] / 8, [1,0,-1; 1,0,-1], [1,2,1; -1,-2,-1], [2,1,-1; 1,-1,-2], and [-1,1,2; -2,-1,1]; predChromaVal = c0C0+c1C1+c2C2+c3C3+c4C4+c5P+c6B; C0 uses the filter [1,2,1; 1,2,1] / 8, retaining the original CCLM downsampling filter; C1 uses the filter [1,0,-1 ; 1,0,-1]; C2 uses filter [1,2,1; -1,-2,-1]; C3 uses filter [2,1,-1; 1,-1,-2]; C4 uses filter [-1,1,2; -2,-1,1]; C5 uses filter [1,2,1; 1,2,1] / 8, retaining the original CCLM downsampling filter; C0 to C5, and P have the same downsampled luminance position (in the typical CCCM cross shape == C); and C1 to C4 are generated by Sobel-based gradient filters in different directions (such as Figures 15A to 15D shown).

[0440] Example 3:

[0441] The candidate filters are [1,2,1; 1,2,1] / 8, [1,0,-1; 1,0,-1], [1,2,1; -1,-2,-1], [0,1,1; 0,1,1], and [1,1,0; 1,1,0]; predChromaVal = c0C0+c1C1+c2C2+c3C3+c4C4+c5P+c6B; C0 uses the filter [1,2,1; 1,2,1] / 8, retaining the original CCLM downsampling filter; C1 uses the filter [1,0,- 1; 1,0,-1]; C2 uses filter [1,2,1; -1,-2,-1]; C3 uses filter [0,1,1; 0,1,1]; C4 uses filter [1,1,0; 1,1,0]; C5 uses filter [1,2,1; 1,2,1] / 8, retaining the original CCLM downsampling filter; C0 to C5, and P have the same downsampled luminance position (in the typical CCCM cross shape == C); C1 to C2 are generated by Sobel-based gradient filters in different directions (such as Figures 15A to 15D and C3 to C4 are generated by a low-pass filter.

[0442] Which CCCM / FLM taps apply coefficient replacement can be predefined at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level (as shown in the example above) or signaled / switched.

[0443] For each CCCM / FLM tap, coefficient candidates for CCCM / FLM downsampling may be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0444] Example 4:

[0445] The candidate filters are [1,2,1; 1,2,1] / 8, [1,0,-1; 1,0,-1], [1,2,1; -1,-2,-1], [2,1,-1; 1,-1,-2], and [-1,1,2; -2,-1,1]; predChromaVal = c0C+c1N+c2S+c3E+c4W+c5P+c6B, using the typical CCCM cross-shape, 7 taps; C uses a switching downsampling filter between the 5 candidate filters; N, S, W, E use the filter [1, 2, 1; 1, 2, 1] / 8, retaining the original CCLM downsampling filter; and P uses a switching downsampling filter between the 5 candidate filters.

[0446] Example 5:

[0447] The candidate filters are: [1,2,1; 1,2,1] / 8, [1,0,-1; 1,0,-1], [1,2,1; -1,-2,-1], [0,1,1; 0,1,1] and [1,1,0; 1,1,0]; predChromaVal = c0C+c1W+c2E+c3P+c4B, that is, a horizontal minus shape, 5 taps; C uses a switching downsampling filter between the 5 candidate filters; W and E use a switching downsampling filter between the 3 candidate filters: [1,2,1; 1,2,1] / 8, [0,1,1; 0,1,1] and [1,1,0; 1,1,0]; and P uses the filter [1, 2, 1; 1, 2, 1] / 8, retaining the original CCLM downsampling filter.

[0448] In one or more aspects of the present disclosure, one or more syntaxes may be introduced to indicate information about GLM. Table 10 below shows an example of GLM syntax.

[0449] Table 10

[0450]

[0451]

[0452] FLC: Fixed Length Code

[0453] TU: Truncated Unary Code

[0454] EGk: Exponential Golomb code with order k, where k can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0455] SVLC: Signed EG0

[0456] UVLC: Unsigned EG0

[0457] Note that the binarization of each syntax element can be changed.

[0458] In one aspect of the present disclosure, the GLM on / off control for Cb / Cr components can be performed jointly. For example, at the CU level, 1 flag can be used to indicate whether GLM is valid for this CU. If valid, 1 flag can be used to indicate whether both Cb / Cr are valid. If neither is valid, 1 flag indicates that either Cb or Cr is valid. When Cb and / or Cr are valid, the filter index / gradient (general) mode can be signaled separately. All flags can have their own context model or be bypass-coded.

[0459] In another aspect of the present disclosure, whether to signal the GLM on / off flag can depend on the luminance / chrominance coding mode and / or CU size. For example, in the ECM5 chrominance intra-mode syntax, when applying MMLM or MMLM_L or MMLM_T, GLM can be inferred to be off; when the CU area < A, where A can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level; if combined with CCCM, when CCCM is on, GLM can be inferred to be off.

[0460] Note that when GLM is combined with MMLM, different models can share the same gradient / general mode or have their own gradient / general mode.

[0461]

[0462] When GLM is combined with CCCM / FLM, if the current CU is enabled for CCCM / FLM, the CU-level GLM enable flag can be inferred to be off.

[0463] hasGlmFlag &=!pu.cccmFlag;

[0464] CCCM without downsampling

[0465] CCCM requires processing of a downsampled luma reference value before calculating model parameters and applying the CCCM model, which increases the burden on the decoder processing cycle. In this section, CCCM without downsampling is proposed, including: using an undownsampled luma reference value and / or the selection of different undownsampled luma reference values. One or more filter shapes can be used for the purposes described below.

[0466] Please note that the methods / examples in this section can be combined / reused with the methods mentioned above, including but not limited to methods related to classification, filter shape, matrix derivation (with special processing), application area, and syntax. In addition, the methods / examples listed in this section can also be applied together with the above methods / examples (more taps) to achieve better performance with certain complexity trade-offs.

[0467] In this disclosure, reference samples / training templates / reconstructed neighborhoods generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0468] Filter shape

[0469] One or more shapes / numbers of filter taps can be used for CCCM prediction, such as Figure 16 、 Figure 17 and 18A to 18B One or more sets of filter taps can be used for FLM prediction, for example in Figures 19A to 19G The selected luma reference value is not downsampled. One or more predefined shapes / numbers of filter taps may be used for CCCM prediction based on previously decoded information on TB / CB / slice / picture / sequence level.

[0470] Although multi-tap filters can fit the training data well (i.e., the reconstructed luma / chroma samples adjacent to the top / left), this may lead to overfitting in some cases where the training data does not capture the complete characteristics of the test data and cannot predict the test data well (i.e., the chroma block samples to be predicted). In addition, different filter shapes can adapt well to different video block contents, resulting in more accurate predictions. To address this issue, the filter shape / number of filter taps can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of filter shape candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) can have different filter switching controls. For example, as shown in the following table, a set of filter shape candidates (index = 0 to 5) are predefined, and the filter shape (1, 2) represents a 2-tap luminance filter, while the filter shape (1, 2, 4) represents a 3-tap luminance filter, etc. Figure 11 As shown in . The filter shape selection for U / V components can be switched in PH or CU / CTU level. Note that N taps can represent N taps with or without offset β as described above.

[0471]

[0472] Different chroma types / color formats can have different predefined filter shapes / taps. For example, for 420 type-0: (1,2,4,5), 420 type-2: (0,1,2,4,7), 422: (1,4), 444: (0,1,2,3,4,5) use predefined filter shapes, such as Figure 12 As shown in .

[0473] Luma / chroma samples that are not available for deriving the MLR model can be filled in from available reconstructed samples. Figure 12 In the 6-tap (0,1,2,3,4,5) filter, for the CU located at the left picture boundary, the left column including (0,3) is unavailable (beyond the picture boundary), so (0,3) is repeated padding from (1,4) to apply the 6-tap filter. Note that the padding process is applied to both the training data (the reconstructed luma / chroma samples adjacent to the top / left) and the test data (the luma / chroma samples in the CU).

[0474] According to one or more embodiments of the present disclosure, unavailable luma / chroma samples for deriving the MLR model may be skipped and not used. Thus, no padding process is required for unavailable luma / chroma samples.

[0475] CCLM / MMLM with LDL decomposition

[0476] CCCM requires processing of LDL decomposition to calculate the model parameters of the CCCM model, thereby avoiding the use of square root operations and requiring only integer operations. In this section, CCLM / MMLM with LDL decomposition is proposed. As mentioned above, LDL decomposition can also be used in ELM / FLM / GLM.

[0477] Please note that the methods / examples in this section can be combined / reused with the methods mentioned above, including but not limited to methods related to classification, filter shape, matrix derivation (with special processing), application area, and syntax. In addition, the methods / examples listed in this section can also be applied together with the above methods / examples to achieve better performance with certain complexity trade-offs.

[0478] In this disclosure, reference samples / training templates / reconstructed neighborhoods generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0479] CCLM / MMLM with extended range

[0480] One or more reference samples can be used for CCLM / MMLM prediction, i.e. Figure 10B As shown in , the reference region can be the same as the reference region in CCCM. Different reference regions can be used for CCLM / MMLM prediction based on previously decoded information at TB / CB / slice / picture / sequence level.

[0481] Although training data with multiple reference regions can be a good fit for the calculation of model parameters, in some cases where the training data does not capture the full characteristics of the test data, this may lead to overfitting and poor prediction of the test data (i.e., the chroma block samples to be predicted). In addition, different reference regions can be well adapted to different video block contents, resulting in more accurate predictions. To address this issue, the number of reference shapes / reference regions can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. A set of reference region candidates can be predefined or signaled / switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Different components (U / V) can have different reference region switching controls. For example, a set of candidate reference regions (index = 0 to 4) is predefined as shown in the table below. The reference region selection for the U / V components can be switched at the PH or CU / CTU level. Different colorimetric types / color formats may have different predefined reference areas.

[0482]

[0483] Luma / chroma samples that are unavailable for deriving the MLR model can be filled in from the available reconstructed samples. The filling process is applied to both the training data (top / left neighboring reconstructed luma / chroma samples) and the test data (luma / chroma samples in the CU).

[0484] According to one or more embodiments of the present disclosure, unavailable luma / chroma samples for deriving the MLR model may be skipped and not used. Thus, no padding process is required for unavailable luma / chroma samples.

[0485] FLM / GLM / ELM / CCCM with minimum sample point restriction

[0486] FLM needs to process the downsampled luma reference value and calculate the model parameters, which increases the burden on the decoder processing cycle, especially for small blocks. In this section, FLM with minimum sample restrictions is proposed, for example, FLM is only used for samples larger than a predefined number (such as 64, 128). One or more different restrictions can be used for this purpose, for example, FLM is only used for samples larger than a predefined number (such as 256) in a single model, and FLM is only used for samples larger than a predefined number (such as 128) in a multi-model.

[0487] According to one or more embodiments of the present disclosure, the number of predefined minimum samples for a single model may be greater than or equal to the number of predefined minimum samples for a multi-model. For example, FLM / GLM / ELM / CCCM is only used for samples greater than or equal to a predefined number (such as 128) in a single model, and FLM / GLM / ELM / CCCM is only used for samples greater than or equal to a predefined number (such as 256) in a multi-model.

[0488] According to one or more embodiments of the present disclosure, the number of predefined minimum samples for FLM / GLM / ELM may be greater than or equal to the number of predefined minimum samples for CCCM. For example, CCCM may be used only in a single mode for a number of samples greater than or equal to a predefined number (such as 0), and CCCM may be used only in a multi-mode for a number of samples greater than or equal to a predefined number (such as 128). FLM may be used only in a single mode for a number of samples greater than or equal to a predefined number (such as 128), and FLM may be used only in a multi-mode for a number of samples greater than or equal to a predefined number (such as 256).

[0489] Please note that the methods / examples in this section can be combined / reused with the methods mentioned above, including but not limited to methods related to classification, filter shape, matrix derivation (with special processing), application area, and syntax. In addition, the methods / examples listed in this section can also be applied together with the above methods / examples (more taps) to achieve better performance with certain complexity trade-offs.

[0490] Figure 20 A workflow of a method 2000 for decoding video data according to one or more aspects of the present invention is shown.

[0491] At step 2010 , method 2000 includes obtaining a video block from a bitstream.

[0492] At step 2020 , the method 2000 includes obtaining an internal luma sample value of the video block, an external luma sample value of an external region of the video block, and an external chroma sample value of an external region.

[0493] At step 2030, method 2000 includes selecting a filter shape candidate from a plurality of filter shape candidates for predicting an internal chroma sample value of a video block based on an external luma sample value and an external chroma sample value, wherein the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting chroma sample values ​​based on corresponding luma sample values.

[0494] At step 2040 , the method 2000 includes predicting intra chroma sample values ​​based on the intra luma sample values ​​using the selected filter shape candidate.

[0495] At step 2050 , method 2000 includes obtaining a decoded video block using the predicted intra chroma sample values.

[0496] In one example, the method 2000 further includes determining a plurality of filter shape candidates for predicting intra chroma sample values.

[0497] In one example, the plurality of filter shape candidates include at least one gradient filter candidate that enables calculation of sample differences between luma sample values.

[0498] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting an inner chroma sample value includes: applying each of the weighting coefficient sets for the plurality of filter shape candidates to an outer region, respectively, to predict a chroma sample value in the outer region based on a corresponding luma sample value for the chroma sample value in the outer region; accumulating, for each of the plurality of filter shape candidates, an error between the predicted chroma sample value in the outer region and the corresponding outer chroma sample value; and selecting, based on the accumulated error, a filter shape candidate from the plurality of filter shape candidates for predicting the inner chroma sample value.

[0499] In one example, method 2000 further includes applying each of the plurality of filter shape candidates to the external area, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on the external luminance sample value and the corresponding external chrominance sample value in the external area.

[0500] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting an internal chroma sample value includes: dividing an external area into two or more parts; applying each of the plurality of filter shape candidates to a first part of the external area, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on external luma sample values ​​and corresponding external chroma sample values ​​in the first part of the external area; applying each of the derived sets of weighting coefficients for the plurality of filter shape candidates to a second part of the external area, respectively, to predict chroma sample values ​​in the second part of the external area based on corresponding luma sample values ​​for chroma sample values ​​in the second part of the external area; accumulating an error between the predicted chroma sample values ​​in the second part of the external area and the corresponding external chroma sample values ​​for each of the plurality of filter shape candidates; and selecting a filter shape candidate from the plurality of filter shape candidates for predicting the internal chroma sample value based on the accumulated error.

[0501] In one example, the first portion of the outer region includes the even rows or even columns of the outer region, and the second portion of the outer region includes the remaining odd rows or odd columns of the outer region.

[0502] In one example, the first portion of the outer region and the second portion of the outer region are alternating portions of the outer region.

[0503] In one example, accumulating the errors between the predicted chroma sample values ​​and the corresponding extrinsic chroma sample values ​​includes accumulating the errors via sum of absolute differences (SAD), sum of squared differences (SSD), or sum of absolute transformed differences (SATD).

[0504] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting intra chroma sample values ​​based on the accumulated error includes selecting a filter shape candidate with a minimum accumulated error from the plurality of filter shape candidates for predicting intra chroma sample values.

[0505] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting an intra chroma sample value based on an accumulated error includes: sorting the plurality of filter shape candidates and selecting two or more filter shape candidates with minimum errors from the plurality of filter shape candidates; and selecting a filter shape candidate from the two or more filter shape candidates based on a received signal indicating the applied filter shape candidate for predicting the intra chroma sample value.

[0506] Figure 21 A workflow of a method for encoding video data according to one or more aspects of the present disclosure is shown.

[0507] At step 2110 , method 2100 includes obtaining a video block.

[0508] At step 2120 , the method 2100 includes obtaining an internal luma sample value of the video block, an external luma sample value of an external region of the video block, and an external chroma sample value of an external region.

[0509] At step 2130, method 2100 includes selecting a filter shape candidate from a plurality of filter shape candidates for predicting an internal chroma sample value of a video block based on an external luma sample value and an external chroma sample value, wherein the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting chroma sample values ​​based on corresponding luma sample values.

[0510] At step 2140 , the method 2100 includes predicting intra chroma sample values ​​based on the intra luma sample values ​​using the selected filter shape candidate.

[0511] At step 2150 , the method 2100 includes generating a bitstream including the encoded video block by using the predicted intra chroma sample values.

[0512] In one example, the method 2100 further includes determining a plurality of filter shape candidates for predicting intra chroma sample values.

[0513] In one example, the plurality of filter shape candidates include at least one gradient filter candidate that enables calculation of sample differences between luma sample values.

[0514] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting an inner chroma sample value includes: applying each of the weighting coefficient sets for the plurality of filter shape candidates to an outer region, respectively, to predict a chroma sample value in the outer region based on a corresponding luma sample value for the chroma sample value in the outer region; accumulating, for each of the plurality of filter shape candidates, an error between the predicted chroma sample value in the outer region and the corresponding outer chroma sample value; and selecting, based on the accumulated error, a filter shape candidate from the plurality of filter shape candidates for predicting the inner chroma sample value.

[0515] In one example, method 2100 further includes applying each of the multiple filter shape candidates to the external area, respectively, to derive a set of weighting coefficients corresponding to each of the multiple filter shape candidates based on the external luminance sample value and the corresponding external chrominance sample value in the external area.

[0516] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting an internal chroma sample value includes: dividing an external area into two or more parts; applying each of the plurality of filter shape candidates to a first part of the external area, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on external luma sample values ​​and corresponding external chroma sample values ​​in the first part of the external area; applying each of the derived sets of weighting coefficients for the plurality of filter shape candidates to a second part of the external area, respectively, to predict chroma sample values ​​in the second part of the external area based on corresponding luma sample values ​​for chroma sample values ​​in the second part of the external area; accumulating an error between the predicted chroma sample values ​​in the second part of the external area and the corresponding external chroma sample values ​​for each of the plurality of filter shape candidates; and selecting a filter shape candidate from the plurality of filter shape candidates for predicting the internal chroma sample value based on the accumulated error.

[0517] In one example, the first portion of the outer region includes the even rows or even columns of the outer region, and the second portion of the outer region includes the remaining odd rows or odd columns of the outer region.

[0518] In one example, the first portion of the outer region and the second portion of the outer region are alternating portions of the outer region.

[0519] In one example, accumulating the errors between the predicted chroma sample values ​​and the corresponding extrinsic chroma sample values ​​includes accumulating the errors via sum of absolute differences (SAD), sum of squared differences (SSD), or sum of absolute transformed differences (SATD).

[0520] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting intra chroma sample values ​​based on the accumulated error includes selecting a filter shape candidate with a minimum accumulated error from the plurality of filter shape candidates for predicting intra chroma sample values.

[0521] In one example, selecting a filter shape candidate from a plurality of filter shape candidates for predicting an intra chroma sample value based on an accumulated error includes: sorting the plurality of filter shape candidates and selecting two or more filter shape candidates with minimum errors from the plurality of filter shape candidates; and selecting the filter shape candidate from the two or more filter shape candidates for predicting the intra chroma sample value.

[0522] In one example, the method 2100 further includes signaling, in the bitstream, a syntax element indicating the selected filter shape candidate.

[0523] Figure 22 An exemplary computing system 2200 according to one or more aspects of the present disclosure is shown. Computing system 2200 may include at least one processor 2210. Computing system 2200 may also include at least one storage device 2220. Storage device 2220 may store computer-executable instructions that, when executed, cause processor 2210 to perform the steps of the method described above. Processor 2210 may be a general-purpose processor or may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Storage device 2220 may store input data, output data, data generated by processor 2210, and / or instructions executed by processor 2210.

[0524] It should be understood that the storage device 2220 may store computer-executable instructions that, when executed, cause the processor 2210 to perform any operations according to the embodiments of the present disclosure.

[0525] Embodiments of the present disclosure may be embodied as computer-readable media, such as non-transitory computer-readable media. The non-transitory computer-readable media may include instructions that, when executed, cause one or more processors to perform any operations according to the embodiments of the present disclosure. For example, when executed, the instructions may cause one or more processors to receive a bitstream and perform a decoding operation as described above. For another example, when executed, the instructions may cause one or more processors to perform an encoding operation and transmit a bitstream including the encoded video information associated with the predicted chroma samples.

[0526] It should be understood that all operations in the method described above are merely exemplary, and the present disclosure is not limited to any operations in the method or the order of these operations, but should cover all other equivalent transformations under the same or similar concept.

[0527] It should also be understood that all modules in the above-described method can be implemented in various ways. These modules can be implemented as hardware, software or a combination thereof. In addition, any module in these modules can be further divided into submodules or combined together functionally.

[0528] The above description is provided so that anyone skilled in the art can implement the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structures and functions that are equivalent to the elements throughout the various aspects described in this disclosure that are known or that will become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be included in the claims.

Claims

1. A method for decoding video data, comprising: Get video chunks from the bitstream; Obtaining an internal luminance sample value of the video block, an external luminance sample value of an external area of ​​the video block, and an external chrominance sample value of the external area; selecting, based on the outer luma sample values ​​and the outer chroma sample values, a filter shape candidate from a plurality of filter shape candidates for predicting inner chroma sample values ​​of the video block, wherein the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting chroma sample values ​​based on corresponding luma sample values; predicting the intra chroma sample values ​​using the selected filter shape candidate based on the intra luma sample values; and The predicted intra chroma sample values ​​are used to obtain a decoded video block.

2. The method according to claim 1, further comprising: The plurality of filter shape candidates for predicting the intra chroma sample values ​​are determined.

3. The method according to claim 1, wherein: The plurality of filter shape candidates include at least one gradient filter candidate that enables calculation of sample differences between luma sample values.

4. The method according to claim 1, wherein: Selecting a filter shape candidate from a plurality of filter shape candidates for predicting the intra chroma sample value comprises: applying each of the sets of weighting coefficients for the plurality of filter shape candidates to the outer region, respectively, to predict the chroma sample values ​​in the outer region based on corresponding luma sample values ​​for the chroma sample values ​​in the outer region; accumulating, for each of the plurality of filter shape candidates, an error between the predicted chroma sample value in the outer region and a corresponding outer chroma sample value; and A filter shape candidate is selected from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error.

5. The method according to claim 4, further comprising: Each of the plurality of filter shape candidates is applied to the external region, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on external luma sample values ​​and corresponding external chroma sample values ​​in the external region.

6. The method according to claim 1, wherein: Selecting a filter shape candidate from a plurality of filter shape candidates for predicting the intra chroma sample value comprises: dividing the outer area into two or more parts; applying each of the plurality of filter shape candidates to a first portion of the external region, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on external luma sample values ​​and corresponding external chroma sample values ​​in the first portion of the external region; respectively applying each of the derived sets of weighting coefficients for the plurality of filter shape candidates to a second portion of the outer region to predict chroma sample values ​​in the second portion of the outer region based on corresponding luma sample values ​​for the chroma sample values ​​in the second portion of the outer region; accumulating, for each filter shape candidate of the plurality of filter shape candidates, errors between the predicted chroma sample values ​​in the second portion of the outer region and corresponding outer chroma sample values; A filter shape candidate is selected from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error.

7. The method according to claim 6, wherein: The first portion of the outer area includes even-numbered rows or even-numbered columns of the outer area, and the second portion of the outer area includes remaining odd-numbered rows or odd-numbered columns of the outer area.

8. The method according to claim 6, wherein: The first portion of the outer region and the second portion of the outer region are alternating portions of the outer region.

9. The method according to claim 4 or claim 6, wherein: The accumulated error between the predicted chroma sample value and the corresponding external chroma sample value comprises: The errors are accumulated via the sum of absolute differences (SAD), the sum of squared differences (SSD), or the sum of absolute transformed differences (SATD).

10. The method according to claim 4 or claim 6, wherein: Selecting a filter shape candidate from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error comprises: A filter shape candidate having a minimum accumulated error is selected from the plurality of filter shape candidates for predicting the intra chroma sample value.

11. The method according to claim 4 or claim 6, wherein: Selecting a filter shape candidate from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error comprises: sorting the plurality of filter shape candidates and selecting two or more filter shape candidates having minimum errors from the plurality of filter shape candidates; and A filter shape candidate is selected from the two or more filter shape candidates based on a received signal indicative of an applied filter shape candidate for predicting the intra chroma sample value.

12. A method for encoding video data, comprising: Get the video block; Obtaining an internal luminance sample value of the video block, an external luminance sample value of an external area of ​​the video block, and an external chrominance sample value of the external area; selecting, based on the outer luma sample values ​​and the outer chroma sample values, a filter shape candidate from a plurality of filter shape candidates for predicting inner chroma sample values ​​of the video block, wherein the plurality of filter shape candidates correspond to different sets of weighting coefficients for predicting chroma sample values ​​based on corresponding luma sample values; predicting the intra chroma sample values ​​using the selected filter shape candidate based on the intra luma sample values; and A bitstream including the encoded video block is generated by using the predicted intra chroma sample values.

13. The method according to claim 12, further comprising: The plurality of filter shape candidates for predicting the intra chroma sample values ​​are determined.

14. The method according to claim 12, wherein: The plurality of filter shape candidates include at least one gradient filter candidate that enables calculation of sample differences between luma sample values.

15. The method according to claim 12, wherein: Selecting a filter shape candidate from a plurality of filter shape candidates for predicting the intra chroma sample value comprises: applying each of the sets of weighting coefficients for the plurality of filter shape candidates to the outer region, respectively, to predict the chroma sample values ​​in the outer region based on corresponding luma sample values ​​for the chroma sample values ​​in the outer region; accumulating, for each of the plurality of filter shape candidates, an error between the predicted chroma sample value in the outer region and a corresponding outer chroma sample value; and A filter shape candidate is selected from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error.

16. The method according to claim 15, further comprising: Each of the plurality of filter shape candidates is applied to the external region, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on external luma sample values ​​and corresponding external chroma sample values ​​in the external region.

17. The method according to claim 12, wherein: Selecting a filter shape candidate from a plurality of filter shape candidates for predicting the intra chroma sample value comprises: dividing the outer area into two or more parts; applying each of the plurality of filter shape candidates to a first portion of the external region, respectively, to derive a set of weighting coefficients corresponding to each of the plurality of filter shape candidates based on external luma sample values ​​and corresponding external chroma sample values ​​in the first portion of the external region; respectively applying each of the derived sets of weighting coefficients for the plurality of filter shape candidates to a second portion of the outer region to predict chroma sample values ​​in the second portion of the outer region based on corresponding luma sample values ​​for the chroma sample values ​​in the second portion of the outer region; accumulating, for each filter shape candidate of the plurality of filter shape candidates, errors between the predicted chroma sample values ​​in the second portion of the outer region and corresponding outer chroma sample values; A filter shape candidate is selected from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error.

18. The method according to claim 17, wherein: The first portion of the outer area includes even-numbered rows or even-numbered columns of the outer area, and the second portion of the outer area includes remaining odd-numbered rows or odd-numbered columns of the outer area.

19. The method according to claim 17, wherein: The first portion of the outer region and the second portion of the outer region are alternating portions of the outer region.

20. The method of claim 15 or claim 17, wherein: The accumulated error between the predicted chroma sample value and the corresponding external chroma sample value comprises: The errors are accumulated via the sum of absolute differences (SAD), the sum of squared differences (SSD), or the sum of absolute transformed differences (SATD).

21. The method of claim 15 or claim 17, wherein: Selecting a filter shape candidate from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error comprises: A filter shape candidate having a minimum accumulated error is selected from the plurality of filter shape candidates for predicting the intra chroma sample value.

22. The method of claim 15 or claim 17, wherein: Selecting a filter shape candidate from the plurality of filter shape candidates for predicting the intra chroma sample value based on the accumulated error comprises: sorting the plurality of filter shape candidates and selecting two or more filter shape candidates having minimum errors from the plurality of filter shape candidates; and A filter shape candidate is selected from the two or more filter shape candidates for predicting the intra chroma sample value.

23. The method according to claim 22, further comprising: A syntax element indicating the selected filter shape candidate is signaled in the bitstream.

24. A computer system comprising: one or more processors; as well as One or more storage devices storing computer-executable instructions which, when executed, cause the one or more processors to perform the operations of the method according to any one of claims 1-23.

25. A computer program product storing computer executable instructions which, when executed, cause one or more processors to perform the operations of the method according to any one of claims 1-23.

26. A computer-readable medium storing computer-executable instructions which, when executed, cause one or more processors to receive a bitstream and perform the operations of the method of any one of claims 1-11 based on the bitstream.

27. A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method of any one of claims 12-23 and send a bitstream including encoded video information associated with the predicted chroma samples.

28. A computer readable medium storing a bit stream, wherein: The bit stream is to be decoded by performing the operations of the method according to any one of claims 1-11.

29. A computer readable medium storing a bit stream, wherein: The bit stream is obtained by performing the operations of the method according to any one of claims 12-23.

Images