Method and apparatus for cross-component prediction for video coding

By using pre-operation and multivariate linear regression models in video blocks, combining filters and classifier methods, cross-component prediction technology is improved, the problem of insufficient encoding and decoding efficiency in the existing technology is solved, and more efficient video data is achieved Compression and decoding effects.

CN118923110APending Publication Date: 2024-11-08BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380029424.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-13
Filing Date
2023-03-23
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing video encoding and decoding technology has shortcomings in improving the encoding and decoding efficiency, especially in cross-component linear model prediction design, which is difficult to effectively reduce redundancy and improve the encoding efficiency.

Method used

An improved cross-component prediction method is proposed to predict chroma sample value by using pre-operation and linear models in video blocks, including calculating sample spreads and applying multivariate linear regression models, combining filters and classifiers to improve predictions Accuracy, and further optimize the codec process through scaling adjustment and edge classification model.

Benefits of technology

It significantly improves the efficiency and accuracy of video encoding and decoding, reduces the bit rate, and avoids quality reduction, achieving more efficient video data compression.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118923110A_ABST
    Figure CN118923110A_ABST
Patent Text Reader

Abstract

The present disclosure provides a method for decoding video data. The method comprises: obtaining a video block from a bitstream; obtaining adjacent brightness sample point values and adjacent chrominance sample point values of the video blocks; performing at least one pre-operation on the adjacent luminance sample value and an internal luminance sample value in the video block to obtain an adjacent luminance sample value and an internal luminance sample value of the pre-operation, where performing the at least one pre-operation includes calculating a sample difference based on the adjacent luminance sample value and the internal luminance sample value; deriving a linear model by using the pre-operated adjacent luma sample values and adjacent chroma sample values; predicting each of the internal chroma sample values in the video block by applying a linear model to one or more corresponding pre-operated internal luma sample values for the internal chroma sample values; and obtaining a decoded video block using the predicted internal chroma sample values.
Need to check novelty before this filing date? Find Prior Art

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 323,010, filed on March 23, 2022, U.S. Provisional Application No. 63 / 329,323, filed on April 8, 2022, and U.S. Provisional Application No. 63 / 330,604, filed on April 13, 2022. The entire contents of these applications are incorporated herein by reference in their entirety. Technical Field

[0002] Generally speaking, aspects of the present disclosure relate to image / video coding and compression, and more particularly, to methods and apparatus for cross-component prediction techniques. Background Art

[0003] Various video coding techniques can be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include Versatile Video Codec (VVC), High Efficiency Video Codec (H.265 / HEVC), Advanced Video Codec (H.264 / AVC), Moving Picture Experts Group (MPEG) codec, etc. Video coding usually adopts a prediction method (e.g., inter-frame prediction, intra-frame prediction, etc.) that makes full use of the redundancy existing in video images or sequences. An important goal of video coding technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality. Summary of the invention

[0004] A brief summary of one or more aspects of the present disclosure is provided below in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all expected aspects, and is neither intended to identify the key or important elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to provide some concepts of one or more aspects in a simplified form as a prelude to a more detailed description provided later.

[0005] According to one aspect of the present disclosure, a method for decoding video data is provided, comprising: obtaining a video block from a bitstream; obtaining adjacent luma sample values ​​and adjacent chroma sample values ​​of the video block; performing at least one pre-operation on the adjacent luma sample values ​​and the intra luma sample values ​​in the video block to obtain pre-operated adjacent luma sample values ​​and the intra luma sample values, wherein performing the at least one pre-operation comprises calculating sample differences based on the adjacent luma sample values ​​and the intra luma sample values; deriving a linear model by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values; predicting each of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luminance sample values ​​for the intra chroma sample value; and obtaining a decoded video block using the predicted intra chroma sample values.

[0006] According to another aspect of the present disclosure, a method for encoding video data is provided, comprising: obtaining a video block from a video frame; obtaining neighboring luma sample values ​​and neighboring chroma sample values ​​of the video block; performing at least one pre-operation on the neighboring luma sample values ​​and intra luma sample values ​​in the video block to obtain pre-operated neighboring luma sample values ​​and intra luma sample values, wherein performing the at least one pre-operation comprises calculating sample differences based on the neighboring luma sample values ​​and the intra luma sample values; deriving a linear model by using the pre-operated neighboring luma sample values ​​and the neighboring chroma sample values; predicting each of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra chroma sample values ​​for the intra chroma sample value; and generating a bitstream including the encoded video block using the predicted intra chroma sample values.

[0007] According to an embodiment, a computer system is provided, including: one or more processors; and one or more storage devices storing computer executable instructions, which, when executed, cause the one or more processors to perform the operations of the method of the present disclosure.

[0008] According to an embodiment, a computer program product is provided that stores computer executable instructions that, when executed, cause one or more processors to perform the operations of the method of the present disclosure.

[0009] According to an embodiment, a computer-readable medium is provided that stores computer-executable instructions that, when executed, cause one or more processors to receive a bitstream and perform the operations of the method of the present disclosure based on the bitstream.

[0010] According to an embodiment, a computer-readable medium is provided that stores computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method of the present disclosure and send a bitstream including encoded video information associated with predicted chroma samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The disclosed aspects will be described below in conjunction with the accompanying drawings provided to illustrate but not limit the disclosed aspects.

[0012] Figure 1 A block diagram of a general block-based hybrid video coding system is shown.

[0013] FIG. 2A to FIG. 2E Five split types are shown, including quaternary split, horizontal binary split, vertical binary split, horizontal ternary split, and vertical ternary split.

[0014] Figure 3 Shown is an overall block diagram of a block-based video decoder.

[0015] Figure 4 An example of the positions of left and upper samples involved in the CCLM mode and samples of the current block is shown.

[0016] FIG. 5A to FIG. 5C An example of deriving CCLM parameters is shown.

[0017] Figure 6 An example of classifying adjacent samples into two groups based on a value Threshold is shown.

[0018] Figure 7 An example of classifying adjacent sample points into two groups based on inflection points is shown.

[0019] Figure 8A-8B The effect of the scaling adjustment parameter "u" is shown.

[0020] Fig. 9 An example of four reference lines adjacent to a prediction block is shown.

[0021] Fig. 10A and Fig. 10B Schematic diagram showing the correlation between chrominance samples and one or more luma samples.

[0022] Fig.11 An example of using 6 taps in a multiple linear regression (MLR) model according to one or more aspects of the present disclosure is shown.

[0023] Fig.12 Exemplary different filter shapes and / or numbers of taps according to one or more aspects of the present disclosure are shown.

[0024] Fig.13 An example is shown where the FLM may use only the top or left luma and / or chroma samples (extended) for parameter derivation.

[0025] Fig.14 An example showing that FLM can use different rows for parameter derivation.

[0026] Fig.15 Some examples of 1-tap / 2-tap pre-operation are shown.

[0027] Fig.16 A workflow of a method for decoding video data according to one or more aspects of the present disclosure is shown.

[0028] Fig.17 A workflow of a method for encoding video data according to one or more aspects of the present disclosure is shown.

[0029] Fig.18 An exemplary computing system in accordance with one or more aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Reference will now be made in detail to specific implementations, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0031] It should be noted that the terms "first", "second", etc. used in the specification and claims of the present disclosure and the drawings are used to distinguish objects, but not to describe any specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in orders other than those shown in the drawings or described in the present disclosure.

[0032] The first version of the VVC standard was finalized in July 2020, providing approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video codec standard HEVC. Although the VVC standard provides significant codec improvements compared to its predecessor, there is evidence that superior codec efficiency can be achieved by leveraging additional codec tools. Recently, the Joint Video Exploration Team (JVET), in collaboration with ITU-TVCEG and ISO / IEC MPEG, began exploring advanced technologies that could significantly enhance the codec efficiency of VVC. In April 2021, a software code base called the Enhanced Compression Model (ECM) was established for future video codec exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, with several existing modules (e.g., intra / inter prediction, transform, loop filter, etc.) further extended and / or improved. In the future, any new codec tools beyond the VVC standard will need to be integrated into the ECM platform and tested using the JVET Common Test Conditions (CTC).

[0033] Like all previous video codec standards, ECM is built on a block-based hybrid video codec framework. Figure 1 A block diagram of a general block-based hybrid video coding system is shown. The input video signal is processed block by block, which is called a coding unit (CU). In ECM-1.0, a CU can be up to 128x128 pixels. However, like VVC, a coding tree unit (CTU) is split into CUs to accommodate different local characteristics based on quadtree / binary tree / ternary tree. In the multi-type tree structure, a CTU is first partitioned according to the quadtree structure. Then, each quadtree leaf node can be further partitioned according to the binary tree and ternary tree structure. As shown in FIG. Figure 2A , 2B As shown in 2C, 2D and 2E, there are five types of splits, four-way split, vertical binary split, horizontal binary split, vertical extended four-way split and horizontal extended four-way split.

[0034] exist Figure 1In the video code, spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples of coded neighboring blocks in the same video picture / strip (which are called reference samples) to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion compensated prediction") uses reconstructed pixels from coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is usually signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference. Moreover, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture storage the temporal prediction signal comes from. After spatial and / or temporal prediction, the mode decision block in the encoder selects the best prediction mode, for example, based on a rate-distortion optimization method. Then, the prediction block is subtracted from the current video block; and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are inverse quantized and inverse transformed to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. In addition, loop filtering (such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF)) may be applied to the reconstructed CU before the reconstructed CU is placed in a reference picture store and used to encode future video blocks. In order to form an output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to an entropy coding unit to be further compressed and packaged to form a bitstream. It should be noted that the term "block" or "video block" as used herein may be a portion of a frame or picture, particularly a rectangular (square or non-square) portion. With reference to, for example, HEVC and VVC, a block or a video block may be or correspond to a coding tree unit (CTU), a CU, a prediction unit (PU), or a transform unit (TU) and / or may be or correspond to a corresponding block (e.g., a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB)) and / or a sub-block.

[0035] Figure 3 The overall block diagram of a block-based video decoder is shown. The video bitstream is first entropy decoded at the entropy decoding unit. The coding mode and prediction information are sent to the spatial prediction unit (if intra-frame coding) or the temporal prediction unit (if inter-frame coding) to form a prediction block. The residual transform coefficients are sent to the inverse quantization unit and the inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then added together. The reconstructed block can be further loop filtered before it is stored in the reference picture storage. The reconstructed video in the reference picture storage is then sent out to drive the display device and is used to predict future video blocks.

[0036] The main focus of this disclosure is to further enhance the codec efficiency of the codec tool, Cross Component Linear Model (CCLM), which is applied to cross-component prediction in ECM. In the following, some relevant codec tools in ECM are briefly reviewed. Then, some deficiencies in the existing design of CCLM are discussed. Finally, solutions to improve the existing CCLM prediction design are proposed.

[0037] Cross-component linear model prediction

[0038] In order to reduce cross-component redundancy, the cross-component linear model (CCLM) prediction mode is used in VVC. For CCLM, the chroma samples are predicted based on the reconstructed luma samples of the same CU using a linear model as shown below: pred C (i,j)=α·rec L ′(i,j)+ β (1) Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L '(i,j) represents the reconstructed brightness sample point rec L (i, j) is the downsampled reconstructed luminance sample of the same CU obtained by downsampling. The above α and β are linear model parameters, which are derived from up to four adjacent chrominance samples and their corresponding downsampled luminance samples, which can be called adjacent luminance-chrominance sample pairs. Assuming that the current chrominance block has a size of W×H, W' and H' are obtained as follows: —When LM mode is applied, W'=W, H'=H; —When LM-A mode is applied, W'=W+H; - When the LM-L mode is applied, H'=H+W. Among them, in LM mode, the upper and left samples of the CU are used together to calculate the linear model coefficients; in LM_A mode, only the upper samples of the CU are used to calculate the linear model coefficients; and in LM_L mode, only the left samples of the CU are used to calculate the linear model coefficients.

[0039] If the positions of the upper neighboring samples of the chroma block are denoted as S[0,-1]…S[W'-1,-1], and the positions of the left neighboring samples of the chroma block are denoted as S[-1,0]…S[-1,H'-1], then the positions of the four neighboring chroma samples are selected as follows: - When LM mode is applied and both upper and left adjacent samples are available, select S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4] as the positions of the four adjacent chroma samples; —When LM-A mode is applied or only upper adjacent samples are available, select S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8, -1], S[7*W' / 8,-1] as the positions of four adjacent chroma samples; – When LM-L mode is applied or only left adjacent samples are available, select S[-1,H' / 8], S[-1,3*H' / 8], S[-1, 5*H' / 8], S[-1,7*H' / 8] as the positions of the four adjacent chrominance samples.

[0040] The four adjacent brightness samples corresponding to the selected position are obtained by downsampling operation, and the obtained four adjacent brightness samples are compared four times to find the two larger values: x 0 A and x 1 A and two smaller values ​​x 0 B and x 1 B The chrominance sample values ​​corresponding to the two larger values ​​and the two smaller values ​​are represented as y and 0 A ,y 1 A ,y 0 B and 1 B Then, we derive X as follows a , X b , Y a and Y b : X a =(x 0 A +x 1 A +1)>>1; X b =(x 0 B +x 1 B +1)>>1; Y a =(y 0 A +y 1 A +1)>>1; Y b =(y 0 B +y 1 B +1)>>1 (2)

[0041] Finally, the linear model parameters α and β are obtained according to the following equations: β=Y b -α·X b (4)

[0042] Figure 4 Examples of the positions of the left and top samples involved in the CCLM mode and the samples of the current block are shown, including the positions of the left and top samples of the N×N chroma block in the CU and the left and top samples of the 2N×2N luminance block in the CU.

[0043] The division operation used to calculate the parameter α is implemented using a lookup table. In order to reduce the amount of memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed by exponential notation. For example, a 4-bit significant part and an exponent are used to approximate diff. Therefore, the table of 1 / diff is simplified to 16 elements for 16 values ​​of the significant digit as shown below: DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0044] This will have the effect of reducing both the complexity of the calculations and the memory size required to store the necessary tables.

[0045] In addition to the upper template and the left template being used together to calculate the linear model coefficients, they can also be used alternately in the other two LM modes (called LM_A and LM_L modes).

[0046] In LM_T mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0047] In LM_LT mode, the left template and the upper template are used to calculate the linear model coefficients.

[0048] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2 to 1 downsampling ratio in both the horizontal and vertical directions. The selection of the downsampling filter is specified by the SPS level flags. The two downsampling filters are shown below, corresponding to "Type 0" and "Type 2" content, respectively.

[0049] Note that when the upper reference line is located at a CTU boundary, only one luma line (normal line buffer in intra prediction) is used to generate the downsampled luma samples.

[0050] This parameter calculation is performed as part of the decoding process, rather than just as an encoder search operation. Therefore, no syntax is used to communicate the alpha and beta values ​​to the decoder.

[0051] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A and LM_L). The chroma mode signaling and derivation process are shown in Table 1. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited. Table 1 - Chroma prediction modes derived from luma mode when CCLM is enabled

[0052] As shown in Table 2, a single binarization table is used regardless of the value of sps_cclm_enabled_flag. Table 2 - Unified binarization table for chroma prediction mode Value of intra_chroma_pred_mode Binary String 4 00 0 0100 1 0101 2 0110 3 0111 5 10 6 110 7 111

[0053] In Table 2, the first binary indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next binary indicates whether it is LM_CHROMA (0) or not LM_CHROMA. If it is not LM_CHROMA, the next binary indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first binary of the corresponding intra_chroma_pred_mode binarization table can be discarded before entropy coding. Or, in other words, the first binary is inferred to be 0 and therefore not encoded. This single binarization table is used for the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in Table 2 are context coded using their own context model, and the remaining bins are bypass coded.

[0054] In addition, to reduce the luma-chroma latency in the dual tree, when the 64×64 luma coding tree node is split using Not Split (and ISP is not used for 64×64 CU) or QT, the chroma CU in the 32×32 / 32×16 chroma coding tree node is allowed to use CCLM as follows: — If a 32×32 chroma node is not split or split QT split, all chroma CUs in the 32×32 node can use CCLM – If the 32×32 chroma node is split using horizontal BT and the 32×16 child node is not split or uses vertical BT If the chroma node is split, all chroma CUs in the 32×16 chroma node can use CCLM.

[0055] Under all other luma and chroma coding tree split conditions, CCLM is not allowed for chroma CUs.

[0056] During the development of the ECM, the simplified derivation of α and β (min-max approximation) was removed. Instead, a linear least squares solution was performed between the causal reconstructed data of the downsampled luma samples and the causal chroma samples to derive the model parameters α and β. Among them, Rec C (i) and Rec' L (i) indicates the reconstructed chroma samples and downsampled luminance samples around the target block, and I indicates the total number of samples of neighboring data.

[0057] The LM_A and LM_L models are also called multi-directional linear models (MDLM). Figure 5A An example of the operation of the MDLM is shown when the block content cannot be predicted from the L-shaped reconstruction region. Figure 5B MDLM_L is shown using only the left reconstructed samples to derive CCLM parameters. Figure 5C MDLM_T is shown using only the top reconstructed samples to derive CCLM parameters.

[0058] An integerization of the least mean square (LMS) method discussed above (see equations (8)-(9)) has been proposed as an improvement to the CCLM. The initial integerization design of the LMS CCLM was first proposed in JCTVC-C206. The method was then improved through a series of simplifications, including the addition of α to the precision n α The ECM LMS version was eventually formed by JCTVC-F0233 / I0178, which reduced the maximum multiplier bit width from 13 to 7, JCTVC-I0151, which reduced the maximum multiplier bit width, and JCTVC-H0490 / I0166, which reduced the division LUT entries from 64 to 32.

[0059] As discussed in equation (1), the integer design uses a linear relationship to model the correlation of the luma and chroma signals. The chroma values ​​are predicted from the reconstructed luma values ​​of the co-located blocks.

[0060] In YUV420 sampling, the luma component and chroma component have different sampling rates. The sampling rate of the chroma component is half of the sampling rate of the luma component, and there is a 0.5 pixel phase difference in the vertical direction. Reconstructing the luma requires downsampling in the vertical direction and subsampling in the horizontal direction to match the size of the chroma signal. For example, downsampling can be achieved by: Rec L ′(i,j)=(rec L (2i,2j)+rec L (2i,2j+1))>>1 (10)

[0061] Floating point operations are required to calculate the linear model parameter α in equation (8) to maintain high data accuracy. And when α is represented by floating point values, floating point multiplication is involved in equation (1). In this section, an integer implementation of the algorithm is designed. Specifically, the fractional part of the parameter α is expressed in n α The parameter α value is obtained by scaling and rounding the integer value α' and a' = a×(1<<n α ) is expressed. Then the linear model of equation (1) is changed to: pred C [x,y]=(α'·Rec L '[x,y]>>n α )+β' (11) Where β' is the rounded value of the floating point β, and α' can be calculated as shown below.

[0062] It is proposed to replace the division operation of equation (12) by table lookup and multiplication. First, A2 is reduced to reduce the table size. A1 is also reduced to avoid product overflow. Then, in A2, only the The value is limited to the most significant bit, while the other bits are set to zero. The approximate value A2' can be calculated as: where [...] indicates rounding operation, and r A2 can be calculated as: Here, bdepth(A2) represents the bit depth of value A2.

[0063] The same operation is performed on A1 as follows:

[0064] Considering the quantized representation of A1 and A2, equation (12) can be rewritten as shown below. in, is represented as having a length of to avoid division.

[0065] In simulation, the constant parameters are set as: ·n α Equal to 13, this value is a trade-off between data accuracy and computational cost. · Equal to 6, resulting in a lookup table size of 64. When bdepth(A2)<6, the table size can be further reduced to 32 by enlarging A2 (e.g., A2<32). ·n table Equal to 15, resulting in a 16-bit data representation of the table elements. · Set to 15 to avoid product overflow and maintain 16-bit multiplication.

[0066] Finally, α' is truncated to [-2 -15 ,2 15 -1] to preserve the 16-bit multiplication in equation (11). With this truncation, when n α When it is equal to 13, the actual a value is limited to [-4,4), which helps prevent error amplification.

[0067] Using the calculated parameter α', the parameter β' is calculated as follows: Here, the division in the above equation can be simply replaced by a shift since the value I is a power of 2.

[0068] Similar to the discussion above with respect to equation (1), in HM6.0, an intra prediction mode called LM is applied to predict the chroma PU based on a linear model using the reconstruction of the co-located luma PU. The parameters of the linear model consist of the slope (a>>k) and the y-intercept (b), which are derived from neighboring luma and chroma pixels using a least mean square solution. The value of the prediction sample predSamples[x,y], where x,y=0…nS-1, where nS specifies the block size of the current chroma PU, is derived as follows: predSamples[x,y]=Clip1 C (((p Y '[x,y]*a)>>k)+b), where x,y=0..nS-1 (17) Among them, P Y '[x,y] is the reconstructed pixel from the corresponding luminance component. When the coordinates x and y are equal to or greater than 0, P Y ' is the reconstructed pixel from the same brightness PU. When x or y is less than 0, P Y ' is the reconstructed adjacent pixel of the same brightness PU.

[0069] Some intermediate variables L, C, LL, LC, k2 and k3 in the derivation process are derived as follows: k2=Log2((2*nS)>>k3) (18-5) k3=Max(0,BitDepthC+Log2(nS)-14) (18-6)

[0070] Therefore, the variables a, b, and k can be derived as: a1=(LC< <k2)–L*C (19-1) a2=(LL< <k2)–L*L (19-2) k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))-14)+2 (19-3) a1s=a1>>Max(0,Log2(abs(a1))-14) (19-4) a2s=abs(a2>>Max(0,Log2(abs(a2))-5)) (19-5) a3=a2s<1?0:Clip3(-2 15 ,215 -1,a1s*lmDiv+(1<<(k1-1))>>k1) (19-6) a=a3>>Max(0,Log2(abs(a3))-6) (19-7) k=13–Max(0,Log2(abs(a))-6) (19-8) b=(L–((a*C)>>k1)+(1<<(k2-1)))>>k2, (19-9) where lmDiv is specified in a 63-entry lookup table (i.e., Table 3), which is generated online as follows: lmDiv(a2s)=((1<<15)+a2s / 2) / a2s. (20) Table 3 - lmDiv Specifications a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 lmDiv 32768 16384 10923 8192 6554 5461 4681 4096 3641 3277 2979 2731 2521 a2s 14 15 16 17 18 19 20 21 22 23 24 25 26 lmDiv 2341 2185 2048 1928 1820 1725 1638 1560 1489 1425 1365 1311 1260 a2s 27 28 29 30 31 32 33 34 35 36 37 38 39 lmDiv 1214 1170 1130 1092 1057 1024 993 964 936 910 886 862 840 a2s 40 41 42 43 44 45 46 47 48 49 50 51 52 lmDiv 819 799 780 762 745 728 712 697 683 669 655 643 630 a2s 53 54 55 56 57 58 59 60 61 62 63 64 lmDiv 618 607 596 585 575 565 555 546 537 529 520 512

[0071] In equation (19-6), a1s is a 16-bit signed integer, and lmDiv is a 16-bit unsigned integer. Therefore, a 16-bit multiplier and 16-bit storage are required. It is proposed to reduce the bit depth of the multiplier to the internal bit depth, as well as the size of the lookup table, as detailed below.

[0072] The bit depth of a1s is reduced to the internal bit depth by changing equation (19-4) to the following: a1s=a1>>Max(0,Log2(abs(a1))–(BitDepth C –2)). (21) The value of lmDiv with internal bit depth is obtained using the following equation (22) and stored in the lookup table: lmDiv(a2s)=((1<<(BitDepth C -1))+a2s / 2) / a2s. (22)

[0073] Table 4 shows an example of an internal bit depth of 10. Table 4 - Specification of lmDiv with internal bit depth equal to 10 a2s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 lmDiv 512 256 171 128 102 85 73 64 57 51 47 43 39 37 34 32 a2s 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 lmDiv 30 28 27 26 24 23 22 21 20 20 19 18 18 17 17 16 a2s 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 lmDiv 16 15 15 14 14 13 13 13 12 12 12 12 11 11 11 11 a2s 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 lmDiv 10 10 10 10 10 9 9 9 9 9 9 9 8 8 8

[0074] Equation (19-3) and Equation (19-8) are also modified as follows: k1=Max(0,Log2(abs(a2))-5)–Max(0,Log2(abs(a1))–(BitDepth C –2)), and (23-1) k = BitDepthC –1–Max(0,Log2(abs(a))-6). (23-2)

[0075] It is also proposed to reduce the number of entries from 63 to 32, and the number of bits for each entry from 16 to 10, as shown in Table 5. By doing so, a memory saving of almost 70% can be achieved. The corresponding changes to equation (19-6), equation (20), and equation (19-8) are as follows: a3=a2s<32?0:Clip3(-2 15 ,2 15 -1,a1s*lmDiv+(1<<(k1-1))>>k1) (24-1) lmDiv(a2s)=((1<<(BitDepth C +4))+a2s / 2) / a2s (24-2) k = BitDepth C +4–Max(0,Log2(abs(a))-6). (24-3) Table 5 - Specification of lmDiv with internal bit depth equal to 10

[0076] Multi-model linear model prediction

[0077] In ECM-1.0, a multi-model LM (MMLM) prediction mode is proposed. For the MMLM prediction mode, the chroma samples are predicted based on the reconstructed luma samples of the same CU using two linear models as shown below: Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma point of the same CU. Threshold is calculated as the average of adjacent reconstructed luma samples. Figure 6 An example of classifying adjacent samples into two groups based on the value Threshold is shown. For each group, the parameter α i and β i (where i is equal to 1 and 2 respectively) is derived from the linear relationship between the luminance and chrominance values ​​from the two samples, which are the minimum luminance samples in the group A(X A , Y A ) and the maximum brightness sample B(X B , Y B ). Here XA , Y A is the x-coordinate value (i.e., brightness value) and y-coordinate value (i.e., chrominance value) of sample point A, and X B , Y B are the x-coordinate value and y-coordinate value for sample point B. The linear model parameters α and β are obtained according to the following equations. β=y A -αx A (26)

[0078] Such a method is also called the min-max method. The division in the above equation can be avoided and replaced by multiplication and shift.

[0079] For coding blocks with a square shape, the above two equations are directly applied. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0080] In addition to the scenario in which the upper template and the left template are used together to calculate the linear model coefficients, the two templates can also be used alternately in the other two MMLM modes (referred to as MMLM_A mode and MMLM_L mode).

[0081] In MMLM_A mode, only the pixel samples in the upper template are used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to a size of (W+W). In MMLM_L mode, only the pixel samples in the left template are used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to a size of (H+H).

[0082] Note that when the upper reference line is located at a CTU boundary, only one luma line (which is stored in the line buffer for intra prediction) is used to generate the downsampled luma samples.

[0083] For chroma intra mode coding, a total of 11 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and six cross-component linear model modes (CCLM, LM_A, LM_L, MMLM, MMLM_A and MMLM_L). The chroma mode signaling and derivation process are shown in Table 6. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since a separate block partitioning structure for luminance components and chrominance components is enabled in the I stripe, a chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited. Table 6 - Derivation of chroma prediction mode from luma mode when MMLM is enabled

[0084] MMLM and LM modes can also be used together in an adaptive manner. For MMLM, the two linear models are as follows: Among them, pred C (i, j) represents the predicted chroma sample in the CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. Threshold can be simply determined based on the luma average and chroma average values, as well as the luma minimum and maximum values, and the chroma minimum and maximum values. Figure 7 An example of classifying adjacent samples into two groups based on the inflection point T (indicated by the arrow) is shown. The linear model parameters α1 and β1 are derived from the straight-line relationship between the luminance value and the chrominance value from the two samples, which are the minimum luminance sample A (X A , Y A ) and Threshold(X T , Y T The linear model parameters α2 and β2 are derived from the linear relationship between the luminance and chrominance values ​​of two sample points, which are the maximum luminance sample point B(X B , Y B ) and Threshold(X T , Y T ). Here X A , Y A is the x-coordinate value (i.e., brightness value) and y-coordinate value (i.e., chrominance value) of sample point A, and X B , Y B is the x-coordinate value and y-coordinate value of sample point B. For each group of linear model parameters α i and β i (where i is equal to 1 and 2, respectively) is obtained according to the following equation. β2=Y T -α2X T (28)

[0085] For coding blocks with a square shape, the above equation is directly applied. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary.

[0086] In addition to the scenario in which the upper template and the left template are used together to determine the linear model coefficients, the two templates can also be used alternately in the other two MMLM modes (referred to as MMLM_A mode and MMLM_L mode, respectively).

[0087] In MMLM_A mode, only the pixel samples in the upper template are used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to a size of (W+W). In MMLM_L mode, only the pixel samples in the left template are used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to a size of (H+H).

[0088] Note that when the upper reference line is located at a CTU boundary, only one luma line (which is stored in the line buffer for intra prediction) is used to generate the downsampled luma samples.

[0089] For chroma intra mode coding, there is a conditional check that is used to select LM mode (CCLM, LM_A and LM_L) or multi-model LM mode (MMLM, MMLM_A and MMAM_L). The conditional check is as follows: Among them, BlkSizeThres LM Indicates the minimum block size for LM mode, and BlkSizeThres MM represents the minimum block size of the MMLM mode. The symbol d represents a predetermined threshold. In one example, d may take a value of 0. In another example, d may take a value of 8.

[0090] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes. The chroma mode signaling and derivation process are shown in Table 1. It is worth noting that for a given CU, if the CU is encoded in the linear model mode, then the linear model mode is determined to be the traditional single model LM mode or the MMLM mode based on the above conditional check. Unlike the case shown in Table 6, there is no separate MMLM mode to be signaled. Chroma mode encoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance components and chrominance components are enabled in the I stripe, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0091] During the ECM development, scaling (slope) adjustments to the CCLM were proposed as a further improvement, for example as described in JVET-Y0055 / Z0049.

[0092] As mentioned above, CCLM uses a 2-parameter model to map luma values ​​to chroma values. The scaling parameter "a" and the bias parameter "b" define the mapping as follows: chromaVal=a*lumaVal+b (30)

[0093] It is proposed to signal an adjustment "u" to the scaling parameter to update the model to the following form: chromaVal=a'*lumaVal+b' (31) Where a' = a + u, and b' = bu * y r .

[0094] With this choice, the mapping function is centered around the brightness value y r The point is tilted or rotated. It is proposed to use the average value of the reference brightness samples used in model creation as y r , in order to provide meaningful modifications to the model. Figure 8A-8B shows the effect of scaling adjustment parameter "u", where Fig. 8A shows a model created without the scaling parameter "u", and Figure 8B A model created with a scaling adjustment parameter "u" is shown.

[0095] In one example, the scaling parameter is provided as an integer between -4 and 4, inclusive, and signaled in the bitstream. The unit of the scaling parameter is 1 / 8 of the chroma sample value per luma sample value (for 10-bit content).

[0096] In one example, the CCLM models ("LM_CHROMA_IDX" and "MMLM_CHROMA_IDX") that can be used when using reference samples both above and to the left of the block are adjusted, but not for "one-sided" mode. This choice is based on the trade-off between codec efficiency and complexity.

[0097] When scaling adjustments are applied to the multi-mode CCLM model, both models may be adjusted and thus up to two scaling updates are signaled for a single chroma block.

[0098] To implement scaling at the encoder, the encoder may perform a SATD-based search for Cr to find the best value for the scaling update and a similar SATD-based search for Cb. If either result is a non-zero scaling parameter, the combined scaling pair (SATD-based update for Cr, SATD-based update for Cb) is included in the RD check list for the TU.

[0099] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Fig. 9 In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not extracted from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0100] The index of the selected reference row (mrl_idx) is signaled and used to generate intra prediction values. For reference row indices greater than 0, only additional reference row modes are included in the MPM list, and only the mpm index is signaled without the remaining modes. The reference row index is signaled before the intra prediction mode, and if a non-zero reference row index is signaled, the intra prediction mode does not include planar mode.

[0101] MRL is disabled for the first row of a block within a CTU to prevent the use of extended reference samples outside the current CTU row. In addition, PDPC is disabled when additional rows are used. For MRL mode, the derivation of the DC value in the DC intra prediction mode for non-zero reference row index is aligned with the derivation of reference row index 0. MRL requires storage of 3 adjacent luma reference rows with a CTU to generate predictions. The Cross Component Linear Model (CCLM) tool also requires 3 adjacent luma reference rows for its downsampling filter. The definition of MRL using the same 3 rows is consistent with CCLM to reduce the storage requirements of the decoder.

[0102] In the existing CCLM or MMLM design, adjacent reconstructed luma-chroma sample pairs are classified into one or more sample groups based on the value Threshold, which only considers the luma DC value. That is, the luma-chroma sample pairs are classified by considering only the intensity of the luma sample. However, the luma component usually retains rich textures, and the current luma sample can be highly correlated with the adjacent luma samples. Such inter-sample correlation (AC correlation) can be beneficial to the classification of luma-chroma sample pairs and can bring additional coding efficiency.

[0103] Marginal Linear Model (ELM)

[0104] In order to improve the coding efficiency of the luma component and the chroma component, a classifier that considers luma edge or AC information is introduced, in contrast to the above-mentioned implementation in which only the luma DC value is considered. In addition to the existing band-classified MMLM, the present disclosure also provides an exemplary classifier. The process of generating a linear prediction model for different groups of samples can be similar to CCLM or MMLM (e.g., via least squares or a simplified minimum-maximum method, etc.), but with metrics for different classifications. Different classifiers can be used to classify adjacent luma samples (e.g., adjacent luma samples in adjacent luma-chroma sample pairs) and / or luma samples corresponding to chroma samples to be predicted. Luma samples corresponding to chroma samples can be obtained by a downsampling operation to match the position of corresponding chroma samples of a 4:2:0 video sequence. For example, luma samples corresponding to chroma samples can be obtained by performing a downsampling operation on more than one (e.g., 4) reconstructed luma samples corresponding to chroma samples (e.g., located around the chroma samples). Alternatively, for example, in the case of a 4:4:4 video sequence, the luma samples may be obtained directly from the reconstructed luma samples. Alternatively, the luma samples may be obtained from corresponding reconstructed luma samples located at corresponding co-located positions of the corresponding chroma samples. For example, the luma sample to be classified may be obtained from one of the four reconstructed luma samples corresponding to the chroma samples, the one reconstructed luma sample being located at the upper left position of the four reconstructed luma samples, which may be regarded as the co-located position of the chroma samples.

[0105] The first classifier can classify the brightness sample according to its edge strength. For example, a direction (e.g., 0 degrees, 45 degrees, or 90 degrees, etc.) can be selected to calculate the edge strength. The direction can be formed by the current sample and the adjacent sample along the direction (e.g., the adjacent sample located 45 degrees to the upper right of the current sample). The edge strength can be calculated by subtracting the adjacent sample from the current sample. The edge strength can be quantized into one of M segments by M-1 thresholds, and the first classifier can classify the current sample using M categories. Alternatively or additionally, N directions can be formed by the current sample and N adjacent samples along N directions. N edge strengths can be calculated by subtracting N adjacent samples from the current sample, respectively. Similarly, if each of the N edge strengths can be quantized into one of M segments by M-1 thresholds, the first classifier can classify the current sample using MN categories.

[0106] The second classifier can be used to classify according to the local pattern. For example, the current luma sample Y0 can be compared with its N adjacent luma samples Yi. If the value of Y0 is greater than the value of Yi, the score can be increased by 1, otherwise, the score can be reduced by 1. The score can be quantized to form K categories. The second classifier can classify the current sample into one of the K categories. For example, the adjacent luma samples can be obtained from four neighbors located above, to the left, to the right, and below the current luma sample (i.e., there are no diagonal neighbors).

[0107] It is contemplated that multiple first classifiers, second classifiers, or different instances of the first classifiers or second classifiers or other classifiers described herein may be combined. For example, the first classifier may be combined with an existing MMLM threshold-based classifier. For another example, an instance A of a first classifier may be combined with another instance B of the first classifier, wherein instance A and instance B adopt different directions (e.g., vertical and horizontal directions, respectively).

[0108] Those skilled in the art will appreciate that, although the existing CCLM design in the VVC standard is used as the basic CCLM method in the description, the proposed cross-component method described in the present disclosure can also be applied to other predictive codecs that adopt a similar design spirit. For example, for chroma from luma (CfL) in the AV1 standard, the proposed method can also be applied by dividing the luma-chroma sample pairs into multiple sample groups.

[0109] Those skilled in the art will recognize that Y / Cb / Cr can also be represented as Y / U / V in the field of video coding. For example, if the video data is in RGB format, the proposed method can also be applied by simply mapping the YUV symbol to GBR.

[0110] Filter-based Linear Model (FLM)

[0111] like Fig. 10A As shown in , CCLM assumes that a given chrominance sample is only related to the corresponding luma sample (L0.5, which can be regarded as a fractional luma sample position), and a simple linear regression (SLR) estimated by ordinary least squares (OLS) is used to predict the given chrominance sample. Fig. 10B As shown in , in some video contents, one chrominance sample may be simultaneously correlated with multiple luminance samples (AC or DC correlated), so a multiple linear regression (MLR) model can further improve the prediction accuracy.

[0112] Therefore, a filter-based linear model (FLM) using the MLR model is introduced as follows to consider the possibility that one chrominance sample can be simultaneously correlated with multiple luma samples.

[0113] For the chroma samples to be predicted, the reconstructed co-located and adjacent luma samples can be used to predict the chroma samples to capture the inter-sample correlation between co-located luma samples, adjacent luma samples, and chroma samples. The reconstructed luma samples are linearly weighted and combined with an "offset" to generate predicted chroma samples (C: predicted chroma samples, L i : the i-th reconstructed co-located or adjacent brightness sample, α i : filter coefficients, β: offset, N: filter taps). Note that the linear weighting plus the offset value directly forms the predicted chroma samples (which can be low-pass, high-pass adaptively according to the video content), and then added to the residual to form the reconstructed chroma samples.

[0114] For a given CU, the top and left reconstructed luma and chroma samples can be used to derive or train the FLM parameters ((α i, , β). As with CCLM, α i and β can be derived via OLS. The top and left training samples are collected and a pseudo-inverse matrix is ​​computed on both the encoder and decoder sides to derive the parameters, which are then used to predict the chroma samples in a given CU. Let N denote the number of filter taps applied to the luma samples, M denote the total top and left reconstructed luma-chroma sample pairs used to train the parameters, represents the luma sample with the i-th sample pair and the j-th filter tap, C i Denotes the chrominance sample with the i-th sample pair, the following equation shows the pseudo-inverse matrix A + derivation and parameters. Fig.11 An example is shown where N is 6 (6 taps), M is 8, the top 2 rows and left 3 columns of luma samples, and the top 1 row and left 1 column of chroma samples are used to derive or train parameters.

[0115] Note that it is possible to i Without the offset β to predict the chrominance samples, this can be a subset of the proposed method.

[0116] The proposed ELM / FLM / GLM (as discussed below) can be directly extended to the CfL design in the AV1 standard, which explicitly sends the model parameters (α, β). For example, (1-tap case) α and / or β are derived at the encoder at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and signaled to the decoder for CfL mode.

[0117] To further improve the encoding and decoding performance, additional designs can be used in FLM prediction. Fig.11 As shown in and discussed above, a 6-tap luma filter is used for FLM prediction. However, although the multi-tap filter can be well adapted to the training data (e.g., top and left adjacent reconstructed luma samples and chroma samples), in some cases, the training data cannot capture all the characteristics of the test data, which may lead to overfitting and may not predict the test data well (i.e., the chroma block samples to be predicted). In addition, different filter shapes can be well adapted to different video block contents, resulting in more accurate predictions.

[0118] To address this issue, the filter shape and the number of filter taps may be predefined or signaled or switched at the sequence parameter set (SPS), adaptation parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, CTU, CU, sub-block, or sample level. A set of filter shape candidates may be predefined, and the selection of the set of filter shape candidates may be signaled or switched at the SPS, APS, PPS, PH, SH, region, CTU, CU, sub-block, or sample level. Different components (e.g., U and V) may have different filter switch controls. For example, a set of filter shape candidates may be predefined (e.g., indicated by indices 0 to 5), and a filter shape (1, 2) may represent a 2-tap luminance filter, a filter shape (1, 2, 4) may represent a 3-tap luminance filter, and so on. Fig.11 As shown in . The filter shape selection for the U component and the V component can be switched in the PH or in the CU or CTU level. Note that as described herein, the N-tap can represent an N-tap with or without an offset β. An example is given in Table 7 as follows. Table 7 - Example signaling and switching for different filter shapes

[0119] Different chroma types and / or color formats may have different predefined filter shapes and / or taps. For example, a predefined filter shape (1, 2, 4, 5) may be used for 4:2:0 type-0, a predefined filter shape (0, 1, 2, 4, 7) may be used for 4:2:0 type-2, and a predefined filter shape (1, 4) may be used for 4:2:2, and a predefined filter shape (0, 1, 2, 3, 4, 5) may be used for 4:4:4, as shown in FIG. Fig.12 as shown in .

[0120] In another aspect of the present disclosure, unavailable luma and chroma samples used to derive the MLR model may be filled from available reconstructed samples. Fig.12 , then for the CU located at the left picture boundary, the left column including sample (0, 3) is unavailable (beyond the picture boundary), so sample (0, 3) is repeatedly padded from sample (1, 4) to apply the 6-tap filter. Note that the padding process can be applied to both the training data (top and left adjacent reconstructed luma and chroma samples) and the test data (luma and chroma samples in the CU).

[0121] As mentioned above, the MLR model (linear equation) must be derived at both the encoder and the decoder. According to one or more aspects of the present disclosure, several methods are proposed to derive the pseudo-inverse matrix A + , or solve the linear equations directly. Other known methods can also be applied, like Newton's method, Cayley-Hamilton method, and eigendecomposition as mentioned in https: / / en.wikipedia.org / wiki / Invertible_matrix.

[0122] In this disclosure, for simplicity, A + It can be represented as A -1 . Linear equations can be solved as follows: 1. Solve A by transposing the adjoint matrix (adjA), closed form, and analytical solution -1 : The following shows a general form of nxn, a 2x2 case, and a 3x3 case. If FLM uses 3x3, then 2 scales plus 1 offset need to be solved. b=Ax,x=(A T A) -1 A T b=A + b, represented by A -1 b By removing the (n-1)×(n-1) submatrix with the j-th row and the i-th column 2. Gauss-Jordan Elimination The linear equations can be solved using Gauss-Jordan elimination by augmenting the matrix [A In] and a series of elementary row operations to obtain the row-reduced echelon form [I|X]. 2x2 and 3x3 examples are shown below. 3. Cholesky Decomposition To solve Ax = b, one can first decompose A through the Cholesky-Crout algorithm to obtain an upper triangular matrix and a lower triangular matrix, and then serially apply one forward substitution and one back substitution to obtain the solution. A 3x3 example is shown below.

[0123] In addition to the above examples, some cases require special handling. For example, if some cases cause the linear equations to be unsolvable, default values can be used to fill the chrominance prediction values. The default values can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, when predefined as 1<<(bitDepth-1) or meanC, meanL, or meanC-meanL (average current chrominance or other chrominance and luminance values from the subset of available or FLM reconstructed adjacent regions).

[0124] The following example shows the case when matrix A is unsolvable, and in this case, default prediction values can be assigned to the entire current block: 1. Solve by closed form (analytical, adjugate matrix), but A is singular (i.e., detA = 0); 2. Solve by Cholesky decomposition, but A cannot be Cholesky decomposed, g jj <REG_SQR, where REG_SQR is a small value and can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0125] Fig.11 Typical cases of deriving FLM parameters using the top 2 and / or left 3 luminance rows and the top 1 and / or left 1 chrominance row are shown. However, as mentioned above, due to different block contents and different reconstruction qualities of adjacent samples, using different regions for parameter derivation can bring coding and decoding benefits. Several methods for selecting the regions for parameter derivation applications are proposed below: 1. Similar to MDLM, FLM derivation may only use the top or left luminance and / or chrominance samples to derive parameters. Whether to use FLM, FLM_L, or FLM_T can be predefined or signaled or switched at the SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Assuming the current chrominance block has a size of W×H, then W' and H' are obtained as follows: — When applying the FLM mode, W' = W, H' = H; - When FLM_T mode is applied, W'=W+We; where We represents the extended top luma / chroma sample; - When FLM_L mode is applied, H'=H+He, where He represents the extended left luma / chroma samples. The number of extended luma / chroma samples (We, He) may be predefined, or signaled or switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, (We, He) = (H, W) is predefined as VVC CCLM, or (W, H) is predefined as ECM CCLM. Unavailable (We, He) luma / chroma samples can be repeatedly filled from the nearest (horizontal, vertical) luma / chroma samples. Fig.13 A diagram of FLM_L and FLM_T is shown (eg, at 4 taps). When FLM_L or FLM_T is applied, only H' or W' luma / chroma samples, respectively, are used for parameter derivation. 2. Similar to MRL, different row indices can be predefined or signaled or switched in SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level to indicate the selected luma-chroma sample pair rows. This may benefit from different reconstruction qualities of different row samples. Fig.14 It is shown that (similar to MRL) FLM can use different rows for parameter derivation (e.g., at 4 taps). For example, FLM can use light blue / light yellow luma and / or chroma samples of index 1. 3. Expand the CCLM region and extract all top N and / or left M rows for parameter derivation. Fig.14 It is shown that all dark blue, light blue and dark yellow, light yellow areas can be used simultaneously. Using a larger area (data) for training may produce a more robust MLR model.

[0126] For FLM prediction, the corresponding syntax may be defined as follows in Table 8, where FLC represents fixed length code, TU represents truncated unary code, EGk represents exponential Golomb code with order k, where k may be fixed or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level, SVLC represents signed EG0, and UVLC represents unsigned EG0. Table 8 - Example of FLM syntax

[0127] Note that the binarization of each syntax element can be changed.

[0128] A new method for cross-component prediction is proposed based on the existing linear model design to further improve the encoding and decoding accuracy and efficiency. The main aspects of the proposed method are detailed as follows.

[0129] While the FLM discussed above offers the best flexibility (leading to the best performance), it requires solving for many unknown parameters if the number of filter taps increases. When the inverse matrix is ​​larger than 3x3, the closed form derivation is not suitable (too many multipliers) and requires an iterative approach like Cholesky, which increases the burden on the decoder processing cycle. In this section, a pre-operation before applying the linear model is proposed, including using the sample gradient to exploit the correlation between the luminance AC information and the chrominance intensity. With the help of the gradient, the number of filter taps can be effectively reduced.

[0130] Please note that the methods / examples in this section can be combined / reused from any of the designs discussed above, including but not limited to classification, filter shapes, matrix derivation (with special processing), application areas, syntax. In addition, the methods / examples listed in this section can also be applied to any of the designs discussed above to obtain better performance at a certain complexity trade-off.

[0131] Please note that the reference samples / training templates / reconstructed neighboring regions used in this article generally refer to luma samples used to derive MLR model parameters, which are then applied to internal luma samples in a CU to predict chroma samples in the CU.

[0132] According to the proposed method, instead of directly using luma sample intensity values ​​as input to the linear model, pre-operations (e.g., pre-linear weighting, sign, scaling / absolute value, thresholding, ReLU) can be applied to reduce the dimensionality of the unknown parameters. In one example, the pre-operations can include calculating sample differences based on luma sample values. As understood by those skilled in the art, sample differences can be characterized as gradients, and thus this new method is also referred to as a gradient linear model (GLM) in some embodiments.

[0133] Note that the following detailed description discusses scenarios in which the proposed pre-operation can be reused / combined with the SLR model (also known as the 1-tap case), as well as reused / combined with the MLR model (also known as the multi-tap case, e.g., 2-tap).

[0134] For example, instead of applying a 2-tap to the 2 luma samples, the 2 luma samples may be pre-operated and then a simpler 1-tap may be applied to reduce complexity. Fig.15 Some examples for 1-tap / 2-tap (with offset) pre-operation are shown, where the 2-tap coefficients are represented as (a, b). Note that Fig.15 Each circle shown in represents an illustrative chroma position in the YUV4:2:0 format. As discussed above, in the YUV 4:2:0 format, a luma sample corresponding to a chroma sample can be obtained by performing a downsampling operation on more than one (e.g., 4) reconstructed luma samples corresponding to the chroma sample (e.g., located around the chroma sample). In other words, a chroma position can correspond to one or more luma samples including a co-located luma sample. Different 1-tap modes are designed for different gradient directions, and use different "interpolated" luma samples (weighted to different luma positions) for gradient calculation. For example, a typical filter [1,0,-1; 1,0,-1] such as Fig.15 As shown in , it represents the following operations: Among them, rec L Represents the reconstructed brightness sample value, and Rec L ″(o,j) represents the pre-operation brightness sample value. Also note that Fig.15 The 1-tap filter shown in can be understood as an alternative to the downsampling filter as used in CCLM (see equations (6)-(7)), where the filter coefficients are changed.

[0135] Pre-operation can be based on gradient, edge direction (detection), pixel intensity, pixel change, pixel variance, Roberts / Prewitt / Compass / Sobel / Laplacian operators, high-pass filter (by calculating gradient or other related operators), low-pass filter (by performing weighted average operation)... etc. The edge direction detector listed in the example can be extended to different edge directions. For example, 1-tap (1,-1) or 2-tap (a,b) applied along different directions to detect different edge gradients. Fig.15 Example (420 Type-0 case), the filter shape / coefficients may be symmetric about the chrominance position.

[0136] Pre-operation parameters (coefficients, sign, scaling / absolute value, thresholding, ReLU) can be fixed or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Note that in this example, if multiple coefficients apply to one sample (e.g., -1, 4), they can be merged (e.g., 3) to reduce operations.

[0137] In one example, the pre-operation may involve calculating sample differences of the luma sample values. Alternatively, the pre-operation may include performing downsampling by a weighted average operation. In some cases, the pre-operation may be applied repeatedly. For example, one may apply a template filter to the template using a low-pass smoothing FIR filter [1,2,1] / 4 or [1,2,1; 1,2,1] / 8 (i.e., downsampling) to remove outliers, and then apply a 1-tap GLM filter to calculate sample differences to infer a linear model. It is contemplated that one may also calculate sample differences and then implement downsampling.

[0138] In one example, pre-operation coefficients (applied last (eg, 3), or intermediately (eg, -1, 4) to each luma sample) may be limited to power-of-2 values ​​to save multipliers.

[0139] In one aspect of the present disclosure, the proposed new method can be reused / combined with the CCLM discussed above, which uses a simple linear regression (SLR) model and uses one corresponding luma sample value to predict the chroma sample value. This is also called the 1-tap case. In this case, deriving the linear model also includes deriving the scaling parameter α and the offset parameter β by using the pre-operated adjacent luma sample values ​​and adjacent chroma sample values. Alternatively, the linear model can be rewritten as: C=α·L+β (33) Where L here represents the "pre-operated" luminance sample. The parameter derivation of the 1-tap GLM can reuse the CCLM design, but consider the directional gradient (a high-pass filter can be used). In one example, the scaling parameter α can be derived by using a division lookup table, as described in detail below, to achieve simplification.

[0140] In one example, when the GLM is combined with the SLR model, the scaling parameter α and the offset parameter β can be derived by using the minimum-maximum method discussed above. Specifically, the scaling parameter α and the offset parameter β can be derived by comparing the adjacent luminance sample values ​​of the pre-operation to determine the minimum luminance sample value Y A and the maximum brightness sample value Y B ; For the minimum brightness sample value Y A and the maximum brightness sample value Y B Determine the corresponding chroma sample value X A and X B ; and according to the following equation, based on the minimum brightness sample value Y A , maximum brightness sample value Y B And the corresponding chroma sample value X A and X B To derive the scaling parameter α and the offset parameter β: β=Y A -αX A . (34)

[0141] In one example, when the GLM is combined with the SLR model, the scaling adjustment discussed above can be reused. In this case, the encoder can determine the scaling adjustment value (e.g., "u") to be signaled in the bitstream and add the scaling adjustment value to the derived scaling parameter α. The decoder can determine the scaling adjustment value (e.g., "u") from the bitstream and add the scaling adjustment value to the derived scaling parameter α. The added value is ultimately used to predict the intra chroma sample value.

[0142] In one aspect of the present disclosure, the proposed new method can be reused / combined with FLM, which uses a multiple linear regression (MLR) model and uses multiple luma sample values ​​to predict chroma sample values. This is also called a multi-tap case, e.g., 2-tap. In this case, the linear model can be rewritten as:

[0143] In this case, multiple scaling parameters α and offset parameters β can be derived by using adjacent luminance sample values ​​and adjacent chrominance sample values ​​of the pre-operation. In one example, the offset parameter β is optional. In one example, at least one of the multiple scaling parameters α can be derived by using sample differences. In addition, another of the multiple scaling parameters α can be derived by using downsampled luminance sample values. In one example, at least one of the multiple scaling parameters α can be derived by using horizontal or vertical sample differences calculated based on downsampled adjacent luminance sample values. In other words, the linear model can combine multiple scaling parameters α associated with different pre-operations.

[0144] The proposed GLM can be combined with the MMLM or ELM discussed above. When combined with classification, a syntax indicating the shape is adopted for each group, and each group can share or have its own filter shape. For example, as an exemplary classifier, the horizontal gradient grad_hor can be classified into the first group, which corresponds to the first linear model, and the vertical gradient grad_ver can be classified into the second group, which corresponds to the second linear model. In one example, the horizontal brightness pattern can be generated only once.

[0145] Additional possible classifiers are provided as shown below. Using the classifier, adjacent and internal luma-chroma sample pairs of the current video block can be classified into a plurality of groups based on one or more thresholds. Note that, as discussed above, each adjacent / internal chroma sample and its corresponding luma sample can be referred to as a luma-chroma sample pair. One or more thresholds are associated with the intensity of the adjacent / internal luma samples. In this case, each of the plurality of groups corresponds to a respective one of the plurality of linear models.

[0146] When combined with the MMLM classifier, the following operations may be performed: adjacent reconstructed luminance-chrominance sample pairs of the current video block are classified into 2 groups based on Threshold; different linear models are derived for different groups, wherein the derivation process may be GLM simplified, i.e., the above pre-operation is adopted to reduce the number of taps; similarly, luminance-chrominance sample pairs within the CU (internal luminance-chrominance sample pairs, wherein each of the internal luminance-chrominance sample pairs includes an internal chrominance sample value to be predicted using the derived linear model) are classified into 2 groups based on Threshold; different linear models are applied to the reconstructed luminance samples in different groups; and chrominance samples in the CU are predicted based on the differently classified linear models. Wherein, Threshold may be the average value of adjacent reconstructed luminance samples. Note that the number of classes (2) can be extended to multiple classes by increasing the number of Thresholds (e.g., averaging based on the min / max of adjacent reconstructed (downsampled) luma samples, fixed, or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level).

[0147] In one example, the filtered values ​​of the FLM / GLM applied to adjacent luma samples are used for classification instead of the MMLM luma DC intensity. For example, if a 1-tap (1,-1) GLM is applied, the average AC value (physical meaning) is used. The processing may be: adjacent reconstructed luma-chroma sample pairs are classified into K groups based on one or more filter shapes, one or more filtered values, and K-1Threshold Ti; different MLR models are derived for different groups, wherein the derivation process may be GLM simplified, i.e., the above-mentioned pre-operation is adopted to reduce the number of taps; luma-chroma sample pairs within the CU (internal luma-chroma sample pairs, wherein each of the internal luma-chroma sample pairs includes an internal chroma sample value to be predicted using the derived linear model) are similarly classified into K groups based on one or more filter shapes, one or more filtered values, and K-1Threshold Ti; different linear models are applied to the reconstructed luma samples in different groups; and chroma samples in the CU are predicted based on differently classified linear models. Among them, Threshold can be predefined (e.g., 0 or can be a table) or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level). For example, Threshold can be the average AC value (filtered value) of adjacent reconstructed (can be downsampled) luma samples (2 groups), or averaged based on minimum / maximum AC (K groups).

[0148] It is also proposed to combine GLM with ELM classifiers. Fig.15 As shown in , a filter shape (e.g., 1-tap) can be selected to calculate the edge strength. A direction is formed by the current sample and N neighboring samples along the direction (e.g., all 6). The processing may then include: calculating an edge strength by the value of the filtered wave (e.g., equivalent value); quantizing the edge strength into M segments by M-1 thresholds Ti; classifying the current sample using K categories (e.g., K == M); deriving different MLR models for different groups, wherein the derivation process may be GLM simplified, i.e., using the above-mentioned pre-operation to reduce the number of taps; classifying the luminance-chrominance sample pairs within the CU into K groups; applying different MLR models to the reconstructed luminance samples in different groups; and predicting the chrominance samples in the CU based on the differently classified MLR models. Please note that the filter shape used for classification may be the same as or different from the filter shape used for MLR prediction. Both the number of thresholds M-1 and the threshold value Ti can be fixed or signaled / switched in SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. In addition, other classifiers / combinations of classifiers as discussed in ELM can also be used for GLM.

[0149] If the number of classified samples in a group is less than a certain number (e.g., a predefined 4), the default values ​​mentioned in the discussion of matrix derivation for the MLR model can be applied to the group parameters (α i, ,β). If the corresponding adjacent reconstructed samples are not available for the selected LM mode, a default value may be applied, for example, when the MMLM_L mode is selected but the remaining samples are invalid.

[0150] Several methods related to simplification of GLM are introduced as follows to further improve the encoding and decoding efficiency.

[0151] Matrix / parameter derivation in FLM requires floating point operations (e.g., closed form division), which is expensive for decoder hardware, thus requiring a fixed point design. For the 1-tap GLM case, it can be viewed as a modified luminance reconstruction sample generation of CCLM (e.g., horizontal gradient direction, from CCLM[1,2,1; 1,2,1] / 8 to GLM[-1,0,1; -1,0,1]), and the original CCLM process can be reused for GLM, including fixed point operations, MDLM downsampling, division table, applied size restrictions, min-max approximation, and scaling. For all items, the 1-tap GLM can have its own configuration or share the same design with CCLM. For example, a simplified min-max method is used to derive parameters (instead of LMS) and combined with scaling after deriving the GLM model.

[0152] This section uses typical reference points (the top row and left column) as examples. Fig.14 As shown in , the extended reconstruction region can also use simplifications with the same essence and can have a syntax (such as MDLM, MRL) to indicate a specific region.

[0153] Note that the following aspects may be combined and commonly applied: For example, combining reference sample downsampling and a division table to perform a division process.

[0154] When applying classification (MMLM / ELM), each group can apply the same or different simplification operations. For example, fill the samples for each group to the target number of samples before applying the right shift, and then apply the same derivation process and the same division table.

[0155] Fixed-point implementation

[0156] The CCLM design can be reused for the 1-tap case, division by n can be implemented by right shift, and division by A2 can be implemented by LUT. α , ntable The integerization parameters and the intermediate parameters used to derive the linear model (Equations (19)-(20)) can be the same as CCLM or have different values ​​to have higher precision. The integerization parameters can be predefined or signaled / switched at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level and can be conditional on the sequence bit depth. For example, n table = bit depth + 4.

[0157] MDLM downsampling

[0158] When GLM is combined with MDLM, the total number of samples currently used for parameter derivation may not be a power of 2 and needs to be filled to a power of 2 to use a right shift operation instead of a division. For example, for an 8x4 chroma CU, MDLM requires W+H=12 samples, while MDLM_T has only 8 samples available (reconstructed), which can then be equally filled with the downsampled 4 samples (0, 2, 4, 6). The code for implementing such an operation is shown as follows: int targetSampNum=1<<(floorLog2(existSampNum-1)+1); if(targetSampNum!=existSampNum) / / if existSampNum not a value ofpower of 2 { xPadMdlmTemplateSample; } int step=(int)(existSampNum / sampNumToBeAdd); for(int i=0; i <sampNumToBeAdd;i++) { pTempSrc[i] = pSrc[i*step]; pTempCur[i]=pCur[i*step]; }

[0159] Other filling methods may also be applied, such as repeated / mirror filling relative to the last neighboring sample point (rightmost / bottommost).

[0160] The filling method used for GLM can be the same as or different from the filling method of CCLM.

[0161] Note that in the ECM version, 8x4 chroma CU MDLM_T / MDLM_L requires 2T / 2L=16 / 8 samples respectively, in which case the same padding method can be applied to meet the target power-of-2 number of samples.

[0162] Division LUT

[0163] The division LUT proposed for CCLM / LIC (local illumination compensation) in known standard developments such as AVC / HEVC / AV1 / VVC / AVS can be used for GLM division. For example, the LUT is reused in the case of JCTVC-I0166 for bit depth = 10 (Table 4). The division LUT may be different from CCLM. For example, CCLM uses the min-max method with DivTable as in Equation 5, but GLM uses a 32-entry LMS division LUT as in Table 5.

[0164] When combining GLM with MMLM, meanL values ​​may not always be positive (e.g., using filtered / gradient values ​​to classify groups), so sgn(meanL) needs to be extracted and abs(meanL) used to look up the division LUT. Note that the division LUT used for MMLM classification and parameter derivation can be different. For example, use a lower precision LUT (such as the LUT in min-max) for mean classification and a higher precision LUT (such as the LUT in LMS) for parameter derivation.

[0165] Size Limits and Latency Constraints

[0166] Similar to the CCLM design, some size constraints can be applied to the ELM / FLM / GLM. For example, the same constraints can be applied to the luma-chroma delay in the dual tree.

[0167] Size limits may be based on CU area / width / height / depth. Thresholds may be predefined or signaled at SPS / DPS / VPS / SEI / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, for chroma CU regions, the predefined threshold may be 128.

[0168] In one example, at least one pre-operation is performed in response to determining that a video block satisfies an enablement threshold, wherein the enablement threshold is associated with an area, width, height, or segmentation depth of the video block. Specifically, the enablement threshold may define a minimum or maximum area, width, height, or segmentation depth of the video block. As will be appreciated by those skilled in the art, a video block may include a current chroma block and its co-located luminance block. It is also proposed that the above-mentioned enablement threshold is applied jointly to the current chroma block and its co-located luminance block. For example, at least one pre-operation is performed in response to determining that an enablement threshold is satisfied for both the current chroma block and its co-located luminance block.

[0169] Line buffer reduction

[0170] Similar to the CCLM design, if the co-located luma region of the current chroma CU contains the first row within a CTU, the top template sample generation can be limited to 1 row to reduce the row buffer storage of CTU rows. Note that when the upper reference row is at the CTU boundary, only one luma row (normal row buffer in intra prediction) is used to generate the downsampled luma samples.

[0171] For example, in Fig.13 In , if the co-located luma region of the current chroma CU contains the first row within a CTU, the top template can be restricted to use only 1 row (instead of 2 rows) for parameter derivation (other CUs can still use 2 rows). This saves luma sample row buffer storage when processing CTUs row by row at the decoder hardware. Several methods can be used to achieve row buffer reduction. Note that the example of a limited "1" row can be extended to N rows using similar operations. Similarly, 2-tap or multi-tap can also apply such operations. When multi-tap is applied, chroma samples may also need to apply operations.

[0172] For example, using Fig.15 The 1-tap filter [1,0,-1,1,0,-1] shown in is explained as an example. The filter can be simplified to [0,0,0; 1,0,-1], i.e., only the lower row coefficients are used. Alternatively, the limited upper row luma samples can be filled (repeated, mirrored, 0, meanL, meanC, ..., etc.) from the lower row luma samples.

[0173] Take N=4 as an example, that is, the video block is located at the top boundary of the current CTU, and the adjacent luminance sample values ​​and the corresponding chrominance sample values ​​of the top 4 rows are used to derive the linear model. Note that the corresponding chrominance sample values ​​may refer to the corresponding adjacent chrominance sample values ​​of the top 4 rows (e.g., for the YUV 4:4:4 format). Alternatively, the corresponding chrominance sample values ​​may refer to the corresponding adjacent chrominance sample values ​​of the top 2 rows (e.g., for the YUV 4:2:0 format). In this case, the adjacent luminance sample values ​​and the corresponding chrominance sample values ​​of the top 4 rows may be divided into two regions-a first region including valid sample values ​​(e.g., the luminance sample values ​​and the corresponding chrominance sample values ​​of the nearest row) and a second region including invalid sample values ​​(e.g., the luminance sample values ​​and the corresponding chrominance sample values ​​of the other three rows). Then, the coefficients of the filter corresponding to the sample positions that do not belong to the first region may be set to zero, so that only the sample values ​​from the first region are used to calculate the sample difference. For example, as discussed above, in this case, the filter [1,0,-1; 1,0,-1] can be simplified to [0,0,0; 1,0,-1]. Alternatively, the closest sample values ​​in the first region can be padded to the second region so that the padded sample values ​​can be used to calculate the sample difference.

[0174] Fig.16 A workflow of a method 1600 for decoding video data according to one or more aspects of the present disclosure is shown.

[0175] At step 1610 , a video block (eg, a CU including a luma block and / or a chroma block) may be obtained from a bitstream.

[0176] At step 1620, neighboring luma and neighboring chroma sample values ​​of the video block may be obtained.

[0177] At step 1630, at least one pre-operation may be performed on the neighboring luma sample values ​​and on the internal luma sample values ​​in the video block to obtain pre-operated neighboring and internal luma sample values, wherein performing the at least one pre-operation includes calculating sample differences based on the neighboring and internal luma sample values.

[0178] At step 1640, a linear model may be derived by using the pre-operated neighboring luma sample values ​​and the neighboring chroma sample values.

[0179] At step 1650, one of the intra chroma sample values ​​in the video block may be predicted by applying a linear model to one or more corresponding pre-operated intra luma sample values ​​for each of the intra chroma sample values ​​in the video block.

[0180] At step 1660, the predicted intra chroma sample values ​​may be used to obtain a decoded video block.

[0181] Fig.17 A workflow of a method 1700 for encoding video data according to one or more aspects of the present disclosure is shown. The method 1700 may be similar to the method 1600, and the processes or steps of the method 1700 may correspond to the processes or steps of the method 1600.

[0182] At step 1710 , a video block (eg, a CU including a luma block and / or a chroma block) may be obtained from a video frame.

[0183] At step 1720, neighboring luma and neighboring chroma sample values ​​of the video block may be obtained.

[0184] At step 1730, at least one pre-operation may be performed on the neighboring luma sample values ​​and on the internal luma sample values ​​in the video block to obtain pre-operated neighboring and internal luma sample values, wherein performing the at least one pre-operation includes calculating sample differences based on the neighboring and internal luma sample values.

[0185] At step 1740, a linear model may be derived by using the pre-operated neighboring luma sample values ​​and the neighboring chroma sample values.

[0186] At step 1750, intra chroma sample values ​​in the video block may be predicted by applying a linear model to one or more corresponding pre-operated intra luma sample values ​​for each of the intra chroma sample values ​​in the video block.

[0187] At step 1760, a bitstream including an encoded video block may be generated using the predicted intra chroma sample values.

[0188] Fig.18 An exemplary computing system 1800 according to one or more aspects of the present disclosure is shown. The computing system 1800 may include at least one processor 1810. The computing system 1800 may also include at least one storage device 1820. The storage device 1820 may store computer executable instructions that, when executed, cause the processor 1810 to perform the steps of the above method. The processor 1810 may be a general-purpose processor, or may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration. The storage device 1820 may store input data, output data, data generated by the processor 1810, and / or instructions executed by the processor 1810.

[0189] It should be understood that the storage device 1820 may store computer-executable instructions that, when executed, cause the processor 1810 to perform any operations according to the embodiments of the present disclosure.

[0190] Embodiments of the present disclosure may be embodied in a computer-readable medium, such as a non-transitory computer-readable medium. The non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform any operation according to an embodiment of the present disclosure. For example, when the instructions are executed, one or more processors may receive a bitstream and perform a decoding operation as described above. For another example, when the instructions are executed, one or more processors may be caused to perform an encoding operation and send a bitstream including encoded video information associated with the predicted chroma samples as described above.

[0191] It should be understood that all operations in these methods are merely exemplary, and the present disclosure is not limited to any operations in these methods or the order of these operations, and should encompass all other equivalents under the same or similar concepts.

[0192] It should also be understood that all modules in these methods can be implemented in various ways. These modules can be implemented as hardware, software or a combination thereof. In addition, any module in these modules can be further divided into submodules or combined together functionally.

[0193] The previous description is provided so that any person skilled in the art can practice the various aspects described herein. It will be apparent to those skilled in the art that various modifications to these aspects will be apparent, and the general principles defined herein can be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or will be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims.

Claims

1. A method for decoding video data, comprising: Get video chunks from bitstream; Obtaining adjacent luminance sample values ​​and adjacent chrominance sample values ​​of the video block; performing at least one pre-operation on the neighboring luma sample values ​​and on intra luma sample values ​​in the video block to obtain pre-operated neighboring luma sample values ​​and intra luma sample values, wherein performing the at least one pre-operation comprises calculating sample differences based on the neighboring luma sample values ​​and the intra luma sample values; inferring a linear model by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values; predicting one of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luma sample values ​​for each of the intra chroma sample values ​​in the video block; as well as The predicted intra chroma sample values ​​are used to obtain a decoded video block.

2. The method according to claim 1, wherein Performing the at least one pre-operation further includes performing downsampling by a weighted averaging operation. The method of claim 2 , further comprising iteratively performing the at least one pre-operation.

4. The method according to claim 1, in, The linear model comprises a simple linear regression (SLR) model; and The step of predicting one of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luminance sample values ​​for each of the intra chroma sample values ​​in the video block comprises predicting the intra chroma sample value by using a corresponding pre-operated intra luminance sample value based on the SLR model.

5. The method according to claim 4, wherein Deriving the linear model includes: A scaling parameter α and an offset parameter β are derived by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values.

6. The method according to claim 5, wherein: The scaling parameter α is derived by using a division lookup table.

7. The method according to claim 5, wherein: Derivation of the scaling parameter α and the offset parameter β by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values ​​includes: Compare the adjacent luminance sample values ​​of the pre-operation to determine the minimum luminance sample value Y A and the maximum brightness sample value Y B ; For the minimum brightness sample value Y A and the maximum brightness sample value Y B Determine the corresponding chroma sample value X A and X B ;as well as According to the following equation, based on the minimum brightness sample value Y A , the maximum brightness sample value Y B And the corresponding chroma sample value X A and X B To derive the scaling parameter α and the offset parameter β: β=Y A -αX A 。 8. The method according to claim 5, wherein Deriving the linear model further includes: A scaling adjustment value is determined from the bitstream and added to the derived scaling parameter α.

9. The method according to claim 1, in, The linear model comprises a multiple linear regression (MLR) model; and The step of predicting one of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luminance sample values ​​for each of the intra chroma sample values ​​in the video block comprises predicting the intra chroma sample value by using the plurality of corresponding pre-operated intra luminance sample values ​​based on the MLR model.

10. The method according to claim 9, wherein: Deriving the linear model includes: A plurality of scaling parameters α and an optional offset parameter β are derived by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values.

11. The method according to claim 10, wherein: At least one of the plurality of scaling parameters α is derived by utilizing the sample difference.

12. The method according to claim 10, wherein: At least one of the plurality of scaling parameters α is derived by using a horizontal sample difference or a vertical sample difference calculated based on down-sampled adjacent luma sample values.

13. The method according to claim 1, in, Adjacent luma-chroma sample pairs and intra luma-chroma sample pairs of the video block are respectively classified into a plurality of groups based on one or more thresholds, and the one or more thresholds are associated with intensities of the adjacent luma samples and the intra luma samples; and The linear model is one linear model among a plurality of linear models, and each of the plurality of groups corresponds to a corresponding one linear model among the plurality of linear models.

14. The method according to claim 1, wherein Performing the at least one pre-operation further comprises: A Roberts, Prewitt, Compass, Sobel, or Laplacian operator is calculated based on the neighboring luma sample values ​​and the interior luma sample values.

15. The method according to claim 1, in, Calculating the sample differences based on the adjacent luma sample values ​​and the intra luma sample values ​​comprises applying a filter to the adjacent luma sample values ​​and the intra luma sample values; and The filter is fixed or switched for each coding unit (CU).

16. The method according to claim 15, wherein The coefficients of the filter are powers of 2.

17. The method according to claim 15, wherein: The filter is symmetric about chrominance positions corresponding to one or more luma samples including a co-located luma sample.

18. The method according to claim 15: in, The video block is located at a top boundary of a current coding tree unit (CTU), and top N rows of adjacent luma sample values ​​and corresponding chroma sample values ​​are used to derive the linear model; wherein the adjacent luminance sample values ​​and corresponding chrominance sample values ​​of the top N rows are divided into a first region and a second region; and Calculating the sample point difference further includes: Filling the closest sample point values ​​in the first area into the second area so that the filled sample point values ​​are used to calculate the sample point differences; or The coefficients of the filter corresponding to sample positions that do not belong to the first region are set to zero, so that only sample values ​​from the first region are used to calculate the sample differences.

19. The method according to claim 1, in, The at least one pre-operation is performed in response to determining that the video block satisfies an enablement threshold; and The enabling threshold is associated with a minimum or maximum area, width, height, or segmentation depth of the video block and is predefined or determined from the bitstream.

20. The method according to claim 19, wherein The video block includes a current chroma block and its co-located luma block, and determining that the video block satisfies the enablement threshold includes determining that the enablement threshold is satisfied for both the current chroma block and its co-located luma block.

21. The method according to claim 1, wherein The adjacent luma sample values ​​and the adjacent chroma sample values ​​of the video block are obtained by performing a padding operation on unavailable luma samples and chroma samples using the closest available luma sample values ​​and chroma sample values.

22. A method for encoding video data, comprising: Obtain video blocks from video frames; Obtaining adjacent luminance sample values ​​and adjacent chrominance sample values ​​of the video block; performing at least one pre-operation on the neighboring luma sample values ​​and on intra luma sample values ​​in the video block to obtain pre-operated neighboring luma sample values ​​and intra luma sample values, wherein performing the at least one pre-operation comprises calculating sample differences based on the neighboring luma sample values ​​and the intra luma sample values; inferring a linear model by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values; predicting one of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luma sample values ​​for each of the intra chroma sample values ​​in the video block; as well as A bitstream including the coded video block is generated by using the predicted intra chroma sample values.

23. The method according to claim 22, wherein Performing the at least one pre-operation further includes performing downsampling by a weighted averaging operation. The method of claim 23 , further comprising iteratively performing the at least one pre-operation.

25. The method according to claim 22, in, The linear model comprises a simple linear regression (SLR) model; and The step of predicting one of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luminance sample values ​​for each of the intra chroma sample values ​​in the video block comprises predicting the intra chroma sample value by using a corresponding pre-operated intra luminance sample value based on the SLR model.

26. The method according to claim 25, wherein Deriving the linear model includes: A scaling parameter α and an offset parameter β are derived by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values.

27. The method according to claim 26, wherein The scaling parameter α is derived by using a division lookup table.

28. The method according to claim 26, wherein Derivation of the scaling parameter α and the offset parameter β by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values ​​includes: Compare the adjacent luminance sample values ​​of the pre-operation to determine the minimum luminance sample value Y A and the maximum brightness sample value Y B ; For the minimum brightness sample value Y A and the maximum brightness sample value Y B Determine the corresponding chroma sample value X A and X B ;as well as According to the following equation, based on the minimum brightness sample value Y A , the maximum brightness sample value Y B And the corresponding chroma sample value X A and X B To derive the scaling parameter α and the offset parameter β: β=Y A -αX A 。 29. The method according to claim 26, wherein Deriving the linear model includes: A scaling adjustment value to be signaled in the bitstream is determined and added to the derived scaling parameter α.

30. The method according to claim 22, in, The linear model comprises a multiple linear regression (MLR) model; and The step of predicting one of the intra chroma sample values ​​in the video block by applying the linear model to one or more corresponding pre-operated intra luminance sample values ​​for each of the intra chroma sample values ​​in the video block comprises predicting the intra chroma sample value by using the plurality of corresponding pre-operated intra luminance sample values ​​based on the MLR model.

31. The method according to claim 30, wherein Deriving the linear model includes: A plurality of scaling parameters α and an optional offset parameter β are derived by using the pre-operated adjacent luma sample values ​​and the adjacent chroma sample values.

32. The method according to claim 31, wherein At least one of the plurality of scaling parameters α is derived by utilizing the sample difference.

33. The method according to claim 31, wherein At least one of the plurality of scaling parameters α is derived by using a horizontal sample difference or a vertical sample difference calculated based on down-sampled adjacent luma sample values.

34. The method according to claim 22, in, Adjacent luma-chroma sample pairs and intra luma-chroma sample pairs of the video block are respectively classified into a plurality of groups based on one or more thresholds, and the one or more thresholds are associated with intensities of the adjacent luma samples and the intra luma samples; and The linear model is one linear model among a plurality of linear models, and each of the plurality of groups corresponds to a corresponding one linear model among the plurality of linear models.

35. The method of claim 22, wherein: Performing the at least one pre-operation includes: A Roberts, Prewitt, Compass, Sobel, or Laplacian operator is calculated based on the neighboring luma sample values ​​and the interior luma sample values.

36. The method according to claim 22, in, Calculating the sample differences based on the adjacent luma sample values ​​and the intra luma sample values ​​comprises applying a filter to the adjacent luma sample values ​​and the intra luma sample values; and The filter is fixed or switched for each coding unit (CU).

37. The method according to claim 36, wherein The coefficients of the filter are powers of 2.

38. The method of claim 36, wherein: The filter is symmetric about chrominance positions corresponding to one or more luma samples including a co-located luma sample.

39. The method according to claim 36: in, The video block is located at a top boundary of a current coding tree unit (CTU), and top N rows of adjacent luma sample values ​​and corresponding chroma sample values ​​are used to derive the linear model; wherein the adjacent luminance sample values ​​and corresponding chrominance sample values ​​of the top N rows are divided into a first region and a second region; and Calculating the sample point difference further includes: Filling the closest sample point values ​​in the first area into the second area so that the filled sample point values ​​are used to calculate the sample point differences; or The coefficients of the filter corresponding to sample positions that do not belong to the first region are set to zero, so that only sample values ​​from the first region are used to calculate the sample differences.

40. The method according to claim 22, in, The at least one pre-operation is performed in response to determining that the video block satisfies an enablement threshold; and The enabling threshold is associated with a minimum or maximum area, width, height, or segmentation depth of the video block and is predefined or signaled in the bitstream.

41. The method according to claim 40, wherein The video block includes a current chroma block and its co-located luma block, and determining that the video block satisfies the enablement threshold includes determining that the enablement threshold is satisfied for both the current chroma block and its co-located luma block.

42. The method of claim 22, wherein: The adjacent luma sample values ​​and the adjacent chroma sample values ​​of the video block are obtained by performing a padding operation on unavailable luma samples and chroma samples using the closest available luma sample values ​​and chroma sample values.

43. A computer system comprising: one or more processors; as well as One or more storage devices storing computer-executable instructions that, when executed, cause the one or more processors to perform the operations of the method of one of claims 1-42.

44. A computer program product storing computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method of one of claims 1-42.

45. A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors to receive a bitstream and perform the operations of the method of one of claims 1-21 based on the bitstream.

46. ​​A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors to perform the operations of the method of one of claims 22-42 and to send a bitstream comprising encoded video information associated with the predicted chroma samples.