METHOD AND APPARATUS FOR CROSS-COMPONENT PREDICTION FOR VIDEO CODING - Patent application

Abstract

This disclosure provides a method for decoding video data, the method including obtaining a video block from a bitstream, determining a region for deriving a multiple linear regression (MLR) model, obtaining luma and chroma sample values ​​in the region, deriving the MLR model using the luma and chroma sample values ​​in the region, predicting each of the chroma samples in the video block by applying the MLR model to a corresponding luma sample for the chroma sample, and obtaining a decoded video block using the predicted chroma samples.

Description

[Technical field]

[0001] Aspects of the present disclosure relate generally to video coding and compression, and more particularly, to methods and apparatus for cross-component prediction techniques. [Background technology]

[0002] Various video coding techniques may be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (H.265 / HEVC), Advanced Video Coding (H.264 / AVC), Moving Picture Experts Group (MPEG) coding, and the like. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, and the like) that exploit redundancy present in a video image or sequence. An important objective of video coding techniques is to compress video data into a format that uses a lower bit rate while avoiding or minimizing degradation to the video quality.

[0003] To reduce cross-component redundancy, cross-component linear model (CCLM) prediction modes are commonly used in video coding standards, for which chroma samples are predicted based on reconstructed luma samples of the same CU. In existing CCLM or Multi-model LM (MMLM) prediction modes, one or more linear prediction models are used for prediction according to one or more sample groups into which the luma samples are classified. The classification in existing CCLM or MMLM prediction modes may generally take into account the luma DC value, which leaves potential room for other aspects that can further improve coding efficiency. Summary of the Invention

[0004] The following presents a simplified summary of one or more aspects according to the present disclosure in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, nor is it intended to identify key or critical elements of all aspects, nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0005] According to one embodiment, a method for decoding video data is provided, the method including obtaining a video block from a bitstream, determining a region for deriving a multiple linear regression (MLR) model, obtaining luma and chroma sample values ​​in the region, deriving the MLR model using the luma and chroma sample values ​​in the region, predicting each of the chroma samples in the video block by applying the MLR model to a corresponding luma sample for the chroma sample, and obtaining a decoded video block using the predicted chroma samples.

[0006] According to one embodiment, a method for encoding video data is provided, the method including obtaining a video block from a video frame, determining a region for deriving a multiple linear regression (MLR) model, obtaining luma and chroma sample values ​​in the region, deriving the MLR model using the luma and chroma sample values ​​in the region, predicting each of the chroma samples in the video block by applying the MLR model to a corresponding luma sample for the chroma sample, and obtaining an encoded video block using the predicted chroma samples.

[0007] According to one embodiment, a computer system is provided having one or more processors and one or more storage devices storing computer-executable instructions that, when executed, cause the one or more processors to perform operations including obtaining a video block from a bitstream; determining a region for deriving a multiple linear regression (MLR) model; obtaining luma and chroma sample values ​​in the region; deriving an MLR model using the luma and chroma sample values ​​in the region; predicting each of the chroma samples in the video block by applying the MLR model to a corresponding luma sample for that chroma sample; and obtaining a decoded video block using the predicted chroma samples.

[0008] By using the proposed multiple linear regression (MLR) model, the inter-sample correlations between the collocated luma samples, adjacent luma samples, and chroma samples to be predicted can be captured to further improve the prediction accuracy. Other advantages of the present disclosure will become apparent from the following description.

[0009] The disclosed aspects are described below in conjunction with the accompanying drawings, which are provided to illustrate, and not to limit, the disclosed aspects. [Brief description of the drawings]

[0010] [Figure 1] Figure 1 shows a block diagram of a typical block-based hybrid video coding system. [Diagram 2] 2A to 2E are diagrams showing five division types including a fourth division, a horizontal half division, a vertical half division, a horizontal third division, and a vertical third division. [Diagram 3] Figure 3 is a schematic block diagram of a block-based video decoder. [Figure 4] FIG. 4 shows an example of the location of the left and top samples and the samples of the current block involved in CCLM mode. [Diagram 5] FIG. 5 is a diagram showing an example of classifying adjacent samples into two groups based on a threshold value. [Figure 6] Figure 6 shows an example of classifying adjacent samples into two groups based on knee points. [Figure 7] FIG. 7 is a diagram illustrating a workflow of a method for decoding video data according to one or more aspects of the present disclosure. [Figure 8] FIG. 8 is a diagram illustrating a workflow of a method for encoding video data according to one or more aspects of the present disclosure. [Figure 9] 9A and 9B are schematic diagrams of correlation between a chroma sample and one or more luma samples. [Figure 10] FIG. 10 illustrates an example in which 6 taps are used in a multiple linear regression (MLR) model, in accordance with one or more aspects of the present disclosure. [Figure 11] FIG. 11 is a diagram illustrating a workflow of a method for encoding video data according to one or more aspects of the present disclosure. [Figure 12] FIG. 12 is a diagram illustrating a workflow of a method for decoding video data according to one or more aspects of the present disclosure. [Figure 13] FIG. 13 illustrates an exemplary computing system in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present disclosure will now be discussed with reference to several exemplary implementations. It should be understood that these implementations are not intended to imply any limitation on the scope of the present disclosure, but are merely discussed to enable those skilled in the art to better understand and thus implement embodiments of the present disclosure.

[0012] The first version of the VVC standard was finalized in July 2020, which gives approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video coding standard HEVC. While the VVC standard offers significant coding improvements over its predecessor, there is evidence that better coding efficiency can be achieved with additional coding tools. Recently, the Joint Video Exploration Team (JVET) under the collaboration of ITU-T VECG and ISO / IEC MPEG started exploring advanced techniques that could enable substantial extension of coding efficiency over VVC. In April 2021, a software code base, called the Enhanced Compression Model (ECM), was established for future video coding exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, with several existing modules (e.g., intra / inter prediction, transform, in-loop filters, etc.) further extended and / or improved. In the future, any new coding tools that go beyond the VVC standard will need to be incorporated into the ECM platform and tested using the JVET common test conditions (CTC).

[0013] Like all previous video coding standards, ECM is built on a block-based hybrid video coding framework. Figure 1 shows a block diagram of a general block-based hybrid video coding system. The input video signal is processed block by block (called coding unit (CU)). In ECM-1.0, a CU can be up to 128x128 pixels. However, like VVC, one coding tree unit (CTU) is divided into CUs based on quad / bin / tert-tree to adapt to varying local characteristics. In the multi-type tree structure, one CTU is first partitioned by a quad-tree structure. Then, each quad-tree leaf node can be further partitioned by bin and tert-tree structures. As shown in Figures 2A, 2B, 2C, 2D and 2E, there are five partition types: quadrant, vertical bisection, horizontal bisection, vertical extension quadrant and horizontal extension quadrant. In FIG. 1, spatial prediction and / or temporal prediction may be performed. Spatial prediction (or "intra prediction") predicts a current video block using pixels from samples of already coded neighboring blocks in the same video picture / slice (called reference samples). Spatial prediction reduces spatial redundancy inherent in video signals. Temporal prediction (also called "inter prediction" or "motion compensated prediction") predicts a current video block using reconstructed pixels from already coded video pictures. Temporal prediction reduces temporal redundancy inherent in video signals. A temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs) that indicate the amount and direction of motion between the current CU and its temporal references. Also, if multiple reference pictures are supported, one reference picture index is further sent, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from. After spatial and / or temporal prediction, a mode decision block in the encoder selects the best prediction mode, for example, based on a rate-distortion optimization method.The prediction block is then subtracted from the current video block, and the prediction residual is decorrelated and quantized using a transform. The quantized residual coefficients are inverse quantized and inverse transformed to form a reconstructed residual, which is then added to the prediction block to form a reconstructed signal for the CU. Further in-loop filtering, such as a deblocking filter, sample adaptive offset (SAO), and adaptive in-loop filter (ALF), is applied to the reconstructed CU before it is put into the reference picture store and used to code future video blocks. To form the output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and the quantized residual coefficients are all sent to an entropy coding unit for further compression and packing to form the bitstream.

[0014] FIG. 3 shows a schematic block diagram of a block-based video decoder. A video bitstream is first entropy decoded in an entropy decoding unit. Coding mode and prediction information are sent to either a spatial prediction unit (if intra-coded) or a temporal prediction unit (if inter-coded) to form a prediction block. Residual transform coefficients are sent to an inverse quantization unit and an inverse transform unit to reconstruct a residual block. The prediction block and the residual block are then summed. The reconstructed block may further go through in-loop filtering before it is stored in a reference picture store. The reconstructed video in the reference picture store is then sent to drive a display device and is also used to predict future video blocks.

[0015] To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VVC, in which a chroma sample is predicted based on the reconstructed luma sample of the same CU by using a linear model as follows: pred C (i,j)=α·rec L '(i,j)+β (1)

[0016] Here, pred C (i,j) represents the predicted chroma sample in CU, rec L '(i,j) is the reconstructed luma sample rec L Let x' denote the downsampled reconstructed luma sample of the same CU obtained by performing downsampling on (i,j), and α and β are linear model parameters derived from at most four adjacent chroma samples and their corresponding downsampled luma samples, which are sometimes called adjacent luma-chroma sample pairs. Assuming that the current chroma block has a size of W×H, W' and H' are obtained as follows:

[0017] When LM mode is applied, W'=W, H'=H.

[0018] When LM-A mode is applied, W'=W+H.

[0019] When the LM-L mode is applied, H'=H+W.

[0020] Here, in LM mode, the top and left samples of the CU are used together to calculate the linear model coefficients, in LM_A mode, only the top samples of the CU are used to calculate the linear model coefficients, and in LM_L mode, only the left samples of the CU are used to calculate the linear model coefficients.

[0021] If the location of the adjacent samples above a chroma block is denoted as S[0,-1]...S[W'-1,-1] and the location of the adjacent samples to the left of a chroma block is denoted as S[-1,0]...S[-1,H'-1], then the positions of the four adjacent chroma samples are selected as follows: - When LM mode is applied and both the top and left neighbor samples are available, S[W' / 4,-1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4] are selected as the positions of the 4 adjacent chroma samples. - When LM-A mode is applied or only the top adjacent samples are available, S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1], S[7*W' / 8,-1] are selected as the positions of the 4 adjacent chroma samples. - When LM-L mode is applied or only left adjacent samples are available, S[-1,H' / 8], S[-1,3*H' / 8], S[-1,5*H' / 8], S[-1,7*H' / 8] are selected as the positions of the 4 adjacent chroma samples.

[0022] The four adjacent luma samples corresponding to the selected location are obtained by the downsampling operation, and the obtained four adjacent luma samples are divided by the larger value of the two, i.e., x 0 A and x 1 A and two smaller values, namely x 0 B and x 1 B The chroma sample values ​​corresponding to the two larger and two smaller values ​​are y 0 A , y 1 A , y 0 B and y 1 B Then, Xa , X b , Y a and Y b is derived as follows: X a =(x 0 A +x 1 A +1)>>1, X b =(x 0 B +x 1 B +1)>>1, Y a =(y 0 A +y 1 A +1)>>1, Y b =(y 0 B +y 1 B +1)>>1 (2)

[0023] Finally, the linear model parameters α and β are obtained according to the following equations:

number

[0024] FIG. 4 shows an example of the locations of left and top samples and current block samples involved in CCLM mode, including the locations of left and top samples of an N×N chroma block in a CU and the locations of left and top samples of a 2N×2N luma block in a CU.

[0025] The division operation for calculating the parameter α is implemented by a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed by exponential notation. For example, the diff is approximated with a 4-bit significant part and an exponent. Thus, the table for 1 / diff is reduced to 16 elements for 16 values ​​of the mantissa as follows: DivTable[ ]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0026] This will have the benefit of both reducing the computational complexity and reducing the memory size required to store the required tables.

[0027] Besides the top and left templates being able to be used together to calculate the linear model coefficients, they can also be used alternatively in two other LM modes, called LM_A and LM_L modes.

[0028] In LM_T mode, only the top template is used to calculate the linear model coefficients. To get more samples, the top template is extended to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To get more samples, the left template is extended to (H+W) samples.

[0029] In LM_LT mode, the left and top templates are used to compute the linear model coefficients.

[0030] To match chroma sample locations for 4:2:0 video sequences, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, which correspond to "Type-0" and "Type-2" content, respectively:

number

[0031] Note that when the upper reference line is at a CTU boundary, only one luma line (a common line buffer in intra prediction) is used to make the downsampled luma samples.

[0032] This parameter computation is performed as part of the decoding process and not simply as an encoder search operation, and as a result, no syntax is used to communicate the α and β values ​​to the decoder.

[0033] For chroma intra mode coding, a total of eight intra modes are enabled for chroma intra mode coding, including five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chroma mode signaling and derivation process is shown in Table 1. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partition structures for luma and chroma components are enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block is directly inherited. [Table 1]

[0034] As shown in Table 2, a single binarization table is used regardless of the value of sps_cclm_enabled_flag. [Table 2]

[0035] In Table 2, the first bin indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next bin indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, the next one bin indicates whether it is LM_L (0) or LM_A (1). In this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table for the corresponding intra_chroma_pred_mode can be discarded before entropy coding. Or, in other words, the first bin is inferred to be 0 and therefore not coded. This single binarization table is used for both cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in Table 22 are context coded with their own context model, and the remaining bins are bypass coded.

[0036] Furthermore, to reduce luma-chroma latency in dual trees, when a 64x64 luma coding tree node is partitioned by Not Split (and ISP is not used for 64x64 CUs) or QT, the chroma CUs in a 32x32 / 32x16 chroma coding tree node are enabled to use CCLM as follows:

[0037] If a 32x32 chroma node is non-split or piecewise QT-split, all chroma CUs in the 32x32 node can use CCLM.

[0038] If a 32x32 chroma node is partitioned with horizontal BT and a 32x16 child node has no partition or uses vertical BT partition, all chroma CUs in the 32x16 chroma node can use CCLM.

[0039] In all other luma and chroma coding tree splitting conditions, CCLM is not enabled for the chroma CU.

[0040] During ECM unfolding, the simplified derivation of α and β (min-max approximation) is removed. Instead, it is a linear least-squares solution between the causal reconstructed data of the downsampled luma samples and the causal chroma samples to derive the model parameters α and β.

number

[0041] Here, Rec C (i) and Rec' L (i) denotes the reconstructed chroma samples and downsampled luma samples around the target block, and I denotes the total number of samples in the neighboring data.

[0042] In ECM-1.0, a multi-model LM (MMLM) prediction mode is proposed, in which a chroma sample is predicted based on the reconstructed luma samples of the same CU by using two linear models as follows:

number

[0043] Here, pred C (i,j) represents the predicted chroma sample in CU, rec L Let '(i,j) denote the downsampled reconstructed luma samples of the same CU. The threshold is calculated as the average value of the neighboring reconstructed luma samples. Figure 5 shows an example of classifying the neighboring samples into two groups based on the threshold value. For each group, we define a parameter α i and β iis the smallest luma sample in the group A(X A ,Y A ) and maximum luma samples B(X B ,Y B ), where X A , Y A are the x-coordinate (i.e., luma) and y-coordinate (i.e., chroma) values ​​for sample A, and X B , Y B are the x and y coordinate values ​​for sample B. The linear model parameters α and β are obtained according to the following equations:

number

[0044] Such a method is also called the min-max method. The divisions in the above formula can be avoided and replaced by multiplications and shifts.

[0045] For coding blocks with a square shape, the above two formulas are applied directly. For non-square coding blocks, the adjacent samples of the longer boundary are first subsampled so that they have the same number of samples as in the case of the shorter boundary.

[0046] Besides the scenario where the top template and the left template are used together to calculate the linear model coefficients, the two templates can also be alternatively used in two other MMLM modes, called MMLM_A and MMLM_L modes.

[0047] In MMLM_A mode, only pixel samples in the top template are used to calculate the linear model coefficients. To get more samples, the top template is expanded to a size of (W+W). In MMLM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To get more samples, the left template is expanded to a size of (H+H).

[0048] Note that when the upper reference line is at a CTU boundary, only one luma row (stored in the line buffer for intra prediction) is used to make the downsampled luma samples.

[0049] For chroma intra mode coding, a total of 11 intra modes are enabled for chroma intra mode coding, including five traditional intra modes and six cross-component linear model modes (CCLM, LM_A, LM_L, MMLM, MMLM_A, and MMLM_L). The chroma mode signaling and derivation process is shown in the table. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partition structures for luma and chroma components are enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block is directly inherited.

[0050] [Table 3]

[0051] Also, the MMLM and LM modes can be used together in an adaptive manner. For MMLM, the two linear models are:

number

[0052] Here, pred C (i,j) represents the predicted chroma sample in CU, rec L Let '(i,j) denote the downsampled reconstructed luma samples of the same CU. The threshold can be determined simply based on the luma and chroma mean values, along with their minimum and maximum values. Figure 6 shows an example of classifying neighboring samples into two groups based on the knee point T indicated by the arrow. The linear model parameters α1 and β1 are A ,Y A ) and threshold (X T ,Y T ) is derived from the linear relationship between the luma and chroma values ​​from two samples. The linear model parameters α2 and β2 are B ,Y B ) and threshold (X T ,Y T ), where X A , Y A are the x-coordinate (i.e., luma) and y-coordinate (i.e., chroma) values ​​for sample A, and X B , Y B are the x and y coordinate values ​​for sample B. The linear model parameters α for each group, where i is equal to 1 and 2, respectively i and β i is obtained according to the following formula:

number

[0053] For coding blocks with a square shape, the above formula is applied directly. For non-square coding blocks, the adjacent samples of the longer boundary are first subsampled so that they have the same number of samples as the shorter boundary.

[0054] Besides the scenario in which the top and left templates are used together to determine the linear model coefficients, the two templates may also be alternatively used in two other MMLM modes, referred to as the MMLM_A and MMLM_L modes, respectively.

[0055] In MMLM_A mode, only pixel samples in the top template are used to calculate the linear model coefficients. To get more samples, the top template is expanded to a size of (W+W). In MMLM_L mode, only pixel samples in the left template are used to calculate the linear model coefficients. To get more samples, the left template is expanded to a size of (H+H).

[0056] Note that when the upper reference line is at a CTU boundary, only one luma row (stored in the line buffer for intra prediction) is used to make the downsampled luma samples.

[0057] For chroma intra mode coding, there are condition checks that are used to select the LM modes (CCLM, LM_A, and LM_L) or the multi-model LM modes (MMLM, MMLM_A, and MMLM_L). The condition checks are as follows:

number

[0058] Here, BlkSizeThres LM represents the smallest block size in LM mode, and BlkSizeThresMM pred C (i,j) represents the smallest block size for MMLM mode. The symbol d represents a predefined threshold. In one example, d may take on a value of 0. In another example, d may take on a value of 8.

[0059] For chroma intra mode coding, a total of eight intra modes are enabled for chroma intra mode coding. They include five traditional intra modes and three cross-component linear model modes. The chroma mode signaling and derivation process is shown in Table 1. It is worth noting that for a given CU, if it is coded under a linear model mode, whether it is a traditional single model LM mode or an MMLM mode is determined based on the above condition check. Unlike the case shown in Table 3, there is no separate MMLM mode to be signaled. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partition structures for luma and chroma components are enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block is directly inherited.

[0060] In existing CCLM or MMLM designs, adjacent reconstructed luma-chroma sample pairs are classified into one or more sample groups based on a threshold value, which considers only the luma DC value. That is, the luma-chroma sample pairs are classified by considering only the intensity of the luma samples. However, the luma component usually preserves rich texture, and the current luma sample may be highly correlated with the neighboring luma samples, and such inter-sample correlation (AC correlation) may benefit the classification of the luma-chroma sample pairs, resulting in additional coding efficiency.

[0061] Furthermore, as shown in Fig. 9A, CCLM assumes that a given chroma sample is only correlated to the corresponding luma sample (which may be taken as a fractional luma sample position, L0.5), and a simple linear regression (SLR) with ordinary least squares (OLS) estimation is used to predict a given chroma sample. However, as shown in Fig. 9B, in some video content, one chroma sample may be simultaneously correlated to multiple luma samples (AC correlation or DC correlation), and thus a multiple linear regression (MLR) model may further improve the prediction accuracy.

[0062] The focus of this disclosure is to improve the coding efficiency of luma and chroma components by introducing a classifier that considers luma edge or AC information. Besides the existing band-classified MMLM, this disclosure provides an exemplary proposed classifier. The process of generating linear prediction models for different sample groups can be similar to CCLM or MMLM (e.g., via least squares or simplified min-max methods, etc.), but with different metrics for classification.

[0063] The focus of this disclosure is also to derive a filter-based linear model (FLM) to further improve the prediction accuracy.

[0064] In one aspect of the disclosure, a first classifier may be used to classify adjacent luma samples (e.g., of adjacent luma-chroma sample pairs) and / or luma samples corresponding to a chroma sample to be predicted. The luma samples corresponding to a chroma sample may be obtained by a downsampling operation to match the locations of the corresponding chroma samples for a 4:2:0 video sequence. For example, the luma samples corresponding to a chroma sample may be obtained by performing a downsampling operation on two or more (e.g., four) reconstructed luma samples corresponding to the chroma sample (e.g., located around the chroma sample). Alternatively, the luma samples may be obtained directly from the reconstructed luma samples, for example, in the case of a 4:4:4 video sequence. Alternatively, the luma samples may be obtained from each of the reconstructed luma samples at the respective co-located positions with respect to the corresponding chroma sample. For example, the luma sample to be classified may be obtained from one of the four reconstructed luma samples corresponding to the chroma sample at the top left position of the four reconstructed luma samples, which may be considered as a collocated position with respect to the chroma sample. The first classifier may classify the luma samples according to their edge strength. For example, one direction (e.g., 0 degrees, 45 degrees, or 90 degrees, etc.) may be selected to calculate the edge strength. The direction may be formed by the current sample and the neighboring sample along that direction (e.g., the neighboring sample located to the top right of the current sample for 45 degrees). The edge strength may be calculated by subtracting the neighbor sample from the current sample. The edge strength may be quantized into one of M segments by M-1 thresholds, and the first classifier may use M classes to classify the current sample. Alternatively or additionally, N directions may be formed by the current sample and N neighboring samples along the N directions. The N edge strengths may be calculated by subtracting N adjacent samples from the current sample, respectively.Similarly, if each of the N edge strengths can be quantized into one of M segments by M-1 thresholds, the first classifier will use M to classify the current sample. N classes may be used.

[0065] In another aspect of the present disclosure, a second classifier may be used to classify according to local patterns. For example, a current luma sample Y0 may be compared to its neighboring N luma samples Yi. If the value of Y0 is greater than the value of Yi, the score may be increased by 1, otherwise the score may be decreased by 1. The scores may be quantized to form K classes. The second classifier may classify the current sample into one of the K classes. For example, the neighboring luma samples may be taken from the four neighbors located above, to the left, to the right, and below the current luma sample, i.e., no diagonal neighbors.

[0066] In one or more aspects of the present disclosure, multiple first classifiers, second classifiers, or different instances of the first or second classifiers or other classifiers described herein may be combined. For example, a first classifier may be combined with an existing MMLM threshold-based classifier. In another example, an instance A of a first classifier may be combined with another instance B of the first classifier, where instance A and instance B adopt different orientations (e.g., adopt vertical and horizontal orientations, respectively).

[0067] Although the existing CCLM design in the VVC standard is used as the basic CCLM method in this specification, it will be appreciated by those skilled in the art that the proposed cross-component method described in this disclosure can also be applied to other predictive coding tools with similar design intent. For example, for chroma from luma (CfL) in the AV1 standard, the proposed method can also be applied by splitting luma / chroma sample pairs into multiple sample groups.

[0068] It will be appreciated by those skilled in the art that Y / Cb / Cr can also be denoted as Y / U / V in the video coding area. If the video data is in RGB format, the proposed method can also be applied, for example, by simply mapping the YUV representation to GBR.

[0069] FIG. 7 illustrates a workflow of a method 700 for decoding video data according to one or more aspects of the disclosure. The method 700 may use any one of the classifiers described herein, or any combination thereof, and may be used by a video decoder (e.g., of FIG. 3). At step 710, a video block (e.g., a CU) of video data may be obtained from a bitstream. For example, an encoded block of luma samples of the video data may be received. The encoded block of luma samples may be decoded to obtain a reconstructed luma sample. At step 720, the luma sample may be classified into one of a plurality of sample groups based on edge information of the luma sample. The luma sample may correspond to a chroma sample of the video block to be predicted and may be obtained from one or more of the reconstructed luma samples. For example, to obtain a luma sample, a downsampling operation may or may not be performed on the reconstructed luma sample. For example, the classification may be performed by using one of the classifiers described herein, or any combination thereof. In step 730, the chroma samples may be predicted by applying one of a plurality of linear prediction models corresponding to the classified sample group to the luma samples.

[0070] In one embodiment, the classification of the luma samples into one of a plurality of sample groups may be further based on the intensity value of the luma samples (eg, combined with an existing MMLM threshold-based classifier).

[0071] In one embodiment, each of the multiple sample groups may correspond to a different linear prediction model of the multiple linear prediction models.

[0072] In one embodiment, the edge information may include the direction and magnitude of the edges of the luma samples.

[0073] In one embodiment, classifying the luma sample into one of a plurality of sample groups may include classifying the luma sample into one of a plurality of sample groups based on a strength of an edge along a direction for the luma sample, or a plurality of strengths of edges along different directions for the luma sample.

[0074] In one embodiment, the method 700 may include classifying adjacent luma samples around the video block into a plurality of sample groups based on edge information of the adjacent luma samples, each of the plurality of linear prediction models may be derived from adjacent luma samples classified into a sample group corresponding to the linear prediction model and adjacent chroma samples corresponding to the adjacent luma samples classified into the sample group (e.g., adjacent luma-chroma sample pairs in which the luma samples are classified into a sample group corresponding to the linear prediction model), for example, through a least squares method, a simplified min-max method, or the like.

[0075] 8 illustrates a workflow of a method 800 for encoding video data according to one or more aspects of the present disclosure. The method 800 may use any one of the classifiers described herein, or any combination thereof, and may be used by a video encoder (e.g., of FIG. 1). In step 810, a video block of video data may be obtained from a video frame. For example, a block of luma samples of the video data may be encoded to obtain an encoded block of luma samples. The encoded block of luma samples may be decoded to obtain a reconstructed luma sample. In step 820, a luma sample corresponding to a chroma sample may be classified into one of a plurality of sample groups based on edge information of the luma sample, and the luma sample is obtained from one or more of the reconstructed luma samples. In step 830, the chroma sample may be predicted by applying one of a plurality of linear prediction models corresponding to the classified sample group to the luma sample.

[0076] In one or more aspects of the present disclosure, for a chroma sample to be predicted, the reconstructed co-located and adjacent luma samples may be used to predict that chroma sample, capturing inter-sample correlations between the co-located luma sample, the adjacent luma samples, and the chroma samples. The reconstructed luma samples are linearly weighted and combined with one “offset” to generate a predicted chroma sample (C: predicted chroma sample, L: predicted chroma sample). i : the i-th reconstructed collocated or adjacent luma sample, α i : filter coefficient, β: offset, N: filter taps). Note that the linearly weighted + offset values ​​directly form the predicted chroma sample (which can be adaptively low pass, high pass according to the video content), which is then added with the residual to form the reconstructed chroma sample.

number

[0077] For a given CU, the FLM parameters (α i The top and left reconstructed luma and chroma samples can be used to derive or train α,β,β. Similar to CCLM, α i and β can be derived via OLS. The top and left training samples are collected and one pseudo-inverse matrix is ​​calculated at both the encoder and decoder sides to derive the parameters, which are then used to predict the chroma samples in a given CU. Let N denote the number of filter taps applied to the luma samples, and M denote the total top and left reconstructed luma and chroma sample pairs used to train the parameters,

number

number

[0078] Alpha without offset beta i Note that this may be a subset of the proposed method, as it is possible to predict chroma samples only by

[0079] Although the existing CCLM design in the VVC standard is used as the basic CCLM method in the following description, those skilled in the art of video coding should note that the proposed cross-component method described in this disclosure can also be applied to other predictive coding tools with similar design intent. For example, for chroma from luma (CfL) in the AV1 standard, the proposed FLM can also be applied by including multiple luma samples to the MLR model.

[0080] Note that Y / Cb / Cr can also be denoted as Y / U / V in the video coding domain.

[0081] Note that if the video is in RGB format, the proposed FLM can also be applied, for example, by simply mapping the YUV representation in the following paragraphs to GBR.

[0082] It should be noted that the figures of this disclosure may be combined with all examples referred to in this disclosure.

[0083] FIG. 11 illustrates a workflow of a method 1100 for encoding video data according to one or more aspects of the present disclosure. At step 1110, a video block (e.g., a CU comprising a luma block and / or a chroma block) may be obtained from a video frame. At step 1120, a region used to derive a multiple linear regression (MLR) model may be determined. For example, the region may include one or more columns to the left and / or one or more rows above the reconstructed chroma samples and luma samples adjacent to the chroma block and its co-located luma block, e.g., three columns to the left and two rows above the reconstructed luma samples adjacent to the co-located luma block and one column to the left and one row above the chroma block, as shown in FIG. 10.

[0084] In step 1130, luma and chroma sample values ​​for the region may be obtained. For example, the reconstructed luma and chroma sample values ​​for the region may be used directly without modification, such as downsampling.

[0085] In step 1140, an MLR model may be derived using the luma and chroma sample values ​​in the region. The MLR model includes at least two coefficients, α i (i=0...N-1, where N is the number of filter taps) and one offset β. For example, to use six luma samples and one offset to predict one chroma sample, six coefficients α i (i.e., the number of filter taps is 6, i = 0...5) and an offset β can be derived. The derivation of the MLR model can include pseudo-inverse matrix calculations and normal equations, and to derive the scalar β, + The rightmost column in must be set to 1.

[0086] At step 1150, each of the chroma samples in the video block may be predicted by applying the MLR model to a corresponding luma sample for that chroma sample. The corresponding luma sample for each chroma sample (e.g., chroma sample 0' shown in FIG. 10) may include co-located luma samples for that chroma sample (e.g., luma samples 0', 1', 2', 3', 4', and 5' shown in FIG. 10). For example, the values ​​of the reconstructed co-located luma samples may be used directly without modification, such as downsampling.

[0087] At step 1160, an encoded video block may be obtained using the predicted chroma samples. For example, the encoded video block may be included in a bitstream for transmission or storage.

[0088] FIG. 12 illustrates a workflow of a method 1200 for decoding video data according to one or more aspects of the present disclosure. The method 1200 may be similar to the method 1100, and the processes or steps of the method 1200 may correspond to the processes or steps of the method 1100. In step 1210, a video block (e.g., a CU) may be obtained from a bitstream. In step 1220, a region used to derive a multiple linear regression (MLR) model may be determined. In step 1230, luma sample values ​​and chroma sample values ​​in the region may be obtained. In step 1240, an MLR model may be derived using the luma sample values ​​and chroma sample values ​​in the region. In step 1250, each of the chroma samples in the video block may be predicted by applying the MLR model to the corresponding luma sample for that chroma sample. In step 1260, a decoded video block may be obtained using the predicted chroma samples.

[0089] FIG. 13 illustrates an exemplary computing system 1300 according to one or more aspects of the present disclosure. The computing system 1300 may include at least one processor 1310. The computing system 1300 may further include at least one storage device 1320. The storage device 1320 may store computer-executable instructions that, when executed, cause the processor 1310 to perform the steps of the methods 700, 800, 1100, and 1200 described above with reference to FIGS. 7-8 and 11-12. The processor 1310 may be a general-purpose processor or may be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The storage device 1320 may store input data, output data, data generated by the processor 1310, and / or instructions executed by the processor 1310.

[0090] It should be appreciated that the storage device 1320 may store computer-executable instructions that, when executed, cause the processor 1310 to perform any of the operations according to the embodiments of the present disclosure described in connection with Figures 1-12.

[0091] An embodiment of the present disclosure may be embodied in a computer-readable medium, such as a non-transitory computer-readable medium. The non-transitory computer-readable medium may comprise instructions that, when executed, cause one or more processors to perform any of the operations according to the embodiments of the present disclosure described in connection with FIGs. 1-12. For example, the instructions, when executed, may cause one or more processors to receive a bitstream and perform a decoding operation, as described above. In another example, the instructions, when executed, may cause one or more processors to perform an encoding operation, as described above, and to transmit a bitstream including encoded video information related to predicted chroma samples.

[0092] It should be appreciated that all operations in the methods described above are examples only, and that the present disclosure is not limited to any operations in the methods, or to the sequence order of those operations, but should cover all other equivalents under the same or similar concepts.

[0093] It should also be appreciated that all of the modules in the methods described above may be implemented in various ways. These modules may be implemented as hardware, software, or a combination thereof. Moreover, any of these modules may be further functionally divided into sub-modules or combined together.

[0094] The previous description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not limited to the aspects shown herein. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or that later become known to those skilled in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims.

Claims

1. 1. A method for decoding video data, comprising: Obtaining a video block from the bitstream; determining a region for deriving a multiple linear regression (MLR) model; obtaining luma and chroma sample values ​​in the region; deriving the MLR model using the luma and chroma sample values ​​in the region; predicting each chroma sample in the video block by applying the MLR model to a corresponding luma sample for that chroma sample; obtaining a decoded video block using the predicted chroma samples; and A method comprising:

2. The MLR model has at least two coefficients α ｉ 2. The method of claim 1, comprising: (i=0 . . . N-1, where N is the number of filter taps) and one offset β.

3. 2. The method of claim 1 , wherein the region comprises one or more columns to the left and / or one or more rows above reconstructed chroma samples and corresponding luma samples adjacent to the video block.

4. The obtaining of luma and chroma sample values ​​in the region includes: using the reconstructed chroma sample and corresponding luma sample values ​​directly, without modification; The method of claim 3 , comprising:

5. deriving the MLR model using the luma and chroma sample values ​​in the region; To predict one chroma sample using six luma samples and one offset, six coefficients α ｉ Derive an MLR model with (i = 0...5) and one offset β The method of claim 1 , comprising:

6. deriving the MLR model using the luma and chroma sample values ​​in the region; Deriving the MLR model using pseudoinverse matrix calculations and normal equations. The method of claim 1 , comprising:

7. The predicting each of the chroma samples in the video block comprises: Use the corresponding reconstructed luma sample values ​​directly, without modification. The method of claim 1 , comprising:

8. 1. A method for encoding video data, comprising: Obtaining a video block from a video frame; determining a region for deriving a multiple linear regression (MLR) model; obtaining luma and chroma sample values ​​in the region; deriving the MLR model using the luma and chroma sample values ​​in the region; predicting each chroma sample in the video block by applying the MLR model to a corresponding luma sample for that chroma sample; obtaining an encoded video block using the predicted chroma samples; and A method comprising:

9. The MLR model has at least two coefficients α ｉ 9. The method of claim 8, comprising: (i=0 . . . N-1, where N is the number of filter taps) and one offset β.

10. 10. The method of claim 8, wherein the region comprises one or more columns to the left and / or one or more rows above reconstructed chroma samples and corresponding luma samples adjacent to the video block.

11. The obtaining of luma and chroma sample values ​​in the region includes: using the reconstructed chroma sample and corresponding luma sample values ​​directly, without modification; The method of claim 10, comprising:

12. deriving the MLR model using the luma and chroma sample values ​​in the region; To predict one chroma sample using six luma samples and one offset, six coefficients α ｉ Derive an MLR model with (i = 0...5) and one offset β The method of claim 8 , comprising:

13. deriving the MLR model using the luma and chroma sample values ​​in the region; Deriving the MLR model using pseudoinverse matrix calculations and normal equations. The method of claim 8 , comprising:

14. The predicting each of the chroma samples in the video block comprises: Use the corresponding reconstructed luma sample values ​​directly, without modification. The method of claim 8 , comprising:

15. one or more processors; One or more storage devices storing computer-executable instructions that, when executed, cause the one or more processors to perform operations of the method of any one of claims 1 to 14; A computer system comprising:

16. 15. A computer program product storing computer-executable instructions that, when executed, cause one or more processors to perform operations of the method of any one of claims 1 to 14.

17. A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors to receive a bitstream and perform operations of a method according to any one of claims 1 to 7 based on the bitstream.

18. 15. A computer-readable medium storing computer-executable instructions that, when executed, cause one or more processors to perform operations of the method of any one of claims 8 to 14 and to transmit a bitstream including encoded video information related to predicted chroma samples.

Images