Linear model derivation for cross-component prediction through multiple reference lines

Through the cross-component prediction technology of multiple reference lines, the problem of low pixel block encoding and decoding efficiency in the prior art is solved, and a more efficient video encoding and decoding effect is achieved.

CN119096542APending Publication Date: 2024-12-06MEDIATEK INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380036558.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-26
Filing Date
2023-04-26
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When processing pixel blocks, existing video encoding and decoding technologies are difficult to effectively use multiple reference lines to predict cross-components, resulting in low encoding and decoding efficiency.

Method used

Multiple reference lines are used for cross-component prediction, chromaticity prediction samples are derived from multiple reference lines through a linear model, and the fused brightness and chromaticity samples are used for encoding or decoding, and downsampling and filtering are performed when necessary to optimize parameters. calculate.

Benefits of technology

The efficiency and quality of video encoding and decoding are improved, and the encoding and decoding artifacts are reduced through the use of multiple reference lines, which enhances the matching prediction ability of pixel block boundaries.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096542A_ABST
    Figure CN119096542A_ABST
Patent Text Reader

Abstract

A video coding method for cross-component prediction using multiple reference lines is provided. A video codec receives data for a block of pixels to be encoded or decoded as a current block of a current picture. A video codec derives a linear model based on luma and chroma samples from one or more reference lines selected from among a plurality of reference lines adjacent to a current block at different distances. A video codec applies a linear model to a luma sample of a current block to obtain a chroma prediction sample of the current block. A video codec encodes or decodes a current block by using chroma prediction samples of the current block. The linear model may be derived using luma and chroma samples of only one selected reference line or two or more selected reference lines.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is part of a non-provisional application that claims priority to U.S. Provisional Patent Application No. 63 / 334,733, filed on April 26, 2022. The contents of the above-listed applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to video coding and decoding. In particular, the present disclosure relates to coding and decoding pixel blocks by cross-component prediction. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video codec standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion compensation DCT transform-like codec architecture. The basic unit of compression, called a coding unit (CU), is a 2Nx2N square pixel block. Each CU can be recursively divided into four smaller CUs until a predetermined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile video coding (VVC) is the latest international video codec standard developed by the Joint Video Expert Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from a reconstructed signal derived from the coded picture region. The prediction residual signal is processed by block transform. The transform coefficients are quantized and entropy coded along with other auxiliary information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by loop filtering to remove codec artifacts. The decoded picture is stored in a frame buffer and used to predict future pictures in the input video signal.

[0007] In VVC, a coded picture is divided into non-overlapping square block areas represented by associated coding tree units (CTUs). A coded picture can be represented by a set of fragments, each fragment containing an integer number of CTUs. The CTUs in a fragment are processed continuously in a raster scan. Intra-frame prediction or inter-frame prediction can be used to decode bi-predictive (B) fragments, where there are at most two motion vectors and reference indices to predict the sample values ​​of each block. Prediction (P) fragments are decoded using intra-frame prediction or inter-frame prediction with at most one motion vector and reference index to predict the sample values ​​of each block. Intra-frame (I) fragments are decoded using only intra-frame prediction.

[0008] CTU can be divided into one or more non-overlapping coding units (CU) using a quadtree (QT) with a nested multi-type-tree (MTT) structure to accommodate various local motion and texture features. CU can be further divided into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center side ternary tree partitioning, and horizontal center side ternary tree partitioning.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, serves as the basic unit for sending prediction information. The specified prediction process is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) for representing prediction residual blocks. A transform unit (TU) consists of a transform block (TB) of luminance samples and two corresponding transform blocks of chrominance samples, each TB corresponding to a residual block of samples from one color component. Integer transform is applied to the transform block. The level value of the quantization coefficient is entropy encoded and decoded in the bitstream together with other auxiliary information. The terms coding tree block (CTB), coding block (CB), prediction block (PB) and transform block (TB) are defined as 2D sample arrays specifying a color component associated with CTU, CU, PU and TU, respectively. Therefore, a CTU consists of a luminance CTB, two chrominance CTBs and related syntax elements. Similar relationships are valid for CU, PU and TU.

[0010] For each inter-predicted CU, motion parameters consisting of motion vector, reference picture index and reference picture list usage index as well as additional information are used for inter-prediction sample generation. Motion parameters can be sent explicitly or implicitly. When a CU is encoded or decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no motion vector increments or reference picture indices are encoded or decoded. Merge mode is specified and the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is the explicit transmission of motion parameters, where motion vectors, corresponding reference picture indices for each reference picture list and reference picture list usage flags and other required information are explicitly sent per CU. Summary of the invention

[0011] The following summary is illustrative only and is not intended to be binding in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits and advantages of the novel and non-obvious technologies described herein. Selected but not all embodiments are further described in the detailed description below. Therefore, the following summary is not intended to identify the essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0012] A video codec method for cross-component prediction using multiple reference lines is provided. A video codec receives data of a pixel block, which is to be encoded or decoded as a current block of a current picture. The video codec derives a linear model based on luminance and chrominance samples from one or more reference lines, wherein the one or more reference lines are selected from multiple reference lines adjacent to the current block at different distances. The video codec applies the linear model to the luminance samples of the current block to obtain chrominance prediction samples of the current block. The video codec encodes or decodes the current block by using the chrominance prediction samples of the current block.

[0013] In some embodiments, at least one of the selected reference lines is adjacent to a boundary of the current block. In some embodiments, at least one of the selected reference lines is not adjacent to a boundary of the current block. In some embodiments, the linear model is derived using luminance and chrominance samples of only one selected reference line. In some embodiments, the linear model is derived using luminance and chrominance samples of two or more selected reference lines. The two or more selected reference lines may be adjacent to each other or may not be adjacent to each other. In some embodiments, the luminance samples and chrominance samples used to derive the linear model may have the same or different selected reference lines. In some embodiments, the luminance samples used to derive the linear model are subsampled without applying downsampling filtering.

[0014] In some embodiments, the encoder sends a syntax element indicating whether to select more than one reference line from the plurality of reference lines for deriving the linear model. In some embodiments, the plurality of reference lines selected from the plurality of reference lines is determined based on a size of the current block or a cross-component prediction mode for the current block.

[0015] In some embodiments, luma samples in multiple reference lines are combined into a fused luma sample line, and chroma samples in multiple reference lines are combined into a fused chroma sample line. A linear model is derived based on the fused luma sample line and the fused chroma sample line. Each fused sample is a weighted average of samples within a window containing multiple reference lines.

[0016] In some embodiments, the linear model is derived from samples of a particular reference line that is selected from a plurality of reference lines and has a minimum boundary matching cost. The boundary matching cost of the reference line is based on the difference between (i) boundary chrominance samples of the current block predicted by the candidate model for the reference line (by applying the candidate model to boundary luminance samples of the current block) and (ii) reconstructed chrominance samples adjacent to the current block. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate embodiments of the present disclosure and together with the description are used to explain the principles of the present disclosure. It is worth noting that the drawings are not necessarily drawn to scale, as certain components may be shown out of proportion to size in actual implementations in order to clearly illustrate the concepts of the present disclosure.

[0018] Figure 1 Chroma and luminance samples used to derive linear model parameters are conceptually shown.

[0019] Figure 2 An example of dividing adjacent samples into two groups is shown.

[0020] Figure 3 Multiple reference lines for the current block used for cross-component prediction are shown.

[0021] Figure 4 The use of luma and chroma samples to derive model parameters is shown without luma subsampling.

[0022] Figure 5A-5B Conceptually illustrated is the combination of multiple adjacent reference lines into one line to derive model parameters for cross-component prediction.

[0023] Figure 6 The implicit selection of a reference line for the current block is conceptually shown.

[0024] Figure 7An example video encoder is shown that may implement cross-component prediction.

[0025] Figure 8 A portion of a video encoder that implements linear model derivation based on multiple reference lines is shown.

[0026] Fig. 9 A process of performing cross-component prediction using a plurality of reference lines is conceptually illustrated.

[0027] Fig.10 An example video decoder is shown that may implement cross-component prediction.

[0028] Fig.11 A portion of a video decoder that implements linear model derivation based on multiple reference lines is shown.

[0029] Fig.12 A process of performing cross-component prediction using a plurality of reference lines is conceptually illustrated.

[0030] Fig.13 An electronic system for implementing some embodiments of the present disclosure is conceptually illustrated. DETAILED DESCRIPTION

[0031] In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. Any variations, derivations, and / or extensions based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, processes, components, and / or circuits associated with one or more example embodiments disclosed herein may be described at a relatively high level without details to avoid unnecessarily obscuring aspects of the teachings of the present disclosure.

[0032] 1. Cross Component Linear Model (CCLM)

[0033] Cross Component Linear Model (CCLM) or Linear Model (LM) mode is a cross component prediction mode in which the chrominance components of a block are predicted from the co-located reconstructed luminance samples by a linear model. The parameters of the linear model (e.g., scale and offset) are derived from the reconstructed luminance and chrominance samples adjacent to the block. For example, in VVC, the CCLM mode uses inter-channel dependencies to predict chrominance samples from reconstructed luminance samples. The prediction is performed using a linear model of the following form:

[0034] P(i, j) = α·rec′ L (i, j)+β Equation (1)

[0035] P(i, j) in equation (1) represents the chroma prediction sample in the CU (or the chroma prediction sample of the current CU), rec′ L (i, j) represents the downsampled reconstructed luma samples of the same CU (or the corresponding reconstructed luma samples of the current CU).

[0036] The CCLM model parameters α (scaling parameter) and β (offset parameter) are derived based on up to four adjacent chroma samples and their corresponding downsampled luma samples. In LM_A mode (also known as LM-T mode), only the upper or top adjacent template is used to calculate the linear model coefficients. In LM_L mode (also known as LM-L mode), only the left template is used to calculate the linear model coefficients. In LM-LA mode (also known as LM-LT mode), both the left and upper templates are used to calculate the linear model coefficients.

[0037] Figure 1 Conceptually illustrating chrominance and luma samples used to derive linear model parameters. The figure shows a current block 100 with luma component samples and chroma component samples in a 4:2:0 format. The luma and chroma samples adjacent to the current block are reconstructed samples. These reconstructed samples are used to derive the cross-component linear model (parameters α and β). Since the current block is in a 4:2:0 format, the luma samples are first downsampled before being used for linear model derivation. In this example, there are 16 pairs of reconstructed luma (downsampled) and chroma samples adjacent to the current block. These 16 pairs of luma and chroma values ​​are used to derive linear model parameters.

[0038] Assuming the current chroma block size is W×H, W′ and H′ are set to:

[0039] When the LM-LT mode is applied, W′=W, H′=H;

[0040] When the LM-T mode is applied, W′=W+H;

[0041] When the LM-L mode is applied, H′=H+W

[0042] The upper neighboring position is denoted as S[0, -1] ... S[W'-1, -1] and the left neighboring position is denoted as S[-1, 0] ... S[-1, H'-1]. Then four samples are selected as:

[0043] When LM mode is applied (both upper and left neighboring samples are available), S[W' / 4, -1], S[3*W' / 4, -1], S[-1, H' / 4], S[-1, 3*H' / 4];

[0044] When LMT mode is applied (only upper adjacent samples are available), S[W' / 8, -1], S[3*W' / 8, -1], S[5*W' / 8, -1], S[7*W' / 8, -1];

[0045] When LM-L mode is applied (only left adjacent samples are available), S[-1, H' / 8], S[-1, 3*H' / 8], S[-1, 5*H' / 8], S[-1, 7*H' / 8];

[0046] The four adjacent luma samples at the selected position are downsampled and compared four times to find two larger values: x0A and x1A, and two smaller values: x0B and x1B. Their corresponding chroma sample values ​​are denoted as y0A, y1A, y0B, and y1B. Then XA, XB, YA, and YB can be derived as:

[0047] Xa=(x0A+x1A+1)>>1;Xb=(x0B+x1B+1)>>1; Equation (2)

[0048] Ya=(y0A+y1A+1)>>1; Yb=(y0B+y1B+1)>>1 Equation (3)

[0049] The linear model parameters α and p are obtained according to the following equations:

[0050]

[0051] β=Y b -α·X b

[0052] Equation (5)

[0053] The operation of calculating the α and β parameters according to equations (4) and (5) can be implemented by a lookup table. In some embodiments, in order to reduce the memory required to store the lookup table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated by a 4-bit significant part and an exponent. Therefore, the table of 1 / diff is reduced to 16 elements of 16 significant values, as shown below:

[0054] DivTable[] = {0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0} Equation (6)

[0055] This reduces the complexity of the computation and the size of memory required to store the required tables.

[0056] In some embodiments, in order to obtain more samples to calculate the CCLM model parameters α and β, for the LM-T mode, the upper template is expanded to include (W+H) samples, and for the LM-L mode, the left template is expanded to include (H+W) samples. For the LM-LT mode, both the expanded left template and the expanded upper template are used to calculate the linear model coefficients.

[0057] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2 to 1 downsampling ratio in both the horizontal and vertical directions. The choice of downsampling filter is specified by the sequence parameter set (SPS) level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0058] rec ′ L(i, j) = [rec L (2i-1, 2j-1)+2j-1)+2*rec L (2i-1, 2j-1)+rec L (2i+1,2j-1)+rec L (2i-1,2j)+rec L (2i+1,2j)+4]>>3

[0059] Equation (7)

[0060] rec′ L (i, j) = [rec L (2i, 2j-1)+rec L (2i-1,2j)+4*rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i, 2j+1)+4]>>3

[0061] Equation (8)

[0062] In some embodiments, when the upper reference line is located at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to generate downsampled luma samples.

[0063] In some embodiments, the α and β parameter calculations are performed as part of the decoding process, rather than just as an encoder search operation. Thus, no syntax is used to communicate the α and β values ​​to the decoder.

[0064] For chroma intra mode codec, a total of 8 intra modes are allowed. These modes include five traditional intra modes and three cross-component linear model modes (LM_LA, LM_A, and LM_L). Chroma intra mode codec can directly depend on the intra prediction mode of the corresponding luma block. Chroma intra mode signaling and the corresponding luma intra prediction mode are shown in the following table:

[0065]

[0066] Since separate block partitioning structures for luma and chroma components are enabled in the I segment, one chroma block can correspond to multiple luma blocks. Therefore, for the chroma derived mode (DM), the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0067] A single unified binarization table (mapped to bin strings) is used for chroma intra prediction modes according to the following table:

[0068]

[0069]

[0070] In the table, the first bin indicates whether it is normal (0) or LM mode (1). If it is LM mode, the next bin indicates whether it is LM_CHROMA (0). If it is not LM_CHROMA, the next bin indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table corresponding to intra_chroma_pred_mode can be discarded before entropy encoding and decoding. Or, in other words, the first bin is inferred to be 0 and therefore not encoded and decoded. This single binarization table is used for the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in the table use their own context model for context encoding and decoding, and the remaining bins are bypass encoded and decoded.

[0071] Additionally, to reduce luma-chroma latency in dual trees, when the 64x64 luma codec tree node is not split (and ISP is not used for 64x64 CU) or is split with QT, the chroma CU nodes in the 32x32 / 32x16 chroma codec tree are allowed to use CCLM as follows:

[0072] If a 32x32 chroma node is not split or is split using QT splitting, all chroma CUs in the 32x32 node can use CCLM

[0073] If the 32x32 chroma node uses horizontal BT partitioning, and the 32x16 child node is not split or uses vertical BT partitioning, all chroma CUs in the 32x16 chroma node can use CCLM.

[0074] In all other luma and chroma codec tree split conditions, CCLM is not allowed for chroma CUs.

[0075] II. Multi-model CCLM (MMLM)

[0076] The multi-model CCLM mode (MMLM) uses two models to predict chrominance samples using the luma samples of the entire CU. Similar to CCLM, three multi-model CCLM modes (MMLM_LA, MMLM_A, and MMLM_L) are used to indicate whether to use both upper and left neighboring samples, only upper neighboring samples, or only left neighboring samples in model parameter derivation.

[0077] In MMLM, the adjacent luma samples and adjacent chroma samples of the current block are divided into two groups, each of which is used as a training set to derive a linear model (i.e., a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules as the classification of the adjacent luma samples.

[0078] Figure 2 An example of dividing neighboring samples into two groups is shown. The threshold is calculated as the average of neighboring reconstructed luminance samples. Neighboring samples at [x, y] with Rec'L[x, y] <= threshold are classified into group 1; and neighboring samples at [x, y] with Rec'L[x, y] > threshold are classified into group 2. Therefore, the multi-model CCLM prediction of chrominance samples is:

[0079] Pred c [x, y] = α 1 ×Rec′ L [x,y]+β 1 If Rec′ L [x, y] ≤ threshold

[0080] Pred c [x, y] = α 2 ×Rec′ L [x,y]+β 2 If Rec′ L [x, y]>Threshold Equation (9)

[0081] III. Linear Model Based on Multiple Reference Lines

[0082] A. Use multiple guides

[0083] In some embodiments, if the chroma component or luma component of the current block has multiple neighboring reference lines, neighboring samples of model parameters for cross-component prediction (eg, CCLM / MMLM) may be adaptively derived based on the reference line selection. Figure 3 A plurality of reference lines of a current block 300 for cross component prediction is shown. As shown, the current block 300 has a plurality of reference lines 311-313, each of which is a line of reconstructed (top and left) neighboring pixels of the current block. In this example, the reference lines 311-313 are at different distances from the border of the current block, where reference line 311 is the line closest to the border of the current block, and reference line 313 is the line farthest from the border of the current block.

[0084] In some embodiments, if the current block has N neighboring reference lines, the i-th neighboring reference line may be selected to derive the model parameters of CCLM / MMLM, where N>1 and N≥i≥1. In an embodiment, if the current block has N neighboring reference lines, more than one reference line may be selected to derive the model parameters of CCLM / MMLM. For example, if the current block has N neighboring reference lines, the video codec may select 2 of the N neighboring reference lines, 3 of the N, ... or N of the N neighboring reference lines to derive the model parameters.

[0085] In some embodiments, if multiple reference lines are selected, the selected reference lines may be adjacent to each other or not adjacent to each other. For example, if two of N adjacent reference lines are selected, the selected two lines may be the first and second reference lines, but may also be the first and third reference lines, the second and fourth reference lines, the first and fourth reference lines, and so on.

[0086] In some embodiments, if a chroma neighboring reference line is selected for CCLM / MMLM model derivation, the video codec may select a luma reference line that may or may not correspond to the selected chroma reference line. For example, if the i-th luma neighboring reference line is selected, the video codec may select the j-th chroma neighboring reference line, where i and j may be different or the same.

[0087] In some embodiments, the video codec may use luma reference line samples when deriving model parameters without performing a luma downsampling process. Figure 4Luma and chroma samples for deriving model parameters without luma subsampling are shown. In the figure, the color components are in 4:2:0 format, various luma sample phases are marked with black circles, and chroma sample phases are marked with hollow circles. A set 410 of luma samples Y0, Y1, Y2, and Y3 is associated with chroma sample C. An adjacent set 411 of luma samples Y'0, Y'1, Y'2, and Y'3 is associated with chroma sample C'. Another adjacent group 412 of luma samples Y"0, Y"1, Y"2, and Y"3 is associated with chroma sample C". The video codec can select {Y0, Y1, (Y0+Y2+1)>>1, (Y'2+(Y0<<1)+Y2+2)>>2, (Y0+(Y2<<1)+Y"0+2)>>2, or (Y0+Y2-Y'2)} to derive model parameters online at specified adjacent luma without applying a luma downsampling filter. For another example, the video codec may select {Y1, Y3, (Y1+Y3+1)>>1, (Y′3+(Y1<<1)+Y3+2)>>2, (Y1+(Y3<<1)+Y”1+2)>>2, or (Y1+Y3-Y′3)} to derive model parameters at specified adjacent luma lines without applying a luma downsampling filter.

[0088] In some embodiments, if more than one adjacent reference line is used for CCLM / MMLM model derivation, the luminance downsampling filters used for different reference lines may be different. For example, if two reference lines are used for CCLM / MMLM model derivation, the downsampling filter for outputting the selected line closer to the current block boundary may be an m-tap high-pass or low-pass filter, while the downsampling filter for outputting the line away from the current block boundary may be an n-tap high-pass or low-pass filter. In some embodiments, m is 6, and n is 4. In some embodiments, m is greater than n. In some embodiments, m is less than n.

[0089] In some embodiments, if a reference line in the plurality of reference lines is invalid due to unavailable neighboring samples or due to CTU line buffer size limitations, the video decoder may replace the invalid reference line with another valid reference line. Figure 3 311-313 are shown. If the third reference line 313 is invalid but the first and second reference lines 311-312 are valid, the video codec can use the first (or second) reference line to replace the third reference line. In some embodiments, only valid reference lines are used in the cross-component model derivation, so that invalid reference lines are not used in the cross-component model derivation.

[0090] B. Merge multiple reference lines

[0091] In some embodiments, if the chroma component or luma component of the current block has multiple adjacent reference lines, the video codec can combine or fuse the multiple adjacent reference lines into one line to derive the model parameters of CCLM / MMLM. Therefore, the luma samples in the multiple reference lines are combined into one fused luma sample line, and the chroma samples in the multiple reference lines are combined into one fused chroma sample line. The fused luma sample line and the fused chroma sample line are then used to derive the linear model of CCLM or MMLM.

[0092] Figure 5A -B conceptually illustrates combining multiple adjacent reference lines into one line to derive model parameters for cross-component prediction. Figure 5A A current block 500 is shown, which has three available neighboring reference lines 511-513. The video codec combines the three neighboring reference lines into one fused line using a 3x3 window, specifically by performing a weighted average of the samples within the moving window. The figure shows an example 3x3 window used to combine 3 neighboring lines of sample Ci, which can be either luma or chroma. The formula for the combined result (or fused sample CF) of the 3x3 window is wi can be positive or negative or 0, and b is an offset value. Similarly, the video codec can first use a 3x2 window to combine three adjacent reference lines.

[0093] Figure 5B It is shown that 3x2 windows are used to combine 3 adjacent lines. The combined result (or fused sample CF) of the 3x2 window is represented as Where wi can be positive or negative and b is the offset value.

[0094] More generally, we can use the formula To fuse multiple rows of brightness samples, and the formula Can be used to fuse multiple rows of chroma samples, where Li and Ci are adjacent luma and chroma samples, S is the applied window size, wi can be positive or negative or 0, and b is an offset value.

[0095] C. Transmission of the selection of reference lines for model derivation

[0096] In some embodiments, the video codec may select an adjacent reference line from a plurality of different adjacent reference lines for CCLM / MMLM model derivation of the current block. Such a selection may be explicitly determined or implicitly derived. For example, if the current block allows one or two reference lines, and the selected line of CCLM / MMLM is explicitly determined, the first bin is used to indicate whether one line or two lines are used. Then, the second bin or multiple bins (encoded by truncated unary or fixed length code) are used to indicate which reference line or which line combination is selected. For example, if one reference line is used for model derivation, the second bin (or more bins) are used to select from {first line, second line, third line...}. If two reference lines are used for model derivation, the second bin (or more bins) are used to select from {first line + second line, second line + third line, first line + third line...}.

[0097] The selection of reference lines for CCLM / MMLM model derivation can also be derived implicitly using decoder side tools such as template cost or boundary matching. For example, in some embodiments, at the decoder side, the final line selection of the CCLM / MMLM model is implicitly determined as the reference line that minimizes the difference between the boundary samples of the current block and the adjacent samples, in other words, the reference line that minimizes the distortion of the adjacent templates.

[0098] For example, after deriving the parameters of a CCLM / MMLM model using a particular reference line, the model is applied to the luminance samples of the boundary to calculate the cost of the derived model (by comparing the boundary chrominance samples predicted by the model with the chrominance samples of the adjacent template). The video codec then derives the parameters of another CCLM / MMLM model by using another reference line and then calculating the cost of the derived model. This process is continuously applied to other models (other reference lines) to calculate their costs, and the model (and reference line) that minimizes the prediction cost of the adjacent template is implicitly selected. The final chrominance prediction of the current block is then calculated by the selected model.

[0099] Figure 6 Conceptually illustrates the implicit selection of reference lines for a current block 600. The current block has boundary samples 605 for which luma samples have been reconstructed. Lines 611-613 adjacent to the current block 600 are used as adjacent reference lines. Reconstructed adjacent samples above and to the left of the current block are used as adjacent templates 620.

[0100] The luma and chroma samples of the reference lines 611 - 613 are used to generate corresponding candidate models 631 - 633 , respectively. The neighboring template 620 may include some of the reference lines 611 - 613 (eg, samples of the reference line 611 adjacent to the current block 600 ).

[0101] The video codec applies each of the candidate models 631-633 to the luma component of the boundary sample 605 to predict the corresponding chroma component. The chroma prediction samples of the boundary 605 produced by each linear model are compared with the chroma samples of the neighboring template 620 (the current upper / left neighboring samples) to determine the cost (e.g., SATD) of the candidate model. The reference line that generates the candidate model that minimizes the cost is implicitly selected (by both the encoder and the decoder) as the CCLM / MMLM model for the current block.

[0102] In some embodiments, the use of more than one reference line may depend on the current block size used to derive the model for different CCLM / MMLM modes (e.g., LM_A, LM_L, LM_LA). For example, in some embodiments, if the current block width is less than a threshold, more than one reference line is used in CCLM_A or MMLM_A (for deriving a cross-component linear model). Similarly, in some embodiments, if the current block height is less than a threshold, more than one reference line is used in CCLM_L or MMLM_L. In some embodiments, if the (width+height) of the current block is less than a threshold, more than one reference line is used in CCLM_LA or MMLM_LA. In some embodiments, if the area of ​​the current block is less than a threshold, more than two reference lines are used in CCLM or MMLM. In some embodiments, more than one reference line is used in CCLM_A, CCLM_L, MMLM_A, or MMLM_L. In some embodiments, syntax is sent at the SPS, PPS, PH, SH, CTU, CU, or PU level to indicate whether more than one reference line is allowed for the current block.

[0103] IV. Example Video Encoder

[0104] Figure 7 An example video encoder 700 is shown that can implement cross-component prediction. As shown, the video encoder 700 receives an input video signal from a video source 705 and encodes the signal into a bitstream 795. The video encoder 700 has several components or modules for encoding the signal from the video source 705, including at least some components selected from the following: a transform module 710, a quantization module 711, an inverse quantization module 714, an inverse transform module 715, an intra-frame estimation module 720, an intra-frame prediction module 725, a motion compensation module 730, a motion estimation module 735, a loop filter 745, a reconstructed picture buffer 750, an MV buffer 765, an MV prediction module 775, and an entropy encoder 790. The motion compensation module 730 and the motion estimation module 735 are part of the inter-frame prediction module 740.

[0105] In some embodiments, modules 710-790 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 710-790 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 710-790 are shown as separate modules, some modules may be combined into a single module.

[0106] The video source 705 provides a raw video signal that presents the pixel data of each video frame without compression. The subtractor 708 calculates the difference between the raw video pixel data of the video source 705 and the predicted pixel data 713 from the motion compensation module 730 or the intra-frame prediction module 725 as a prediction residual 709. The transform module 710 converts the difference (or residual pixel data or residual signal 708) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 711 quantizes the transform coefficients into quantization data (or quantization coefficients) 712, which are encoded into a bitstream 795 by the entropy encoder 790.

[0107] The inverse quantization module 714 dequantizes the quantized data (or quantized coefficients) 712 to obtain transform coefficients, and the inverse transform module 715 performs an inverse transform on the transform coefficients to generate a reconstructed residual 719. The reconstructed residual 719 is added to the predicted pixel data 713 to generate reconstructed pixel data 717. In some embodiments, the reconstructed pixel data 717 is temporarily stored in a line buffer (not shown) for intra prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 745 and stored in the reconstructed picture buffer 750. In some embodiments, the reconstructed picture buffer 750 is a memory external to the video encoder 700. In some embodiments, the reconstructed picture buffer 750 is a memory internal to the video encoder 700.

[0108] The intra estimation module 720 performs intra prediction based on the reconstructed pixel data 717 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 790 to be encoded into a bitstream 795. The intra prediction data is also used by the intra prediction module 725 to generate predicted pixel data 713.

[0109] The motion estimation module 735 performs inter-frame prediction by generating MVs to refer to pixel data of a previously decoded frame stored in the reconstructed picture buffer 750. These MVs are provided to the motion compensation module 730 to generate predicted pixel data.

[0110] Instead of encoding the complete actual MV in the bitstream, the video encoder 700 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 795.

[0111] The MV prediction module 775 generates a predicted MV based on a reference MV generated for encoding a previous video frame, i.e., a motion compensated MV for performing motion compensation. The MV prediction module 775 obtains the reference MV from the previous video frame from the MV buffer 765. The video encoder 700 stores the MV generated for the current video frame in the MV buffer 765 as a reference MV for generating a predicted MV.

[0112] The MV prediction module 775 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame is encoded into the bit stream 795 by the entropy encoder 790.

[0113] The entropy encoder 790 encodes various parameters and data into a bitstream 795 by using an entropy coding technique such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 790 encodes various header elements, flags, along with quantized transform coefficients 712 and residual motion data as syntax elements into the bitstream 795. The bitstream 795 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0114] The loop filter 745 performs filtering or smoothing operations on the reconstructed pixel data 717 to reduce encoding and decoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0115] Figure 8 A portion of a video encoder 700 that implements linear model derivation based on multiple reference lines is shown. As shown, a video source 705 provides input luma and chroma samples 802 and 804, while a reconstructed picture buffer 750 provides reconstructed luma and chroma samples 806.

[0116] The chroma prediction module 810 is applied to the input luma samples 802 to generate the chroma predictor 812. The chroma predictor 812 is then used to produce a chroma prediction residual 815 by subtracting the input chroma samples 804. The chroma prediction residual signal 815 is encoded (transform, inter / intra prediction, etc.) in place of the regular chroma samples.

[0117] The cross component prediction model 820 is a linear model for cross component prediction (from luminance to chrominance) of the current block. The parameters (e.g., α and β) of the cross component prediction model 820 are derived by the model derivation module 840 from samples of one or more reference lines adjacent to the current block. The samples of multiple reference lines are provided by the reconstructed picture buffer 750 as reconstructed luminance and chrominance samples 806. The reference line selection module 830 selects one or more of the reference lines as the luminance and chrominance sample sources to be used for linear model derivation. When at least two reference lines are selected, the two reference lines may or may not be adjacent to each other. The indication of the selected reference line may or may not be provided to the entropy encoder 790 to be sent to the decoder.

[0118] In some embodiments, the model derivation module 840 combines samples from multiple different reference lines into one fused sample line. In some embodiments, the model derivation module 840 may subsample luma samples without applying a downsampling filter.

[0119] In some embodiments, the model derivation module 840 derives multiple candidate linear models 825 for multiple reference lines such that the luma and chroma samples of each reference line are used to derive a corresponding candidate linear model. The cross-component prediction model 820 can determine the cost of each candidate model and identify the lowest cost candidate model (and its corresponding reference line). The cost of the candidate model (and therefore the corresponding reference line) can be a boundary matching cost calculated based on (i) the chroma prediction samples generated by the candidate model at the boundary of the current block and (ii) the reconstructed chroma samples adjacent to the current block.

[0120] Fig. 9 A process 900 for performing cross-component prediction using multiple reference lines is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 700 perform process 900 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 700 performs process 900.

[0121] The encoder receives (at block 910) data for a block of pixels that is to be encoded as a current block of a current picture.

[0122] The encoder derives (at block 920) a linear model based on luma and chroma samples from one or more reference lines, the one or more reference lines being selected from a plurality of reference lines adjacent to the current block at different distances. In some embodiments, at least one of the selected reference lines is adjacent to a boundary of the current block. In some embodiments, at least one of the selected reference lines is not adjacent to a boundary of the current block. In some embodiments, luma and chroma samples of only one selected reference line are used to derive the linear model. In some embodiments, luma and chroma samples of two or more selected reference lines are used to derive the linear model. The two or more selected reference lines may or may not be adjacent to each other. In some embodiments, the luma samples and chroma samples used to derive the linear model may have the same or different selected reference lines. In some embodiments, the luma samples used to derive the linear model are subsampled without applying a downsampling filtering process, such as a reference line. Figure 4 described.

[0123] In some embodiments, the encoder sends a syntax element indicating whether to select more than one reference line from a plurality of reference lines for deriving the linear model. In some embodiments, the plurality of reference lines selected from the plurality of reference lines is determined based on the size of the current block or the cross-component prediction mode (e.g., CCLM_A, CCLM_L, CCLM_LA) used for the current block.

[0124] In some embodiments, luma samples in multiple reference lines are combined into a line of fused luma samples, and chroma samples in multiple reference lines are combined into a line of fused chroma samples. A linear model is derived based on the line of fused luma samples and the line of fused chroma samples. Each fused sample is a weighted average of samples within a window containing multiple reference lines.

[0125] In some embodiments, the linear model is derived from samples of a particular reference line selected from the plurality of reference lines with a minimum boundary matching cost. The boundary matching cost of the reference line is determined based on the difference between (i) boundary chrominance samples of the current block predicted by the candidate model of the reference line (by applying the candidate model to boundary luminance samples of the current block) and (ii) reconstructed chrominance samples adjacent to the current block.

[0126] The encoder applies (at block 930) the linear model to the luma samples of the current block to obtain chroma prediction samples of the current block. The encoder encodes (at block 940) the current block by generating a chroma prediction residual for the current block using the chroma prediction samples of the current block.

[0127] V. Example Video Decoder

[0128] In some embodiments, the encoder may send (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0129] Fig.10 An example video decoder 1000 is shown that can perform motion vector refinement. As shown, the video decoder 1000 is an image decoding or video decoding circuit that receives a bitstream 1095 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 1000 has several components or modules for decoding the bitstream 1095, including some components selected from the following: an inverse quantization module 1011, an inverse transform module 1010, an intra-frame prediction module 1025, a motion compensation module 1030, a loop filter 1045, a decoded picture buffer 1050, an MV buffer 1065, an MV prediction module 1075, and a parser 1090. The motion compensation module 1030 is part of the inter-frame prediction module 1040.

[0130] In some embodiments, modules 1010-1090 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1010-1090 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1010-1090 are shown as separate modules, some modules may be combined into a single module.

[0131] The parser 1090 (or entropy decoder) receives the bitstream 1095 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1012. The parser 1090 parses out various syntax elements by using entropy coding techniques (e.g., context-adaptive binary arithmetic coding (CABAC) or Huffman encoding).

[0132] The inverse quantization module 1011 dequantizes the quantized data (or quantized coefficients) 1012 to obtain transform coefficients, and the inverse transform module 1010 inversely transforms the transform coefficients 1016 to generate a reconstructed residual signal 1019. The reconstructed residual signal 1019 is added to the predicted pixel data 1013 from the intra prediction module 1025 or the motion compensation module 1030 to generate decoded pixel data 1017. The decoded pixel data is filtered by the loop filter 1045 and stored in the decoded picture buffer 1050. In some embodiments, the decoded picture buffer 1050 is a memory external to the video decoder 1000. In some embodiments, the decoded picture buffer 1050 is a memory internal to the video decoder 1000.

[0133] The intra prediction module 1025 receives intra prediction data from the bitstream 1095 and, based thereon, generates predicted pixel data 1013 from decoded pixel data 1017 stored in the decoded picture buffer 1050. In some embodiments, the decoded pixel data 1017 is also stored in a line buffer (not shown) for intra prediction and spatial MV prediction.

[0134] In some embodiments, the contents of the decoded picture buffer 1050 are used for display. The display device 1055 either obtains the contents of the decoded picture buffer 1050 for direct display or obtains the contents of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded picture buffer 1050 via pixel transfer.

[0135] The motion compensation module 1030 generates predicted pixel data 1013 from decoded pixel data 1017 stored in the decoded picture buffer 1050 according to motion compensated MVs (MC MVs). These motion compensated MVs are decoded by adding residual motion data received from the bitstream 1095 to the predicted MVs received from the MV prediction module 1075.

[0136] The MV prediction module 1075 generates a predicted MV based on a reference MV generated for decoding a previous video frame (e.g., a motion compensated MV for performing motion compensation). The MV prediction module 1075 obtains the reference MV of the previous video frame from the MV buffer 1065. The video decoder 1000 stores the motion compensated MV generated for decoding the current video frame in the MV buffer 1065 as a reference MV for generating a predicted MV.

[0137] The loop filter 1045 performs filtering or smoothing operations on the decoded pixel data 1017 to reduce encoding and decoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0138] Fig.11 A portion of a video decoder 1000 is shown that implements linear model derivation based on multiple reference lines. As shown, a decoded picture buffer 1050 provides decoded luma and chroma samples to a chroma prediction module 1110, which generates reconstructed chroma samples 1135 for display or output by predicting chroma samples based on luma samples.

[0139] The chroma prediction module 1110 receives decoded pixel data 1017, which includes reconstructed luma samples 1125 and chroma prediction residuals 1115. The chroma prediction module 1110 uses the reconstructed luma samples 1125 to generate chroma predictors 1112. The chroma predictors 1112 are then added to the chroma prediction residuals 1115 to generate reconstructed chroma samples 1135. The reconstructed chroma samples 1135 are then stored in the decoded picture buffer 1050 for display and for reference by subsequent blocks and pictures.

[0140] The cross component prediction model 1120 is a linear model for cross component prediction (from luminance to chrominance) of the current block. The parameters (e.g., αα and β) of the cross component prediction model 1120 are derived from samples of one or more reference lines adjacent to the current block by the model derivation module 1140. The samples of multiple reference lines are provided by the decoded picture buffer 1050 as reconstructed luminance and chrominance samples 1106. The reference line selection module 1130 selects one or more reference lines as the luminance and chrominance sample sources to be used for linear model derivation. When at least two reference lines are selected, the two reference lines may be adjacent to each other or may not be adjacent to each other. The entropy decoder 1090 may provide an indication of the selected reference line parsed from the bitstream 1095.

[0141] In some embodiments, the model derivation module 1140 combines samples from multiple different reference lines into one fused sample line. In some embodiments, the model derivation module 1140 may subsample luma samples without applying a downsampling filter.

[0142] In some embodiments, the model derivation module 1140 derives multiple candidate linear models 1125 for multiple reference lines such that the luma and chroma samples of each reference line are used to derive a corresponding candidate linear model. The cross-component prediction model 1120 can determine the cost of each candidate model and identify the lowest cost candidate model (and its corresponding reference line). The cost of the candidate model (and therefore the corresponding reference line) can be a boundary matching cost calculated based on (i) the chroma prediction samples generated by the candidate model at the boundary of the current block and (ii) the reconstructed chroma samples adjacent to the current block.

[0143] Fig.12 A process 1200 for performing cross-component prediction using multiple reference lines is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 1000 perform process 1200 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 1000 performs process 1200.

[0144] The decoder receives (at block 1210) data for a block of pixels that is to be decoded as a current block of a current picture.

[0145] The decoder derives (at block 1220) a linear model based on luma and chroma samples from one or more reference lines, the one or more reference lines being selected from a plurality of reference lines adjacent to the current block at different distances. In some embodiments, at least one of the selected reference lines is adjacent to a boundary of the current block. In some embodiments, at least one of the selected reference lines is not adjacent to a boundary of the current block. In some embodiments, luma and chroma samples of only one selected reference line are used to derive the linear model. In some embodiments, luma and chroma samples of two or more selected reference lines are used to derive the linear model. The two or more selected reference lines may or may not be adjacent to each other. In some embodiments, the luma samples and chroma samples used to derive the linear model may have the same or different selected reference lines. In some embodiments, the luma samples used to derive the linear model are subsampled without applying a downsampling filtering process, as described above with reference to Figure 4 described.

[0146] In some embodiments, the decoder receives a syntax element indicating whether to select more than one reference line from a plurality of reference lines for deriving the linear model. In some embodiments, the plurality of reference lines selected from the plurality of reference lines is determined based on a size of the current block or a cross-component prediction mode (e.g., CCLM_A, CCLM_L, CCLM_LA) for the current block.

[0147] In some embodiments, luma samples in multiple reference lines are combined into a line of fused luma samples, and chroma samples in multiple reference lines are combined into a line of fused chroma samples. A linear model is derived based on the fused luma sample line and the fused chroma sample line. Each fused sample is a weighted average of samples within a window containing multiple reference lines.

[0148] In some embodiments, the linear model is derived from samples of a particular reference line selected from the plurality of reference lines with a minimum boundary matching cost. The boundary matching cost of the reference line is determined based on the difference between (i) boundary chrominance samples of the current block predicted by the candidate model for the reference line (by applying the candidate model to boundary luminance samples of the current block) and (ii) reconstructed chrominance samples adjacent to the current block.

[0149] The decoder applies (at block 1230) the linear model to the luma samples of the current block to obtain the chroma prediction samples of the current block. The decoder reconstructs (at block 1240) the current block by using the chroma prediction samples of the current block (to combine with the chroma prediction residual). The decoder can then provide the reconstructed current block for display as part of a reconstructed current picture.

[0150] VI. Example Electronic System

[0151] Many of the above features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, compact disc read-only memory (CD-ROM), flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted by wireless or wired connections.

[0152] In this specification, the term "software" is intended to include firmware residing in read-only memory or application programs stored in magnetic memory, which can be read into memory for processing by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that jointly implement the software inventions described herein are within the scope of this disclosure. In some embodiments, the software program, when installed to run on one or more electronic systems, defines one or more specific machine implementations that process and execute the operations of the software program.

[0153] Fig.13 An electronic system 1300 for implementing some embodiments of the present disclosure is conceptually shown. The electronic system 1300 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1300 includes a bus 1305, a processing unit 1310, a graphics processing unit (GPU) 1315, a system memory 1320, a network 1325, a read-only memory 1330, a permanent storage device 1335, an input device 1340, and an output device 1345.

[0154] Bus 1305 collectively represents all system, peripheral, and chipset buses that communicatively couple numerous internal devices of electronic system 1300. For example, bus 1305 communicatively couples processing unit 1310 with GPU 1315, read-only memory 1330, system memory 1320, and permanent storage device 1335.

[0155] The processing unit 1310 obtains instructions to be executed and data to be processed from these various memory units in order to perform the processing of the present disclosure. In different embodiments, the processing unit can be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 1315. The GPU 1315 can offload various calculations or supplement the image processing provided by the processing unit 1310.

[0156] Read-only-memory (ROM) 1330 stores static data and instructions used by processing unit 1310 and other modules of the electronic system. On the other hand, permanent storage device 1335 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1300 is turned off. Some embodiments of the present disclosure use a large-capacity memory device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 1335.

[0157] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like the permanent storage device 1335, the system memory 1320 is a read-write memory device. However, unlike the permanent storage device 1335, the system memory 1320 is a volatile read-write memory, such as a random access memory. The system memory 1320 stores some instructions and data used by the processor at runtime. In some embodiments, the processing according to the present disclosure is stored in the system memory 1320, the permanent storage device 1335 and / or the read-only memory 1330. For example, according to some embodiments of the present disclosure, various memory units include instructions for processing multimedia clips. From these various memory units, the processing unit 1310 obtains instructions to be executed and data to be processed in order to perform the processing of some embodiments.

[0158] The bus 1305 is also connected to an input device 1340 and an output device 1345. The input device 1340 enables a user to convey information and select commands to the electronic system. The input device 1340 includes an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"), a camera (e.g., a webcam), a microphone, or a similar device for receiving voice commands, etc. The output device 1345 displays an image generated by the electronic system or outputs data. The output device 1345 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices used as input and output devices, such as a touch screen.

[0159] Finally, if Fig.13 As shown, bus 1305 also couples electronic system 1300 to a network 1325 via a network adapter (not shown). In this manner, the computer may be part of a computer network, such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of multiple networks, such as the Internet. Any or all components of electronic system 1300 may be used in conjunction with the present disclosure.

[0160] Some embodiments include electronic components, such as microprocessors, storage devices, and memory, which store computer program instructions in machine-readable or computer-readable media (or referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, double-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer program or computer code may be a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer codes include machine code such as produced by a compiler, and files including high-level code that is executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0161] Although the above discussion mainly involves microprocessors or multi-core processors executing software, many of the above features and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0162] As used in this specification and any claims of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display refer to displaying on an electronic device. As used in this specification and any claims of this application, the terms "computer-readable medium," "computer-readable medium," and "machine-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other transient signals.

[0163] Although the present disclosure has been described with reference to many specific details, those skilled in the art will recognize that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Fig. 9 and Fig.12 ) conceptually illustrates the processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process may be implemented using several sub-processes or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited by the foregoing illustrative details, but is defined by the appended claims.

[0164] Additional Notes

[0165] The subject matter described herein sometimes represents different components, which are contained in or connected to other different components. It is understood that the described structure is only an example, and in fact, it can be implemented by many other structures to achieve the same function. Conceptually, the arrangement of any components that achieve the same function is actually "associated" to achieve the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to achieve a specific function are considered to be "interrelated" to achieve the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to achieve a specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting components, and / or wirelessly interactive and / or wirelessly interactive components, and / or logically interactive and / or logically interactive components.

[0166] Furthermore, with respect to the use of substantially any plural and / or singular terms, those skilled in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For sake of clarity, the present invention expressly sets forth various singular / plural arrangements.

[0167] In addition, it will be understood by those skilled in the art that, in general, the terms used in the present invention, especially in the claims, and the subject matter of the claims, are generally used as "open" terms, for example, "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least having", "including" should be interpreted as "including but not limited to", etc. It will be further understood by those skilled in the art that if a specific number of claim contents are intended to be introduced, it will be clearly indicated in the claim, and will not be displayed in the absence of such contents. For example, to aid understanding, the following claims may contain the phrases "at least one" and "one or more" to introduce the contents of the claims. However, the use of these phrases should not be understood to imply the use of the indefinite article "a" or "an" to introduce the claim contents and restrict any specific patent scope. Even when the same claim includes the introductory phrases "one or more" or "at least one", the indefinite article, such as "a" or "an", should be interpreted to mean at least one or more, and the same is true for the use of the explicit description used to introduce the claim. Furthermore, even when introductory content explicitly refers to a specific number, a person of ordinary skill in the art will recognize that such content should be interpreted as indicating the number being referenced, for example, "two references" without other modifications means at least two references, or two or more references. Furthermore, where a statement similar to "at least one of A, B, and C" is used, it is generally stated so that a person of ordinary skill in the art can understand the statement, for example, "a system includes at least one of A, B, and C" will include but is not limited to a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc. A person of ordinary skill in the art will further understand that any separated words and / or phrases represented by two or more alternative terms, whether in the specification, claims, or drawings, should be understood to include the possibility of one of these terms, one of them, or both of these terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

[0168] As can be seen from the foregoing, various embodiments have been described for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and application are indicated by the claims.

Claims

1. A video encoding and decoding method, comprising: Receiving data of a pixel block, wherein the data of the pixel block is to be encoded or decoded into a current block of a current picture; deriving a linear model based on a plurality of luma samples and a plurality of chroma samples from one or more reference lines, the one or more reference lines being selected from a plurality of reference lines adjacent to the current block at different distances; Applying the linear model to a plurality of luma samples of the current block to obtain a plurality of chroma prediction samples of the current block; as well as The current block is encoded or decoded using the plurality of chroma prediction samples of the current block.

2. The video encoding and decoding method according to claim 1, characterized in that: The linear model is derived using a plurality of luma samples and a plurality of chroma samples of only one selected reference line.

3. The video encoding and decoding method according to claim 1, characterized in that: The linear model is derived using a plurality of luma samples and a plurality of chroma samples of two or more selected reference lines.

4. The video encoding and decoding method according to claim 3, characterized in that: The two or more selected reference lines are adjacent to each other.

5. The video encoding and decoding method according to claim 3, characterized in that: The two or more selected reference lines are not adjacent to each other.

6. The video encoding and decoding method according to claim 1, characterized in that: At least one of the selected reference lines is adjacent to a boundary of the current block.

7. The video encoding and decoding method according to claim 1, characterized in that: At least one of the selected reference lines is not adjacent to a boundary of the current block.

8. The video encoding and decoding method according to claim 1, characterized in that: The plurality of luma samples used to derive the linear model originate from a first selected reference line, and the plurality of chroma samples used to derive the linear model originate from a second selected reference line, the second selected reference line being different from the first selected reference line.

9. The video encoding and decoding method according to claim 1, characterized in that: The plurality of luma samples and the plurality of chroma samples used to derive the linear model originate from the same selected reference line.

10. The video encoding and decoding method according to claim 1, characterized in that: The plurality of luma samples used to derive the linear model are subsampled without using a filter.

11. The video encoding and decoding method according to claim 1, characterized in that: The multiple luma samples in the multiple reference lines are combined into a fused luma sample line, and the multiple chroma samples in the multiple reference lines are combined into a fused chroma sample line, wherein the linear model is derived based on the fused luma sample line and the fused chroma sample line, wherein each fused sample is a weighted average of multiple samples within a window containing multiple reference lines.

12. The video encoding and decoding method according to claim 1, characterized in that: Also included is receiving or sending a syntax element indicating whether to select more than one reference line from the plurality of reference lines to derive the linear model.

13. The video encoding and decoding method according to claim 1, characterized in that: The linear model is derived from multiple samples of a specific reference line selected from the multiple reference lines with a minimum boundary matching cost, wherein the boundary matching cost of the reference line is determined based on (i) a plurality of boundary chrominance samples of the current block predicted by the candidate model of the reference line and (ii) a difference between a plurality of reconstructed chrominance samples adjacent to the current block.

14. The video encoding and decoding method according to claim 1, characterized in that: Determining the plurality of reference lines from among the plurality of reference lines is based on a size of the current block or a mode of cross-component prediction (CCLM_A, CCLM_L, CCLM_LA) for the current block.

15. An electronic device, comprising: The video codec circuitry is configured to perform a number of operations, including: Receiving data of a pixel block, wherein the data of the pixel block is to be encoded or decoded into a current block of a current picture; deriving a linear model based on a plurality of luma samples and a plurality of chroma samples from one or more reference lines, the one or more reference lines being selected from a plurality of reference lines adjacent to the current block at different distances; Applying the linear model to a plurality of luma samples of the current block to obtain a plurality of chroma prediction samples of the current block; and The current block is encoded or decoded using the plurality of chroma prediction samples of the current block.

16. A video decoding method, comprising: Receiving data of a pixel block, wherein the data of the pixel block is to be decoded into a current block of a current picture; deriving a linear model based on a plurality of luma samples and a plurality of chroma samples from one or more reference lines, the one or more reference lines being selected from a plurality of reference lines adjacent to the current block at different distances; Applying the linear model to a plurality of luma samples of the current block to obtain a plurality of chroma prediction samples of the current block; as well as The current block is reconstructed using the multiple chroma prediction samples of the current block.

17. A video encoding method, comprising: Receiving data of a pixel block, wherein the data of the pixel block is to be encoded as a current block of a current picture; deriving a linear model based on a plurality of luma samples and a plurality of chroma samples from one or more reference lines, the one or more reference lines being selected from a plurality of reference lines adjacent to the current block at different distances; Applying the linear model to a plurality of luma samples of the current block to obtain a plurality of chroma prediction samples of the current block; as well as The current block is encoded using the plurality of chroma prediction samples of the current block.