Simplifications of coding modes based on neighboring samples-dependent parametric models

Abstract

To improve and simplify the design of modes similarly to CCLM or LIC, based on neighboring samples-dependent parametric models.SOLUTION: Proposed modifications relate to ways of deriving parameters of a parametric model and designing parametric-model-based prediction tools contained in a codec in a unified and simplified way. In one embodiment, an approach proposes simplification of a cross-component linear model process for deriving linear parameters. It is proposed to replace the least mean square method to derive parameters instead as parameters of a straight line passing through two points corresponding to the minimum and maximum luma values among all luma neighboring reconstructed samples.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] At least one of the present embodiments generally relates to a method or apparatus for video encoding, decoding, compression, or decompression. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes typically employ prediction, including motion vector prediction and transform, to exploit spatial and temporal redundancy in video content. Typically, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and then the difference between the original and predicted image, often referred to as prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by the inverse processes corresponding to entropy coding, quantization, transform, and prediction. Summary of the Invention

[0003] At least one of the present embodiments relates generally to a method or apparatus for video encoding or decoding, and more particularly to a method or apparatus for coding mode simplification based on a neighboring samples dependent parametric model.

[0004] According to a first aspect, a method is provided, the method including determining a prediction of a sample in a current block from at least one of neighboring samples in the current block and from a parametric model calculated from the neighboring samples in the current block and a reference sample in a reference frame, and encoding the sample in the current block based on the prediction.

[0005] According to a second aspect, a method is provided, the method including determining a prediction of a sample in the current block from at least one of neighboring samples in the current block and from a parametric model calculated from the neighboring samples in the current block and a reference sample in a reference frame, and decoding the sample in the current block based on the prediction.

[0006] According to another aspect, an apparatus is provided, the apparatus including a processor, the processor may be configured to encode blocks of video or decode a bitstream by performing any of the methods described above.

[0007] According to another general aspect of at least one embodiment, a device is provided, the device including an apparatus according to any of the embodiments for decoding, and at least one of (i) an antenna configured to receive a signal including the video block, (ii) a band limiter configured to limit the received signal to a band of frequencies including the video block, or (iii) a display configured to display an output representing the video block.

[0008] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that includes data content generated according to any of the described encoding embodiments or variations thereof.

[0009] According to another general aspect of at least one embodiment, there is provided a signal including video data generated according to any of the described encoding embodiments or variations thereof.

[0010] According to another general aspect of at least one embodiment, a bitstream containing data content generated according to any of the described encoding embodiments or variations thereof is formatted.

[0011] According to another general aspect of at least one embodiment, a computer program product is provided that includes instructions that, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variations thereof.

[0012] These and other aspects, features, and advantages of the overall embodiment will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0013] [Figure 1] Exemplary locations of samples used for the derivation of a and b are shown. [Figure 2] An example of the LM_A mode will be described. [Figure 3] An example of the LM_L mode will be described. [Figure 4] The LIC mode display in JEM is shown below. [Figure 5] 1 illustrates a standard general-purpose video compression scheme. [Figure 6] 1 shows a standard general-purpose video restoration scheme. [Figure 7] Shown are selected samples from the top line in the rightmost position and the left column in the bottommost position. [Figure 8] An exemplary block for using two samples at specific adjacent locations is shown. [Figure 9] Shows selected samples from the top line in the rightmost position and the left column in the bottommost position. [Figure 10] Selected samples from the top line are shown in the rightmost and leftmost positions. [Figure 11] Selected samples from the left column are shown in the top and bottom positions. [Figure 12] Selected samples are shown in the bottom left position, top left position, and top right position. [Figure 13] Selected samples from the top line are shown in the rightmost and leftmost positions. [Figure 14] Selected samples from the left column are shown in the top and bottom positions. [Figure 15] (a) shows selected samples in more than three positions, (b) shows selected samples in the top two positions and in the left two positions, and (c) shows selected samples in the top three positions and in the left three positions. [Figure 16] FIG. 1 shows an exemplary block diagram for testing the reliability of a linear model derivation. [Figure 17] Indicates the weight used in intra-prediction-inter-prediction mixing. [Figure 18] 1 illustrates one embodiment of a method under the described aspect. [Figure 19] 1 illustrates an exemplary processor-based subsystem for implementation of the generally described aspects. [Figure 20] A block diagram of CCLM / MDLM processing is shown. [Figure 21] 1 shows a modified block diagram of CCLM / MDLM processing according to a first embodiment. [Figure 22] 10 shows a modified block diagram of CCLM / MDLM processing according to a second embodiment. [Figure 23] 10 shows a modified block diagram of CCLM / MDLM processing according to a variant of the second embodiment; [Figure 24] FIG. 10 shows a modified block diagram of CCLM / MDLM processing according to a variant of the third embodiment. [Figure 25] 10 illustrates another embodiment of the method under the described aspect. [Figure 26] 1 illustrates an exemplary apparatus under the described embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] TECHNICAL FIELD The embodiments described herein are in the field of video compression and generally relate to video compression and video encoding and decoding.

[0015] To achieve high compression efficiency, image and video coding schemes typically employ prediction, including motion vector prediction and transform, to exploit spatial and temporal redundancy in video content. Typically, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and then the difference between the original and predicted image, often referred to as prediction error or prediction residual, is transformed, quantized, and coded. To reconstruct video, the compressed data is decoded by the inverse processes corresponding to entropy coding, quantization, transform, and prediction.

[0016] The HEVC (High Efficiency Video Coding, ISO / IEC 23008-2, ITU-TH.265) video compression standard employs motion-compensated temporal prediction to exploit the redundancy that exists between consecutive pictures of a video.

[0017] To do this, a motion vector is associated with each prediction unit (PU). Each coding tree unit (CTU) is represented by a coding tree in the compressed domain, which is a quadtree partition of the CTU, with each leaf called a coding unit (CU).

[0018] Each CU is then given some intra- or inter-prediction parameters (prediction information). To do this, it is spatially partitioned into one or more prediction units (PUs), and each PU is assigned some prediction information. An intra- or inter-coding mode is assigned at the CU level.

[0019] The Joint Video Exploration Team (JVET) proposal for a new video compression standard, known as the Joint Exploration Model (JEM), has proposed allowing a quadtree-binary tree (QTBT) block partitioning structure due to its high compression performance. Blocks in a binary tree (BT) may be divided into two equally sized sub-blocks by splitting them either horizontally or vertically in the middle. As a result, BT blocks may have rectangular shapes with unequal widths and heights, unlike blocks in a QT, where blocks always have square shapes with equal heights and widths. HEVC defines angular intra prediction directions from 45 degrees to -135 degrees across a 180-degree angle, which are maintained in the JEM, with the definition of the angular direction dependent on the shape of the target block.

[0020] To encode these blocks, intra prediction is used to provide an estimated version of the block using previously reconstructed neighboring samples. The difference between the source block and the prediction is then coded. In the classical codecs mentioned above, a single line of reference samples is used, located to the left and above the current block.

[0021] In High Efficiency Video Coding (H.265), encoding of frames of a video sequence is based on a quadtree (QT) block partitioning structure. A frame is divided into square coding tree units (CTUs), and all CTUs undergo quadtree-based splitting into multiple coding units (CUs) based on a rate-distortion (RD) criterion. Each CU is either intra-predicted, i.e., spatially predicted from causal neighboring CUs, or inter-predicted, i.e., temporally predicted from an already decoded reference frame. In an I slice, all CUs are intra-predicted, whereas in both P slices and B slices, CUs may be intra-predicted or inter-predicted. For intra prediction, HEVC defines 35 prediction modes, including one planar mode (indexed as mode 0), one DC mode (indexed as mode 1), and 33 angular modes (indexed as modes 2 to 34). Angle modes are associated with prediction directions ranging from 45 degrees to −135 degrees in the clockwise direction. Because HEVC supports a quadtree (QT) block partitioning structure, all prediction units (PUs) have a square shape. Therefore, defining prediction angles from 45 degrees to −135 degrees is justified in terms of the shape of the PU (prediction unit). For a target prediction unit of size N×N pixels, the top reference array and the left reference array are each of size 2N+1 samples, which are required to cover the above-mentioned angle range for all target pixels. Considering that the height and width of a PU are of equal length, it also makes sense that the lengths of the two reference arrays are equal.

[0022] The present invention is in the field of video compression, with a particular focus on modes that use parametric models to perform prediction for a given block, where the parameters of the model are derived from neighboring samples of the block. Two examples of such modes are the "cross-component linear model" (CCLM) mode and the "Local illumination compensation" (LIC) mode. The present invention aims to simplify and improve the design of such modes.

[0023] Description of CCLM and its variants The following sections describe different variants of CCLM.

[0024] Basic CCLM mode explanation In its earlier version (see JVET_K1002), the CCLM mode consists of predicting chroma samples based on the reconstructed luma samples of the same block or CU using a linear model, as in equation (1). pred C (i,j)=a.rec L '(i,j)+b Equation (1) pred C (i,j) represents the predicted chroma sample in a CU, and rec L '(i,j) represents the downsampled reconstructed luma samples of the same CU. The parameters a and b are derived by minimizing the regression error between the neighboring reconstructed luma and chroma samples around the current block, as in Equation (2) and Equation (3). a=(SLC-SL.SC) / (SLL-SL.SL) Formula (2) b=SC-a.SL Equation (3) L(i,j) represents the downsampled top-neighboring reconstructed luma sample and the downsampled left-neighboring reconstructed luma sample, C(i,j) represents the top-neighboring reconstructed chroma sample and the left-neighboring reconstructed chroma sample, N is equal to twice the minimum of the width and height of the current chroma coding block, and SL, SC, SLL, SLC are defined as follows (the symbol Σ represents the sum over the top-neighboring sample and the left-neighboring sample): -SL=ΣL(n) -SC=ΣC(n) -SLC=N·Σ(L(n)·C(n)) -SLL=N·Σ(L(n)·L(n))

[0025] For coding blocks with a square shape, the above two formulas are directly applied. For non-square coding blocks, the adjacent samples of the longer boundary are first subsampled to have the same number of samples as the samples for the shorter boundary. Figure 1 shows the positions of the left sample and the top sample, as well as the positions of the samples of the current block involved in CCLM mode.

[0026] When a CU is coded using CCLM mode, the least mean squares (LMS) method is performed in the decoding process. As a result, no syntax is used to communicate the a and b values to the decoder.

[0027] MDLM mode The MDLM mode is an improvement of the basic CCLM design proposed in JVET-L0338, in which, in addition to the (top+left) reference sample template, it is possible to select a left-only template or a top-only template to derive the linear model coefficients α and β. This means that two new CCLM modes, called LM_A and LM_L values, are added.

[0028] In LM_A mode (see Figure 2), only the top template is used to calculate the linear model coefficients. To obtain more samples, the top template is extended to (W+H), where W is the width of the block and H is its height. In LM_L mode (see Figure 3), only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is extended to (H+W).

[0029] For non-square blocks, the top template is dilated to W+W and the left template is dilated to H+H.

[0030] If no top / left template is available, LM_A / LM_L mode is not checked or signaled. If the number of available samples is not large enough, the template is padded by duplicating the rightmost (for the top template) or bottommost (for the left template) sample to the nearest log2 number.

[0031] CCLM / MDLM with line buffer constraints In the current CCLM coefficient derivation process, two luma line buffers are generally used for downsampling to obtain the top template of the CCLM mode (CCLM or MDLM), but only one luma line buffer is used in conventional luma component intra prediction. To reduce the line buffer, only the LM_L mode is used for the CU along the top boundary of the CTU. In this case, no additional line buffer is required.

[0032] Local illumination compensation explained In this tool, the decoder calculates some prediction parameters based on some reconstructed picture samples localized to the left and / or top of the current block to be predicted and reference picture samples localized to the left and / or top of the motion-compensated block (Figure 4). In the considered prior art codec (JEM), the use of LCI for a given block depends on a flag associated with this block, called the LIC flag.

[0033] The LIC parameters (a, b) are based on the least mean square minimization, which minimizes the distortion shown in equation (4). dist=Σ (r∈Vcur,s∈Vref) (Rcur(r)-a.Rref(s)-b) 2 Formula (4) Rcur(r) is the adjacent reconstructed sample, and Rref(s) is the reference sample. The derivation of a and b is similar to how a and b were derived in the previous section (Equations (2) and (3)).

[0034] Once the LIC parameters for the current CU are obtained by the encoder or decoder, the prediction pred(i,j) of the current CU is constructed from the following equation (5) (unidirectional prediction case): pred(i,j)=a.ref(i,j)+b Equation (5) ref(i,j) is the reference block used for temporal prediction of the current block.

[0035] The overall aspects described herein aim to improve and simplify the design of CCLM or LIC-like modes based on adjacent sample-dependent parametric models. The proposed modifications relate to the manner in which the parameters of the parametric model are derived and how to design a parametric model-based predictor included in the codec in a unified and simplified manner compared to the prior art.

[0036] One approach proposes a simplification of the CCLM process to derive linear parameters: it is proposed to replace the LMS method to derive the parameters a and b as parameters of a line that passes through two points corresponding to the minimum and maximum luma values among all adjacent reconstructed luma samples.

[0037] The values of a and b are derived as shown in equations (6) and (7). a=(C B -C A ) / (L B -L A ) Formula (6) b=C A -aL A Formula (7) (L A ,C A ) is, on the other hand, L A is the couple of luma and chroma values in adjacent reconstructed samples that has the smallest value among all luma values, and (L B ,C B ) is, on the other hand, L B is the couple of luma and chroma values in adjacent reconstructed samples that has the maximum value among all luma values.

[0038] This approach still requires performing multiple tests to identify the minimum and maximum luma values. A and L A You may encounter problems when the

[0039] The LMS approach used in early CCLM and LIC schemes has other problems. One important one is that LMS can lead to bias when the input samples are corrupted by noise, which is evident as the samples result from coding or prediction. This can reduce the coding efficiency of the tool.

[0040] The overall aspect described herein proposes the following various modifications. - Simplified selection of samples used to derive the parameters of a parametric model: samples are fetched from predefined locations -The use of alternative coding modes when deriving the parameters of parametric models is unreliable Inserting corrective terms into the derivation of the parameters of the parametric model, possibly signaled in the bitstream. -Unify the derivation process of parametric model parameters between LIC and CCLM - Extending CCLM for interblock and mixed intra-interblock.

[0041] Consider the general problem of predicting a current block Pcur(p) of samples at position p in a block of size W columns by H lines from their co-located reference samples Rref(p). Also consider that the samples are represented using a bit depth of B bits. In CCLM, the reference samples are reconstructed luma samples. In LIC, the reference samples are samples from motion-compensated blocks in the reference picture. Also, reconstructed current samples (Rcur) and reconstructed reference samples (Rref) are available in the neighborhood of the block to be predicted. This is shown in Figure 7. The neighboring samples are not necessarily in the nearest line / column of the block.

[0042] The objective is to derive Pcur(p) for p within the block from a parametric model calculated from Rref(p) within the block and from samples Rcur and Rref located adjacent to the block (typically in the upper lines and left columns outside the block).

[0043] Embodiment 1 - Use of two directly selected samples at specific locations within the reference sample array In one embodiment, to simplify the derivation of the parameters of the parametric model, the parameters are derived from at least two samples of adjacent samples, where the samples are selected to be spatially separated.

[0044] In one implementation, the following process is applied (the process described in FIG. 8). If both the top and left samples are available (step 401), the available sample of the outer top line in the rightmost position is selected (Rref A ,Rcur A ), the lowest available sample in the outer left column is selected (Rref B ,Rcur B ) (Step 403) (See illustration in Figure 9) Else, if a top sample is available (step 402), the available sample of the outer top line in the rightmost position is selected (Rref A ,Rcur A ), the available sample of the outermost top line in the leftmost position is selected (Rref B ,Rcur B ) (Step 405) (See illustration in FIG. 10) Otherwise, if a left sample is available (step 404), the lowest available sample in the outer left column is selected (Rref A ,Rcur A ), the available sample in the outer left column with the highest position is selected (Rref B ,Rcur B ) (Step 407) (See illustration in FIG. 11) Otherwise, the CCLM mode is not applied (step 406).

[0045] The parameters a and b are derived as shown in equations (8) and (9). a=(Rcur B -Rcur A ) / (Rref B -Rref A) Formula (8) b=Rcur A -a.Rref A Formula (9) Also, the intra-block prediction for any position p is calculated as in equation (10). Pcur(p)=a.Rref(p)+b Equation (10)

[0046] Compared to JVET-L0191, the solution avoids the multiple tests required to identify the minimum and maximum values of adjacent reference samples.

[0047] The same concept is directly applicable to MDLM mode. For example, when the top sample is selected for use in MDLM, the sample shown in Figure 10 is used. When the left sample is selected for use in MDLM, the sample shown in Figure 11 is used.

[0048] Embodiment 2 - Use of directly selected 3+ samples at specific locations within the reference sample array In this embodiment, to simplify the derivation of the parameters of the parametric model, the parameters are derived from at least three samples of adjacent samples, selected so that the samples are spatially separated, as shown in Figure 12. The concept also applies to the MDLM case shown in Figures 13 and 14.

[0049] Unlike the above-mentioned approach, where all samples are compared to find the minimum and maximum luma value samples, only three or more samples are used to calculate the minimum and maximum luma values, with the worst case being limited to two comparisons in the three sample case.

[0050] The linear model parameters are calculated as in the previous approach, equations (6) and (7) below.

[0051] As shown in FIG. 15, more than three samples at a particular position may be selected.

[0052] In another variant, a maximum of four samples are used as follows: For the reference sample of the top line of size Wtop, the sample at position x=0, x=Wtop-1 is used; For the reference sample of the left column of size Hleft, the sample at position y=0, y=Hleft-1 is used. This is shown in Figure 15(b).

[0053] In another variant, a maximum of six samples are used as follows: For the reference samples of the top line of size Wtop, the samples at positions x=0, x=Wtop-1 and one sample in the middle (e.g., position x=Wtop / 2) are used; For the reference samples of the left column of size Hleft, the samples at positions y=0, y=Hleft-1 and one sample in the middle (e.g., position y=Hleft / 2) are used. This is shown in Figure 15(c).

[0054] In an embodiment, the minimum and maximum values of the reference sample (L A and L B ), their minimum and maximum values are calculated using only the selected reference luma samples. In the above embodiment, the maximum number of reference samples is reduced to 2, 3, 4, 5, or 6, which significantly limits the number of tests required to identify the minimum and maximum luma sample values. In contribution JVET-L0191, in the worst case, this number of tests is equal to (Wtop+Hleft)×2 for a given block with Wtop top reference sample and Hleft left reference sample. Using the present invention, this number is reduced to 2×2, 3×2, 4×2, 5×2, or 6×2.

[0055] Embodiment 3 - Use of alternative modes when the linear model is not well defined The calculation of the linear parameters involves division, which in the case of LMS is constructed from equation (11). a=(SLC-SL.SC) / (SLL-SL.SL) Equation (11)

[0056] In the previous approach, it consists of equation (12). a=(Rcur B -Rcur A ) / (Rref B -Rref A ) Formula (12)

[0057] In both cases, a is obtained as a=Num / Den, where Num is the numerator of the division and Den is the denominator of the division. This can be problematic when Den is of small amplitude and can lead to unstable estimates of the linear parameters.

[0058] Also, for blocks of very small size, it may be considered that the number of samples for deriving the linear parameters is not large enough to obtain reliable estimates.

[0059] In an embodiment, prediction based on a linear model is used only if the derivation of the linear parameters is considered well-defined, otherwise an alternative mode is used (example block diagram in FIG. 16).

[0060] Different schemes for checking the reliability of the linear parameter derivation may be used, for example, if one of the following conditions is true, the linear parameter derivation is applied: -If Den>T1, - T1 is a predefined threshold that may depend on the block size, and B is the sample bit depth. For example, -T1=T2×W×H×2 B -T2 is a predefined threshold -If (WxH>Nmin), linear parameter derivation is applied -W and H are the width and height of the block Otherwise, a simplified model is used.

[0061] The thresholds T1 or T2 may also be signaled at various levels, for example, per SPS, PPS, slice, tile group, tile, CTU, or CU. A specific threshold may be signaled per block size.

[0062] An alternative mode may be based on using a simplified model. -Additive model: a is forced to 1 and only b is derived. Pcur(p)=Rref(p)+b -Scaling model: b is forced to 0 and only a is derived. Pcur(p)=a.Rref(p)

[0063] Embodiment 4 - Use of Correction Parameters in Derivation of Linear Parameters In an embodiment, a correction parameter CP is introduced into the equations used to derive the linear parameters.

[0064] The advantage of this correction parameter is that it increases coding efficiency compared to the prior art, thanks to the flexibility introduced by the multiple possible correction parameters CP.

[0065] In additive or multiplicative mode, CP may be used to correct the numerator or denominator when deriving the scaling parameters of a linear model. For example, the following correction modes may be applied: -Num'=CP×Num, and a=Num' / Den -Num'=(Num+CP×sign(Num)), and a=Num' / Den -Den' = CP × Den, and a = Num / Den' -Den'=(Den+CP×sign(Den)), and a=Num / Den'

[0066] The correction parameters CP may be signaled at various levels, for example, per SPS, PPS, slice, tile group, tile, CTU, or CU.

[0067] The parameters are given by K possible predefined values {CP0,CP1,...,CP K-1} may be fetched from a restricted set. Only indices corresponding to the index of values in this set may be coded.

[0068] CP may depend on Num or Den. In particular, when CP is an additive parameter, CP can be increased by the considered value as follows: -Num'=(Num+(abs(Num)>>K2)×sign(Num)), and a=Num' / Den -Den'=(Den+(abs(Den)>>K2)×sign(Den)), and a=Num / Den' Or alternatively, CP can be increased using a negative correction, as follows: -Num'=(Num-(abs(Num)>>K2)×sign(Num)), and a=Num' / Den -Den' = (Den - (abs(Den)>>K2) × sign(Den)), and a = Num / Den' Or alternatively, CP can be increased using a negative correction, as follows: -Num'=(Num-(abs(Num)>>K2)×sign(Num)), and a=Num' / Den -Den' = (Den - (abs(Den)>>K2) × sign(Den)), and a = Num / Den' K2 is a given predefined value, for example K2=6, which is equivalent to CP=k / 64. abs(x) is a function that returns the module of x.

[0069] Use of a Modified Lookup Table to Cause Division in Embodiment 4a To facilitate implementation, division associated with the derivation of linear parameters, which may complicate the implementation, can be achieved by a lookup table.

[0070] In fact, the division a = Num / Den can be achieved without division as a = (Num × Int((1 << K0) / Den) + offset0) >> K0, where K0 is a given value corresponding to the precision of the division, offset0 is a given offset value typically equal to (1 << (K0 - 1)), and Int() is an integer operator or floor operator (rounding to the nearest lower integer value).

[0071] More generally, it can be achieved as follows. a = (Num × (1 << Int(Den / (1 << K1))) × Int((1 << K0) / (Den % K1)) + offset0) >> K0 K1 is a given parameter that fixes the maximum size of the LUT (equal to (1 << K1)), and "%" is the modulo operator.

[0072] The value Int((1 << K0) / k) may be stored in the lookup table divLUT[k].

[0073] In an embodiment, the lookup table divLUT[k] is modified to introduce a bias in the estimation using a correction parameter CP. For example, the following correction modes may be applied. divLUT[k] = Int(2 K0 / (k + CP)) Equation (13) divLUT[k] = Int(2 K0 / (k × CP)) Equation (14) divLUT[k] = Int((2 K0 + CP) / k) Equation (15) divLUT[k] = Int((2 K0×CP) / k) Equation (16)

[0074] CP may depend on k. In particular, when CP is additive (as in the case of equation (13) or equation (15)), the module of CP can be augmented by k.

[0075] In an embodiment, CP=k>>K2, or CP=-k>>K2, where K2 is a given predefined value, for example, K2=6, which is equivalent to CP=k / 64 or CP=-k / 64.

[0076] The LUT may be stored in the decoder, or alternatively, it may be calculated on the fly, and the correction parameters CP or K2 may be signaled in the stream at various levels, for example, per SPS, PPS, slice, tile group, tile, CTU, or CU.

[0077] Embodiment 5 - Unification of LIC and CCLM In the current design of LIC, LMS processing is applied to derive the linear parameters. In the current CCLM, the linear parameters are derived from two sets of samples corresponding to the minimum and maximum values of the reference luma samples.

[0078] In an embodiment, the derivation of LIC parameters and CCLM parameters is unified and uses the same simplified process, e.g., the same derivation process based on identifying two sets of samples is used in both tools.

[0079] In one embodiment, both the LIC linear parameter derivation and the CCLM linear parameter derivation are A and Rref B corresponds to the minimum and maximum values of the neighboring reference samples (Rref A ,Rcur A ) and (Rref B ,Rcur B ) is identified.

[0080] In another embodiment, both the LIC linear parameter derivation and the CCLM linear parameter derivation are performed using two sets of samples (Rref A ,Rcur A ) and (Rref B ,Rcur B ) is identified.

[0081] In both cases, the linear parameters are derived as in equations (17) and (18), a=(Rcur B -Rcur A ) / (Rref B -Rref A ) Formula (17) b=Rcur A -a.Rref A Formula (18) The prediction in the block for any position p is calculated as in equation (19). Pcur(p)=a.Rref(p)+b Equation (19)

[0082] The variations discussed in embodiments 2 and 3 may also be applied to both cases.

[0083] Embodiment 6 - Extension of CCLM to Interblocks In the current design, CCLM is only applied to intra-CU or intra-block.

[0084] In an embodiment, CCLM is enabled to predict chroma components of inter-CUs. Therefore, a new mode, mixed inter-CCLM, is introduced herein. The mode may be signaled per CU using a CU-level flag. The luma component is coded using inter mode. - A full process of prediction and reconstruction of luma component samples is performed. A full reconstruction process is performed up to the full reconstruction of the luma block samples. The chroma component samples of a block are predicted using the reconstructed luma samples of the block using CCLM mode, i.e. using linear parameters calculated from neighboring reconstructed luma and chroma samples of the block. This means that the chroma component samples of the block are not constructed using temporal prediction.

[0085] In terms of pipelining operations, this new mode presents the same problem as the conventional CCLM mode: since reconstructed samples from neighbors are required, as well as reconstructed luma samples from the current block, it is desirable to delay processing of blocks coded with mixed inter CCLM mode once all intra- and inter-luma blocks have been processed.

[0086] Embodiment 7 - Extension of CCLM to mixed intra-inter blocks A new mode, mixed intra-inter, is introduced in the Versatilie Video Coding Test Model (VTM). The mode combines one intra prediction and one merge-indexed temporal prediction. For merged CUs, one flag is signaled for the merge mode to select an intra mode from the intra candidate list when the flag is true. For the luma component, an intra candidate list is derived from four intra prediction modes, including DC mode, planar mode, horizontal mode, and vertical mode, and the size of the intra candidate list may be three or four depending on the block shape. When the CU width is greater than twice the CU height, the horizontal mode is excluded from the intra mode list, and when the CU height is greater than twice the CU width, the vertical mode is removed from the intra mode list. One intra prediction mode selected by the intra mode index and one merge-indexed prediction selected by the merge index are combined using a weighted average. For chroma components, DM is always applied without extra signaling.

[0087] The weights for combining predictions are explained as follows (also shown in Figure 17): When DC mode or planar mode is selected, or the block width or height is less than 4, equal weights are applied. For those blocks with width and height equal to or greater than 4, when horizontal / vertical mode is selected, one block is first divided vertically / horizontally into four equal area regions. (w_intra i ,w_inter i), where i ranges from 1 to 4, and each weight set, (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6), is applied to a corresponding region. (w_intra1, w_inter1) is for the region closest to the reference sample, and (w_intra4, w_inter4) is for the region farthest from the reference sample. A combined prediction may then be calculated by summing the two weighted predictions and shifting three bits to the right. Furthermore, the intra prediction mode for the intra hypothesis of the predictor may be saved for subsequent reference to neighboring CUs.

[0088] In the proposed embodiment, CCLM is enabled to predict the chroma components of mixed intra-inter CUs. Therefore, a new mode, mixed inter CCLM, is introduced. The mode may be signaled per CU using a CU-level flag. The flag indicates whether DM mode or CCLM mode is used.

[0089] Instead of applying a DM mode for chroma, as in the other approaches, a CCLM mode is applied instead of the DM mode. The luma component is coded using mixed intra-inter mode. - A full process of prediction and reconstruction of luma component samples is performed. The chroma component samples of a block are predicted using the reconstructed luma samples of the block using CCLM mode, i.e. using linear parameters calculated from neighboring reconstructed luma and chroma samples of the block.

[0090] In the first version, there is no mixture of intra and inter prediction for the chroma components, and the chroma blocks are fully predicted using CCLM mode.

[0091] In a variant, weighted blending of intra prediction and inter prediction is still applied to the chroma components, similar to how the DM mode corresponds to the horizontal mode or the vertical mode, which means that the final prediction of chroma is a blend of inter prediction and CCLM. Blending processes described in the prior art may also be applied.

[0092] Alternatively, the same equal weights may be used for the entire chroma block, as in the prior art where DM corresponds to DC and planar modes.

[0093] As in the previous embodiment, in terms of pipelining the operations, it is possible to delay processing of blocks coded in mixed inter-CCLM mode once all intra-luma blocks, inter-luma blocks, and mixed intra-inter-luma blocks have been processed.

[0094] Reduced memory usage in CCLM In its actual implementation, the CCLM processing in contribution JVET-L0191 is achieved as follows (B represents the bit depth of the luma and chroma signals):

[0095] Minimum and maximum luma values L A , L B , and their associated chroma values C A , C B Once is identified, the linear parameters are derived as follows: The process is shown in the block diagram of Figure 20 with pseudocode descriptions for each step.

[0096] The variables a, b, and shift_pred are derived as follows: The parameters shift, add, diff, and k are derived as follows: - If (B>8), then shift is set equal to (B-9), otherwise shift is set equal to 0 (step 501). If -(shift>0), then add is set equal to (1<<(shift-1)), otherwise add is set equal to 0 (step 502). -diff=(L B -L A +add)>>shift (Step 503) -shift_pred=16 If -diff is greater than 0 (step 504), the following applies: -div=((C B -C A )×LUT_low[diff-1]+2 15 )>>16(Step 505) -a=((C B -C A ) × LUT_high[diff-1] + div + add) >> shift (Step 506) Otherwise (step 504), the following applies: -a=0 (Step 507) -b is derived as follows (step 508): -b=C A -((a×L A )>>shift_pred) LUT_high and LUT_low are two lookup tables of 512 elements, each of which is derived as follows: LUT_high[x]=Floor(2 16 / diff) LUT_low[x]=Floor(2 32 / diff)-Floor(2 16 / diff)x2 16 Floor(x) is the largest integer less than or equal to x.

[0097] The predicted sample Pcur(p) for any p in the chroma block is derived (step 509) as follows: -Pcur(p)=((pRef(p)×a)>>shift_pred)+b

[0098] Clipping is also applied to keep the signal within an allowed range defined by the signal bit depth.

[0099] The following problems are observed: Two lookup tables of -512 integers are required: LUT_high and LUT_low. For signals with more than -8 bits, a right shift of (B-9) is applied to derive the parameter a, which may lead to a loss of precision. When generating the predicted sample Pcur(p), a right shift by the parameter k is applied to the first term of the equation, which may lead to a loss of accuracy.

[0100] The following embodiments aim to address these issues, which may also be combined together.

[0101] Embodiment 8 - Elimination of one of the lookup tables In one embodiment, the processing is simplified by eliminating the lookup table LUT_low. The parameter a is derived as follows: a=((C B -C A )×LUT_high[diff-1]+add)>>shift

[0102] In one variation, LUT_high[x] is derived as follows: LUT_high[x]=Floor((2 16 +(diff / 2)) / diff) This allows reducing memory requirements by a factor of two.

[0103] The modified process is shown in Figure 21, with the changed blocks shown in bold: The new block is step 606 instead of the previous step 506, and the previous step 505 has been removed.

[0104] Embodiment 9 - Correcting Access to Lookup Tables In one embodiment, the access to the lookup table is modified as follows. shift=(L B -L A ) / 2 K , or similarly, shift=(L B -L A )>>K K is an integer less than B.

[0105] This allows for the following: -Lookup table size 2 K When K=8, this limits the table to 256 elements instead of 512 in the reference implementation of JVET-L0191. -2 B Even if is larger than the size of the lookup table, (L B -L A ) is 2 K This increases the accuracy of the calculation of a when it is smaller than 2 B As soon as is larger than the actual size of the lookup table (512), (L B -L A ) is split by (B-9), which is not the case in the implementation of the reference in JVET-L0191.

[0106] The modified process is shown in Figure 22, with the changed blocks shown in bold. The new block is step 701 instead of the previous step 506.

[0107] In an embodiment, an additional step 701a is introduced after step 701 and before step 502 to modify the shift value as follows: -If shift>0, shift=1+Floor(Log2(shift)) Log2(x) is the binary logarithm of x.

[0108] The modification is shown in Figure 23. For example, if K=8 (size 2 K = 256 element table) and the input signal bit depth is B = 10, the following results are obtained: -(L B -L A ) is between 0 and 255, shift is set equal to 0 -Not so, (L B -L A ) is between 256 and 511, then shift is set equal to 1 -Not so, (L B -L A ) is between 512 and 1023, then shift is set equal to 2 This process ensures that the value of (diff-1) remains within the maximum table index value.

[0109] Embodiment 10 - Linear Prediction Adaptation In one embodiment, to increase accuracy in the calculation of the predicted signal, the parameter b is calculated as follows: b=(C A < <shift_pred)-(a×L A )+(1<<(shift_pred-1)) Also, linear prediction is performed as follows: Pcur(p)=(pRef(p)×a+b)>>shift_pred

[0110] The modified process is shown in Figure 24, with the changed blocks shown in bold. The new blocks are step 808 instead of the previous step 508, and step 809 instead of the previous step 509.

[0111] One embodiment of a method 1800 under the general aspects described herein is shown in Figure 18. The method begins at start block 1801, and control passes to block 1810, which predicts samples in the current block from at least one of neighboring samples in the current block and from a parametric model calculated from the neighboring samples in the current block and reference samples in a reference frame. Control passes from block 710 to block 720, which encodes the block using the predicted samples.

[0112] Another embodiment of a method 2500 under the general aspects described herein is shown in Figure 25. The method begins at start block 2501, and control passes to block 2510, which predicts samples in the current block from at least one of neighboring samples in the current block and from a parametric model calculated from the neighboring samples in the current block and reference samples in a reference frame. Control passes from block 2510 to block 2520, which decodes the block using the predicted samples.

[0113] 26 shows one embodiment of an apparatus 2600 for encoding, decoding, compressing, or decompressing video data using a simplified coding mode based on an adjacent sample-dependent parametric model. The apparatus includes a processor 2610, which may be interconnected to a memory 2620 through at least one port. Both the processor 2610 and the memory 2620 may also have one or more additional interconnections to external connections.

[0114] The processor 2610 is also configured to either insert information into the bitstream or receive information in the bitstream, and to either compress, encode, or decode using any of the described aspects.

[0115] This application describes various aspects, including tools, mechanisms, embodiments, models, approaches, and the like. Many of these aspects are described in detail at least to illustrate their individual characteristics. However, this is for purposes of clarity of description and does not limit the scope of the application or these aspects. In fact, all of the different aspects may be combined or interchanged to provide further aspects. Moreover, aspects may be combined or interchanged with aspects also described in previous applications.

[0116] The aspects described and contemplated in this application may be implemented in many different forms. While Figures 5, 6, and 19 provide some embodiments, other embodiments are contemplated, and the discussion of Figures 5, 6, and 19 is not intended to limit the scope of implementation aspects. At least one of the aspects generally relates to video encoding and video decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects may be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0117] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side and the term "decoded" is used on the decoder side.

[0118] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for the proper operation of the method, the order and / or specific steps and / or actions may be modified or combined.

[0119] Various methods and other aspects described in this application may be used to modify modules, such as intra-prediction modules, entropy coding modules, and / or decoding modules (160, 360, 145, 330), of the video encoder 100 and decoder 200 of Figures 5 and 6. Moreover, the aspects are not limited to VVC or HEVC, but may also apply to, for example, other standards and recommendations, whether pre-existing or yet to be developed, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically precluded, the aspects described in this application may be used individually or in combination.

[0120] Various numerical values are used in this application, for example, specific values are intended, and the described aspects are not limited to those specific values.

[0121] 5 shows an encoder 100. Although variations of this encoder 100 are contemplated, the encoder 100 is described below for the sake of clarity without describing all possible variations.

[0122] Before being encoded, the video sequence may undergo a pre-coding process (101), for example, applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components, in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and may be added to the bitstream.

[0123] In the encoder 100, a picture is encoded by the encoder elements as described below. The picture to be encoded is partitioned (102) and processed, for example, in units of CUs. Each unit is encoded, for example, using either intra mode or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides whether to use intra mode or inter mode to encode the unit and indicates the intra / inter decision, for example, by a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0124] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, along with the motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may also bypass both the transform and quantization, i.e., the residual is coded directly without applying the transform and quantization processes.

[0125] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the predicted block are combined (155) to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture to reduce coding artifacts, for example, to perform deblocking / SAO (Sample Adaptive Offset) filtering. The filtered image is stored in a reference picture buffer (180).

[0126] Figure 6 shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs an encoding pass and a decoding pass reciprocally as described in Figure 5. The encoder 100 also generally performs video decoding as part of encoding the video data.

[0127] In particular, the decoder's input includes a video bitstream, such as may be generated by video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. Picture partition information indicates how the picture is partitioned. Thus, the decoder divides the picture according to the decoded picture partition information (235). The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode prediction residuals. The decoded prediction residuals and predicted blocks are combined (255) to reconstruct image blocks. The predicted blocks may be obtained from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275) (270). A loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0128] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0129] FIG. 19 shows a block diagram of an example system in which various aspects and embodiments can be implemented. System 1000 may be embodied as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, a personal computer, a laptop computer, a smartphone, a tablet computer, a digital multimedia set-top box, a digital television receiver, a personal video recording system, a connected home appliance, and a server. The elements of system 1000, alone or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, system 1000 processing elements and encoder / decoder elements are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the aspects described herein.

[0130] The system 1000 is configured to execute instructions loaded therein to implement various aspects described herein, for example. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a nonvolatile memory device). The system 1000 includes a storage device 1040, which may include nonvolatile and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drives, and / or optical disk drives. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device.

[0131] System 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. Encoder / decoder module 1030 represents a module or modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated within processor 1010 as a combination of hardware and software known to those skilled in the art.

[0132] Program code to be loaded into the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and then loaded into the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of mathematical expressions, formulas, operations, and computational logic.

[0133] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing necessary during encoding or decoding. However, for one or more of those functions, memory external to the processing device (e.g., the processing device may be either the processor 1010 or the encoder / decoder module 1030) is used. The external memory may be the memory 1020 and / or the storage device 1040, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, for example, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, a high-speed external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, with 13818-1 also known as H.222 and 13818-2 also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, an emerging standard being developed by the Joint Video Experts Team (JVET)).

[0134] Inputs to the elements of system 1000 may be provided through various input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives RF signals transmitted wirelessly by a broadcaster, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high definition multimedia interface (HDMI) input terminal. Other examples not shown in FIG. 19 include composite video.

[0135] In various embodiments, the input devices of block 1130 have associated respective input processing elements known in the art. For example, the RF section may be associated with appropriate elements to (i) select a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) downconvert the selected signal, (iii) band-limit again to a narrower band of frequencies to select a signal frequency band, which in particular embodiments may be referred to as a channel, (iv) demodulate the downconverted, band-limited signal, (v) perform error correction, and (vi) demultiplex to select a desired stream of data packets. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a multiplexer. The RF section may include, for example, a tuner that performs various of these functions, including downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of those elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0136] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 1000 to other electronic devices across the USB and / or HDMI connections. It will be appreciated that various aspects of input processing, e.g., Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or within processor 1010. Similarly, aspects of USB interface processing or HDMI interface processing may be implemented as needed within a separate interface IC or within processor 1010. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and an encoder / decoder 1030 operating in combination with memory and storage elements, to process the data streams as needed for presentation on an output device.

[0137] The various elements of system 1000 may be provided within an integrated housing, where the various elements may be interconnected and data may be transmitted therebetween using any suitable connection arrangement, for example, an internal bus known in the art, including an Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0138] System 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0139] In various embodiments, data is streamed or otherwise provided to system 1000 using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received through communication channel 1060 and communication interface 1050 adapted for Wi-Fi communication. Communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 1000 using a set-top box that delivers data through the HDMI connection of input block 1130. Still other embodiments provide streamed data to system 1000 using the RF connection of input block 1130. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0140] System 1000 may provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. Display 1100 in various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 1100 may be for a television, a tablet, a laptop, a cell phone, or other device. Display 1100 may also be integrated with other components (e.g., as in a smartphone) or may be separate (e.g., an external monitor for a laptop). Other peripheral devices 1120, in various example embodiments, include one or more of a standalone digital video disc (or digital versatile disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000.

[0141] In various embodiments, control signals are communicated between system 1000 and display 1100, speaker 1110, or other peripheral device 1120 using signaling such as AV Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 using communication channel 1060 via communication interface 1050. Display 1100 and speaker 1110 may be integrated in a single unit with other components of system 1000 in an electronic device such as a television. In various embodiments, display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0142] Display 1100 and speakers 1110 may instead be separate from one or more of the other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments in which display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0143] The embodiments may be performed by the processor 1010 or computer software implemented by hardware, or a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be any type of memory appropriate to the technical environment and may be implemented using any suitable data technology, such as, by way of non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be any type of processor appropriate to the technical environment and may include, by way of non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0144] Various implementations involve decoding. As used herein, "decoding" may include all or part of the processing performed on a received encoded sequence to generate a final output suitable for display, for example. In various embodiments, such processing includes one or more of the processing typically performed by a decoder, e.g., entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processing additionally or instead includes processing performed by the decoders of the various implementations described herein.

[0145] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. It is intended to be clear based on the context of the particular description and is believed to be known by those skilled in the art that the phrase "decoding process" refers to an operation or a subset of the decoding process more broadly overall.

[0146] Various implementations involve encoding. In a manner similar to "decoding," "encoding," as used herein, may include all or part of the processing performed on an input video sequence, e.g., to generate an encoded bitstream. In various embodiments, such processing includes one or more of the processing typically performed by an encoder, e.g., partitioning, differential encoding, transforming, quantizing, and entropy encoding. In various embodiments, such processing additionally or instead includes processing performed by the encoders of the various implementations described herein.

[0147] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and differential encoding. It is intended to be clear based on the context of the particular description and is believed to be known by those skilled in the art that the phrase "encoding process" refers to a subset of the operations or encoding processes more broadly overall.

[0148] It should be noted that the syntax elements used herein are descriptive terms. As such, they do not preclude the use of other syntax element names.

[0149] When a figure is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0150] Various embodiments refer to parametric models. In particular, a balance or trade-off between rate and distortion is usually considered during the encoding process, subject to significant computational complexity. This may be measured through a rate-distortion optimization (RDO) metric, least mean squares (LMS), mean absolute error (MAE), or other such measures. Rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of the rate and the distortion. Different approaches exist for solving the rate-distortion optimization problem. For example, an approach may be based on extensive testing of all encoding options, including all considered modes or coding parameter values, along with a thorough evaluation of their coding costs and associated distortions of the reconstructed signal after encoding and decoding. Faster approaches may also be used to reduce encoding complexity, particularly by calculating approximated distortions based on predicted or predicted residual signals rather than reconstructed ones. A hybrid of these two approaches may also be used, such as by using approximated distortions for only some of the possible encoding options and full distortions for other encoding options. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform optimization, but the optimization is not necessarily a complete assessment of both the coding cost and the associated distortion.

[0151] Implementations and aspects described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even when discussed only in the context of a single form of implementation (e.g., discussed as a method), the implementation of the discussed feature may be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. A method may be implemented in, for example, a processor, which refers to a processing device, and processing devices generally include, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic circuit. Processors also include, for example, communication devices such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0152] "One embodiment," "embodiment," "one implementation," or "implementation," along with other variations thereof, means that particular features, structures, characteristics, etc. described in connection with an embodiment are included in at least one embodiment. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "in one implementation," or "in an implementation," along with any other variations thereof, appearing in various places throughout this application do not necessarily all refer to the same embodiment.

[0153] Additionally, the application may refer to "determining" various portions of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0154] Additionally, the application may refer to "accessing" various portions of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, replicating information, calculating information, determining information, predicting information, or estimating information.

[0155] Additionally, the application may refer to "receiving" various portions of information. Receiving, along with "accessing," is intended to be broad terms. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" may typically involve, in one manner or another, operations such as, for example, storing information, processing information, transmitting information, moving information, duplicating information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0156] For example, the use of any of " / ," "and / or," and "at least one of" in the cases of "A / B," "A and / or B," and "at least one of A and B" is intended to encompass the selection of only the first listed option (A), the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass the selection of only the first listed option (A), the selection of only the second listed option (B), the selection of only the third listed option (C), the selection of only the first listed option and the second listed option (A and B), the selection of only the first listed option and the third listed option (A and C), the selection of only the second listed option and the third listed option (B and C), or the selection of all three options (A, B, and C). This may be expanded to list many items, as would be obvious to one of ordinary skill in the art.

[0157] Also, as used herein, the term "signaling" refers, among other things, to indicating something to a corresponding decoder. For example, in certain embodiments, an encoder signals a specific one of multiple coding modes or flags. In this way, in embodiments, the same parameters are used on both the encoder and decoder sides. Thus, for example, an encoder can send a specific parameter to a decoder (explicit signaling), so that the decoder can use the same specific parameter. Conversely, if the decoder already has a specific parameter as well as other parameters, signaling may be used without sending (implicit signaling) to simply enable the decoder to recognize and select the specific parameter. By avoiding sending any actual function, bit savings are realized in various embodiments. It will be appreciated that signaling can be achieved in various manners. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. While the foregoing relates to the verb form of the term "signaling," the term "signaling" may also be used herein as a noun.

[0158] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be, for example, stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of a described implementation. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0159] Several embodiments have been described. The features of those embodiments may be provided alone or in any combination. Furthermore, the embodiments may include one or more of the following features, devices, or aspects, alone or in any combination, across various claimed categories and types. Modifying the coding mode process applied in the decoder and / or encoder. - Enabling some enhanced coding mode prediction methods in the decoder and / or encoder. Inserting syntax elements into the signaling that allow the decoder to identify the coding mode prediction method to be used. - Selecting the coding mode prediction method to apply at the decoder based on these syntax elements. - Applying coding mode prediction methods for derivation at the decoder. - Deriving the parameters by the above prediction process and by eliminating look-up tables. - Deriving the parameters by the above prediction process and by modifying the look-up table. -Deriving prediction parameters using linear prediction. Adapting the residual in the encoder according to any of the embodiments discussed. - A bitstream or signal containing one or more of the described syntax elements or variations thereof. - A bitstream or signal including syntax carrying information generated according to any of the described embodiments. - generating, transmitting, receiving, and / or decoding according to any of the described embodiments. -A method, process, apparatus, instruction storage medium, data storage medium, or signal according to any of the described embodiments. Inserting syntax elements into the signaling that allow the decoder to determine the coding mode in a manner that corresponds to that used by the encoder. - Generating, transmitting, receiving, and / or decoding a bitstream or signal that includes one or more of the described syntax elements or variations thereof. A TV, set-top box, mobile phone, tablet, or other electronic device that performs coding mode decision according to any of the described embodiments. A TV, set-top box, mobile phone, tablet, or other electronic device that performs coding mode decisions according to any of the described embodiments and displays the resulting images (e.g., using a monitor, screen, or other type of display). A TV, set-top box, mobile phone, tablet, or other electronic device that selects, bandlimits, or tunes (e.g., using a tuner) a channel to receive a signal containing an encoded image and performs coding mode decision according to any of the described embodiments. A TV, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image wirelessly (e.g., using an antenna) and performs a coding mode decision. [Explanation of symbols]

[0160] 2600 equipment 2610 processor 2620 memory

Claims

[Claim 1] determining a CCLM intra prediction of a sample in a current block from at least one of neighboring samples in the current block and from a parametric model calculated from the neighboring samples of the current block, wherein four reference samples are used for the prediction, consisting of a reference sample from a top-neighboring row of reference samples and a reference sample from a left-neighboring column of reference samples, wherein two samples from the top-neighboring row of reference samples and two samples from the left-neighboring column of reference samples are used if available, and another sample in the top-neighboring row is used if only the top-neighboring row is available, and another sample in the left-neighboring column is used if only the left-neighboring column is available; a limited range of indexes into a single lookup table to perform division with adaptive precision for deriving parameters of said parametric model based on luma differences; encoding the samples in the current block based on the prediction; A method comprising:

Images