Simplification of coding mode based on adjacent sample related parameter model

Abstract

The invention relates to simplification of coding modes based on adjacent sample related parameter models. Aspects aim to improve and simplify the design of modes similar to CCLM or LIC based on adjacent sample related parameter models. The proposed modifications relate to the way of deriving the parameters of the parametric model and the way of designing a prediction tool based on the parametric model contained in a codec in a unified and simplified way. In one embodiment, a method proposes simplification of a cross-component linear model process for deriving linear parameters. It is proposed to derive a parameter instead of a least mean square method as a parameter of a straight line passing through two points corresponding to a minimum luminance value and a maximum luminance value in all luminance adjacent reconstructed samples.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201980071930.7, filed on November 1, 2019, entitled “Simplification of Decoding Mode Based on Adjacent Sample Correlation Parameter Model”, and the contents of the parent application are incorporated herein by reference. Technical Field

[0002] At least one embodiment of the present invention generally relates to a method or apparatus for video encoding or decoding, compression or decompression. Background Art

[0003] In order to achieve high compression efficiency, image and video coding schemes usually use prediction, which includes motion vector prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image and the predicted image, which is usually represented as a prediction error or prediction residual, is transformed, quantized and entropy decoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to the entropy decoding, quantization, transformation and prediction. Summary of the invention

[0004] At least one embodiment of the present invention generally relates to a method or apparatus for video encoding or decoding, and more particularly, to a method or apparatus for simplifying a decoding mode based on a parameter model related to adjacent samples.

[0005] According to a first aspect, a method is provided. The method comprises the steps of determining a prediction of a sample in a current block, the determination being based on at least one of neighboring samples in the current block and on a parameter model calculated from neighboring samples in the current block and reference samples in a reference frame; and encoding the sample in the current block based on the prediction.

[0006] According to a second aspect, a method is provided. The method comprises the steps of determining a prediction of a sample in a current block, the determination being based on at least one of neighboring samples in the current block and on a parameter model calculated from neighboring samples in the current block and reference samples in a reference frame; and decoding the sample in the current block based on the prediction.

[0007] According to another aspect, a device is provided. The device includes a processor. The processor can be configured to encode a block of a video or decode a bitstream by performing any of the above methods.

[0008] According to another main aspect of at least one embodiment, there is provided an apparatus comprising: an apparatus according to any of the decoding embodiments; and (i) an antenna configured to receive a signal comprising a video block, (ii) a band limiter configured to limit the received signal to a frequency band comprising the video block, or (iii) a display configured to display an output representing the video block.

[0009] According to another main aspect of at least one embodiment, a non-transitory computer-readable medium is provided, containing data content generated according to any of the described encoding embodiments or variations.

[0010] According to another main aspect of at least one embodiment, there is provided a signal comprising video data generated according to any of the described encoding embodiments or variations.

[0011] According to another main aspect of at least one embodiment, a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variations.

[0012] According to another main aspect of at least one embodiment, there is provided a computer program product comprising instructions which, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variations.

[0013] These and other aspects, features and advantages of the main aspects will become apparent from the following detailed description of exemplary embodiments, which is to be read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Example locations of samples used to derive a and b are shown.

[0015] Figure 2 An example of the LM_A mode is shown.

[0016] Figure 3 An example of the LM_L mode is shown.

[0017] Figure 4 A diagram of the LIC pattern in JEM is shown.

[0018] Figure 5 shows a standard general video compression scheme

[0019] Figure 6 A standard, common video decompression scheme is shown.

[0020] Figure 7 Samples selected from the top line at the rightmost position and the left column at the bottommost position are shown.

[0021] Figure 8 An exemplary block diagram using 2 samples at a specific location in the neighborhood is shown.

[0022] Fig. 9 Samples selected from the top row at the rightmost position and the bottom leftmost column are shown.

[0023] Fig.10 Samples selected from the top row in the rightmost and leftmost positions are shown.

[0024] Fig.11 Samples selected from the left column are shown in the topmost and bottommost positions.

[0025] Fig.12 Selected samples at the lower left, upper left and upper right positions are shown.

[0026] Fig.13 Samples of the rightmost and leftmost positions selected from the top row are shown.

[0027] Fig.14 A sample of a selection from the left column is shown in the topmost and bottommost positions.

[0028] Fig.15 Shown are (a) selected samples at more than three positions, (b) selected samples at the top two positions and 2 positions on the left, and (c) selected samples at the top three positions and three positions on the left.

[0029] Fig.16 An example block diagram for testing the reliability of a linear model derivation is shown.

[0030] Fig.17 The weights used in hybrid intra-inter prediction are shown.

[0031] Fig.18 An embodiment of a method according to the described aspects is shown.

[0032] Fig.19 An exemplary processor-based subsystem is shown for implementing the broadly described aspects.

[0033] Fig. 20 A block diagram of the CCLM / MDLM process is shown.

[0034] Fig.21 A block diagram showing a modification of the CCLM / MDLM process according to the first embodiment.

[0035] Fig. 22 A block diagram showing a modification of the CCLM / MDLM process according to the second embodiment.

[0036] Fig.23 A block diagram showing a modification of the CCLM / MDLM process according to a variation of the second embodiment.

[0037] Fig.24 A block diagram showing a modification of the CCLM / MDLM process according to the third embodiment.

[0038] Fig.25 A further embodiment of a method according to the described aspects is shown.

[0039] Fig.26 Example apparatus according to the described aspects is shown. DETAILED DESCRIPTION

[0040] The embodiments described herein belong to the field of video compression and are primarily related to video compression and video encoding and decoding.

[0041] In order to achieve high compression efficiency, image and video coding schemes generally use prediction (including motion vector prediction) and transform to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original image and the predicted image, which is usually represented as a prediction error or prediction residual, is transformed, quantized, and entropy decoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to the entropy decoding, quantization, transformation, and prediction.

[0042] In the HEVC (High Efficiency Video Coding, ISO / IEC 23008-2, ITU-T H.265) video compression standard, motion compensated temporal prediction is used to exploit the redundancy existing between consecutive pictures in a video.

[0043] To do this, a motion vector is associated with each prediction unit (PU). Each coding tree unit (CTU) is represented by a coding tree in the compressed domain. This is a quadtree partitioning of the CTU, where each leaf is called a coding unit (CU).

[0044] Each CU is then given some intra or inter prediction parameters (prediction information). To this end, it is spatially partitioned into one or more prediction units (PUs), each PU being assigned some prediction information. Intra or inter coding modes are assigned on CU level.

[0045] In the JVET (Joint Video Exploration Team) proposal for a new video compression standard, called the Joint Exploration Model (JEM), it has been proposed to accept a quadtree-binary tree (QTBT) block partition structure due to high compression performance. A block in a binary tree (BT) can be divided into two sub-blocks of equal size by dividing it horizontally or vertically in the middle. Therefore, a BT block can have a rectangular shape with unequal width and height, unlike a block in QT, which always has a square shape with equal height and width. In HEVC, the angular intra prediction directions are defined from 45° to -135° over a 180° angle, and they have been maintained in JEM, which has made the definition of the angular direction independent of the target block shape.

[0046] To encode these blocks, intra prediction is used to provide an estimated version of the block using previously reconstructed neighboring samples. The difference between the source block and the prediction is then encoded. In the above classical codec, a single row of reference samples is used, to the left and top of the current block.

[0047] In HEVC (High Efficiency Video Coding, H.265), the encoding of frames of a video sequence is based on a quadtree (QT) block partition structure. Frames are divided into square coding tree units (CTUs), all of which are divided into multiple coding units (CUs) based on a rate-distortion (RD) criterion and undergo quadtree-based partitioning. Each CU is intra-predicted (i.e., it is spatially predicted from a causally adjacent CU) or inter-predicted (i.e., it is temporally predicted from a decoded reference frame). In I slices, all CUs are intra-predicted, while in P and B slices, CUs can be intra- or inter-predicted. For intra prediction, HEVC defines 35 prediction modes, including one plane mode (indexed as mode 0), one DC mode (indexed as mode 1), and 33 angular modes (indexed as modes 2 to 34). The angular modes are associated with prediction directions ranging from 45 degrees to -135 degrees in the clockwise direction. Since HEVC supports a quadtree (QT) block partition structure, all prediction units (PUs) have a square shape. Therefore, from the perspective of the PU (prediction unit) shape, the definition of prediction angles from 45 degrees to -135 degrees is reasonable. For a target prediction unit of size N×N pixels, the top reference array and the left reference array each have a size of 2N+1 samples, which is required to cover the above angle range of all target pixels. Considering that the height and width of the PU have equal lengths, it is reasonable for the lengths of the two reference arrays to be equal.

[0048] The present invention belongs to the field of video compression, more specifically, the present invention focuses on modes that use a parametric model to perform the prediction of a given block, the parameters of the model being derived from the neighboring samples of said block. Two examples of such modes are the "Cross Component Linear Model" (CCLM) mode and the "Local Illumination Compensation" (LIC) mode. The purpose of the present invention is to simplify and improve the design of these modes.

[0049] Description of CCLM and variants

[0050] The following sections describe different variations of CCLM.

[0051] Basic CCLM Mode Description

[0052] In its initial version (cf_VET_K1002), the CCLM mode consists in predicting the chrominance samples based on the reconstructed luma samples of the same block or CU by using the following linear model:

[0053] pred C (i,j)=a.rec L '(i,j)+b(Equation 1)

[0054] where pred C (i,j) represents the predicted chroma sample in CU, rec L '(i,j) represents the downsampled reconstructed luma sample of the same CU. Parameters a and b are derived by minimizing the regression error between the adjacent reconstructed luma and chroma samples around the current block, as follows:

[0055] a=(SLC–SL.SC) / (SLL–SL.SL) (Equation 2)

[0056] b = SC – a.SL (Equation 3)

[0057] Where L(i,j) represents the downsampled top and left adjacent reconstructed luma samples, C(i,j) represents the top and left adjacent reconstructed chroma samples, N is equal to twice the minimum of the width and height of the current chroma decoding block, and SL, SC, SLL, SLC are defined as follows (the symbol represents the sum over the top and left adjacent samples):

[0058] SL=∑L(n)

[0059] SC = ∑C(n)

[0060] SLC = N ∑ (L(n) C(n))

[0061] SLL = N ∑ (L(n) L(n))

[0062] For a coding block with a square shape, the above two equations are directly applied. For a non-square coding block, the neighboring samples of the longer boundary are first subsampled to obtain the same number of samples as the shorter boundary. Figure 1 The samples of the current block and the positions of the left and above samples involved in CCLM mode are shown.

[0063] When a CU is decoded using CCLM mode, a least mean square (LMS) method is performed in the decoding process. As a result, no syntax is used to transmit a and b values ​​to the decoder.

[0064] MDLM Mode

[0065] The MDLM mode is an improvement on the basic CCLM design proposed in JVET-L0338, where in addition to the (top + left) reference sample templates, one can choose to derive the linear model coefficients α and β using only the left or only the top template. This means that 2 new CCLM modes called LM_A and LM_L are added.

[0066] In LM_A mode (see Figure 2 ), only the upper template is used to calculate the linear model coefficients. To get more samples, the upper template is expanded to (W+H), where W is the width of the block and H is its height. In LM_L mode (see Figure 3 ), only the left template is used to calculate the linear model coefficients. In order to obtain more samples, the left template is expanded to (H+W).

[0067] For non-square blocks, the upper template is expanded to W+W and the left template is expanded to H+H.

[0068] If the top / left template is not available, the LM_A / LM_L mode will not be checked or signaled. If the number of available samples is not large enough, the template is padded by copying the rightmost (for the top template) or bottommost (for the left template) sample to the nearest log2 number.

[0069] CCLM / MDLM with line buffer constraints

[0070] In the current CCLM mode coefficient derivation process, two luma line buffers are in principle used for downsampling to obtain the top template of the CCLM mode (CCLM or MDLM), while only one luma line buffer is used in the normal luma component intra prediction. In order to reduce the line buffer, only the LM_L mode is used for the CU and CTU top boundaries. In this case, no additional line buffer is required.

[0071] Description of local illumination compensation

[0072] In this tool, the decoder calculates some prediction parameters ( Figure 4 ). In the considered prior art codec (JEM), the use of LIC for a given block depends on a flag associated with the block, which is called the LIC flag.

[0073] The LIC parameters (a, b) are based on least mean square minimization, which minimizes the following distortion:

[0074] dist=∑ r∈Vcur,s∈Vref (Rcur(r)-a.Rref(s)-b) 2 (Equation 4)

[0075] Where Rcur(r) is the adjacent reconstructed sample and Rref(s) is the reference sample. The derivation of a and b is similar to the way a and b are derived in the previous section (Equations 2 and 3).

[0076] Once the encoder or decoder obtains the LIC parameters for the current CU, the prediction pred(i, j) of the current CU includes the following (unidirectional prediction case):

[0077] pred(i,j)=a.ref(i,j)+b (Equation 5)

[0078] where ref(i,j) is the reference block used for temporal prediction of the current block.

[0079] The main aspects described in this paper aim to improve and simplify the design of modes similar to CCLM or LIC based on a parametric model of neighboring sample correlation. The proposed modifications concern the way the parameters of said parametric model are derived and how the prediction tools based on said parametric model included in the codec can be designed in a unified and simplified way compared to the prior art.

[0080] One approach proposes to simplify the CCLM procedure for deriving linear parameters. An alternative LMS method is proposed to derive parameters a and b as parameters of a straight line passing through two points corresponding to the minimum and maximum luminance values ​​among all luminance-adjacent reconstructed samples.

[0081] The a and b values ​​are derived as follows:

[0082] a=(C B –C A ) / (L B –L A )(Equation 6)

[0083] b=C A –aLA (Equation 7)

[0084] Among them (L A ,C A ) is a pair of brightness and chrominance values ​​in adjacent reconstructed samples, L A has the minimum value among all brightness values, and (L B ,C B ) is a pair of brightness and chrominance values ​​in adjacent reconstructed samples, L B Has the maximum value among all brightness values.

[0085] This approach still requires performing multiple checks to identify the minimum and maximum brightness values. A and L A When it gets close, it may face problems.

[0086] The LMS method used in the original CCLM and LIC schemes has other problems. One important problem is that LMS can cause bias when the input samples are corrupted by noise, which is obviously present because the samples are obtained from decoding or prediction. This may reduce the decoding efficiency of the tool.

[0087] The main aspects described in this article propose various changes:

[0088] – Simplify sample selection for deriving parameters of the parametric model: retrieve samples from predefined locations

[0089] – Use alternative decoding modes when the derivation of the parameters of the parameter model is unreliable

[0090] - Insertion of correction terms, possibly signaled in the bitstream, in the derivation of the parameters of the parameter model

[0091] – Unify the derivation process of parameters of the parameter model between LIC and CCLM

[0092] – Extended CCLM for inter-blocks and hybrid intra-inter blocks.

[0093] Consider the general problem of predicting a current block of a sample Pcur(p) at position p in a block of size W columns x H rows from their co-located reference sample Rref(p). Also consider using a bit depth of B bits to represent the samples. In CCLM, the reference samples are reconstructed luma samples. In LIC, the reference samples are samples from a motion compensated block in a reference picture. Then, also consider that in the neighborhood of the block to be predicted, a reconstructed current sample (Rcur) and a reconstructed reference sample (Rref) are available. This is in Figure 7 Neighboring samples are not necessarily in the nearest row / column of the block.

[0094] The goal is to derive Pcur(p) for p in a block from Rref(p) in the block and from a parametric model computed from samples Rcur and Rref located in a neighborhood of the block (usually the upper rows and left columns outside the block).

[0095] Example 1 - Using 2 samples directly selected at specific positions in the reference sample array

[0096] In one embodiment, in order to simplify the derivation of parameters of the parametric model, the parameters are derived from at least 2 samples of neighboring samples, the at least two samples being selected such that the samples are spatially far apart.

[0097] In one implementation, the following process is applied: Figure 8 )

[0098] - If both top and left samples are available (step 401), select the available sample of the outer top row in the rightmost position (Rref A ,Rcur A ), and select the available sample (Rref) in the outer left column in the bottommost position B ,Rcur B )(Step 403)(See Fig. 9 ).

[0099] – Otherwise, if only top samples are available (step 402), select the available sample of the outer top row at the rightmost position (Rref A ,Rcur A ), and select the available sample in the outer top row at the leftmost position (Rref B ,Rcur B )(Step 405)(See Fig.10 )

[0100] – Otherwise, if only left samples are available (step 404), select the available sample in the outer left column at the bottom position (Rref A ,Rcur A ), and select the available sample in the outer left column at the top position (Rref B ,Rcur B )(Step 407)(See Fig.11 ).

[0101] – Otherwise, CCLM mode is not applied (step 406).

[0102] The parameters a and b are derived as follows:

[0103] a=(Rcur B –Rcur A ) / (Rref B –Rref A )(Equation 8)

[0104] b=Rcur A –a.Rref A (Equation 9)

[0105] And the prediction for any position p in the block is calculated as:

[0106] Pcur(p) = a.Rref(p) + b (Equation 10)

[0107] Compared to JVET-L0191, this solution avoids the multiple checks required to identify the minimum and maximum values ​​of reference samples in the neighborhood.

[0108] The same concept can be directly applied to MDLM mode. For example, when selecting the top sample for MDLM, use Fig.10 When the sample on the left is selected for use with MDLM, the Fig.11 Sample shown.

[0109] Example 2 - Using 3+ samples directly selected at specific positions in a reference sample array

[0110] In this embodiment, in order to simplify the derivation of the parameters of the parameter model, the parameters are derived from at least three samples of the adjacent samples, and the at least three samples are selected so that the samples are as follows: Fig.12 Shown as being spatially distant.

[0111] This concept also applies to MDLM situations, such as Fig.13 and Fig.14 shown.

[0112] Unlike the previously mentioned method of comparing all samples to find the minimum and maximum brightness value samples, only 3 or more samples are used to calculate the minimum and maximum brightness values. The worst case is limited to two comparisons in the three sample case.

[0113] Following the previous method, the linear model parameters are calculated according to Equations 6 and 7.

[0114] like Fig.15 As shown in (a), more than three samples can be selected at a specific location.

[0115] In another variation, up to 4 samples are used as follows. For the reference sample of the top row of size Wtop, the sample at position x=0, x=Wtop-1 is used. For the reference sample of the left column of size Hleft, the sample at position y=0, y=Hleft-1 is used. This Fig.15 As shown in (b).

[0116] In another variation, up to 6 samples are used as follows. For the reference samples of the top row of size Wtop, the samples at positions x=0, x=Wtop-1 and one sample in the middle (e.g., position x=Wtop / 2) are used. For the reference samples of the left column of size Hleft, the samples at positions y=0, y=Hleft-1 and one sample in the middle (e.g., position y=Hleft / 2) are used. This Fig.15 As shown in (c).

[0117] In an embodiment, based on a reference sample (L A and L B ), only the selected reference luminance samples are used to calculate these minimum and maximum values. Since the maximum number of reference samples is reduced to 2, 3, 4, 5 or 6 in the above embodiments, this significantly limits the number of checks required to identify the minimum and maximum luminance sample values. In contribution JVET-L0191, in the worst case, for a given block with Wtop top reference samples and Hleft left reference samples, the number of checks is equal to (Wtop+Hleft)x2. Using the present invention, this number is reduced to 2x2, 3x2, 4x2, 5x2 or 6x2.

[0118] Example 3 - When the linear model is not well defined, use alternative models

[0119] The calculation of linear parameters involves division. In the case of LMS, it consists in:

[0120] a = (SLC - SL.SC) / (SLL - SL.SL) (Equation 11)

[0121] In the previous method, it is

[0122] a=(Rcur B –Rcur A ) / (Rref B –Rref A )(Equation 12)

[0123] In both cases, a is obtained as a=Num / Den, where Num is the numerator and Den is the denominator of the division. This can be problematic when Den has a small magnitude, which can lead to unstable estimates of linear parameters.

[0124] It can also be considered that for blocks of too small a size, the number of samples used to derive the linear parameters is insufficient to obtain a reliable estimate.

[0125] In one embodiment, linear model based prediction is used only if the linear parameter derivation is considered to be well defined. Otherwise, an alternative model ( Fig.16 ).

[0126] Different ways of checking the reliability of the linear parameter derivation can be used. For example, the linear parameter derivation is applied if one of the following conditions is true:

[0127] -If Den>T1,

[0128] ○ Where T1 is a predefined threshold which may depend on the block size, and B is the sample bit depth. For example,

[0129] ■T1=T2*W*H*2^B

[0130] ■Where T2 is a predefined threshold

[0131] - If (WxH>Nmin), then apply the linear parameter derivation

[0132] ○Where W and H are the width and height of the block

[0133] Otherwise, the simplified model is used.

[0134] The threshold T1 or T2 may also be signaled at various levels (eg, per SPS, PPS, slice, tile group, tile, CTU, or CU). A specific threshold may be signaled per block size.

[0135] The alternative model may be based on using a simplified model:

[0136] Additive model: a is forced to 1, and only b is derived.

[0137] Pcur(p)=Rref(p)+b

[0138] · Scaling model: b is forced to 0, and only a is derived.

[0139] Pcur(p)=a.Rref(p)

[0140] Example 4 - Use of correction parameters in the derivation of linear parameters

[0141] In one embodiment, a correction parameter CP is introduced in the formula used to derive the linear parameters.

[0142] Compared with the prior art, due to the flexibility introduced by multiple possible correction parameters CP, the correction parameter has the advantage of improving the decoding efficiency.

[0143] When the scaling parameter a of the linear model is derived in additive or multiplicative mode, CP can be used to correct the numerator or denominator. For example, the following correction mode can be applied:

[0144] -Num'=CP*Num and a=Num' / Den

[0145] -Num'=(Num+CP*sign(Num)) and a=Num' / Den

[0146] -Den'=CP*Den and a=Num / Den'

[0147] -Den'=(Den+CP*sign(Den)) and a=Num / Den'

[0148] The correction parameters CP may be signaled at various levels (eg, per SPS, PPS, slice, tile group, tile, CTU, or CU).

[0149] can be selected from K possible predefined values ​​{CP0, CP1, …, CP K-1 The parameter can be retrieved from a finite set of}. Only the index corresponding to the index of the value in the set can be decoded.

[0150] CP can depend on Num or Den. In particular, when CP is additive, CP can increase with the value considered:

[0151] -Num'=(Num+(abs(Num)>>K2)*sign(Num)) and a=Num' / Den

[0152] -Den'=(Den+(abs(Den)>>K2)*sign(Den)) and a=Num / Den'

[0153] Or alternatively, use negative correction:

[0154] -Num'=(Num-(abs(Num)>>K2)*sign(Num)) and a=Num' / Den

[0155] -Den’ = (Den - (abs(Den) >> K2) * sign(Den)) and a = Num / Den’

[0156] Alternatively, negative correction is used

[0157] -Num’ = (Num - (abs(Num) >> K2) * sign(Num)) and a = Num’ / Den

[0158] -Den’ = (Den - (abs(Den) >> K2) * sign(Den)) and a = Num / Den’

[0159] where K2 is a given predetermined value. For example, K2 = 6, which corresponds to CP = k / 64. abs(x) is a function that returns the modulus of x.

[0160] Example 4a - Using a modified lookup table to generate the division

[0161] To simplify the implementation, the division involved in the derivation of the linear parameter can be done through a lookup table (which may increase the implementation complexity).

[0162] In fact, the division

[0163] a = Num / Den

[0164] can be implemented without any division as:

[0165] a = (Num * Int((1 << K0) / Den) + offset0) >> K0

[0166] where K0 is a given value corresponding to the division precision, offset0 is a given offset value, usually equal to (1 << (K0 - 1)), and int() is an integer or base operator (rounded to the nearest lower integer value).

[0167] More generally, it can be implemented as follows:

[0168] a = (Num * (1 << Int(Den / (1 << K1))) * Int((1 << K0) / (Den % K1)) + offset0) >> K0

[0169] where K1 is a given parameter that fixes the maximum size of the LUT (equal to (1 << K1)), and "%" is the modulo operator.

[0170] The value Int((1 << K0) / k) can be stored in the lookup table divLUT[k].

[0171] In one embodiment, the lookup table divLUT[k] is modified using a correction parameter CP to introduce a bias in the estimate. For example, the following correction pattern may be applied:

[0172] divLUT[k]=Int(2^K0 / (k+CP)) (Equation 13)

[0173] divLUT[k]=Int(2^K0 / (k*CP))(Equation 14)

[0174] divLUT[k]=Int((2^K0+CP) / k) (Equation 15)

[0175] divLUT[k]=Int((2^K0*CP) / k) (Equation 16)

[0176] CP may depend on k. In particular, when CP is additive (as in the case of Equation 13 or 15), the CP module may increase with k.

[0177] In one example,

[0178] CP=k>>K2,

[0179] or

[0180] CP=-k>>K2,

[0181] Wherein K2 is a given predetermined value. For example, K2=6, which is equivalent to CP=k / 64 or CP=-k / 64.

[0182] The LUT may be stored in the decoder. Alternatively, it may be calculated on-the-fly and the correction parameters CP or K2 may be signaled in the stream at various levels (e.g., per SPS, PPS, slice, tile group, tile, CTU or CU).

[0183] Example 5 - Unifying LIC and CCLM

[0184] In the current design of LIC, the LMS process is applied to derive the linear parameters. While in the current CCLM, the linear parameters are derived from two sets of samples corresponding to the minimum and maximum values ​​of the reference brightness samples.

[0185] In one embodiment, the derivation of LIC parameters and CCLM parameters is unified and uses the same simplified process. For example, the same derivation process based on identifying two groups of samples is used in both tools.

[0186] In one embodiment, both LIC and CCLM linear parameter derivation are to identify the two sets of sample sets (Rref A,Rcur A ) and (Rref B ,Rcur B ), where Rref A and Rref B Corresponds to the minimum and maximum values ​​of adjacent reference samples.

[0187] In another embodiment, both LIC and CCLM linear parameter derivation consists in identifying two groups of samples (Rref A ,Rcur A ) and (Rref B ,Rcur B ).

[0188] In both cases, the linear parameters are derived as:

[0189] a=(Rcur B –Rcur A ) / (Rref B –Rref A )(Equation 17)

[0190] b=Rcur A –a.Rref A (Equation 18)

[0191] And the prediction for any position p in the block is calculated as:

[0192] Pcur(p) = a.Rref(p) + b (Equation 19)

[0193] The variations discussed in Examples 2 and 3 can also be applied to these two cases.

[0194] Example 6 - Extending CCLM to inter-frame blocks

[0195] In the current design, CCLM is only applied to intra CUs or blocks.

[0196] In one embodiment, CCLM is enabled to predict the chroma components of inter CUs. Therefore, a new mode, hybrid inter CCLM, is introduced here. The mode can be signaled per CU using a CU level flag.

[0197] - The luminance component is decoded using inter-frame mode.

[0198] -Perform the whole process of prediction and reconstruction of the luma component samples.

[0199] o Perform a complete reconstruction process until complete reconstruction of the luma block samples.

[0200] - predicting the chroma component samples of the block by using the reconstructed luma samples of the block, by using the CCLM mode, ie using linear parameters calculated from neighboring reconstructed luma and chroma samples of the block.

[0201] ○ This means that no temporal prediction is used to construct the chroma component samples of the block.

[0202] In terms of the pipeline of operation, this new mode creates the same problems as the traditional CCLM mode. Since reconstructed samples from the neighborhood are required as well as reconstructed luma samples from the current block, the processing of blocks decoded in the hybrid inter-CCLM mode is preferably delayed once all intra and inter luma blocks have been processed.

[0203] Example 7 - Extending CCLM to Mix Intra-Inter Blocks

[0204] In VTM (Common Video Decoding Test Model), a new mode, namely, hybrid intra-inter, is introduced. This mode combines an intra prediction and a merge index temporal prediction. In the merged CU, a flag is signaled for the merge mode to select an intra mode from the intra candidate list when the flag is true. For the luma component, the intra candidate list is derived from four intra prediction modes including DC, planar, horizontal and vertical modes, and the size of the intra candidate list can be 3 or 4, depending on the block shape. When the CU width is greater than twice the CU height, the horizontal mode is not included in the intra mode list, and when the CU height is greater than twice the CU width, the vertical mode is removed from the intra mode list. Weighted averaging is used to combine one intra prediction mode selected by the intra mode index and one merge index prediction selected by the merge index. For chroma components, DM is always applied without additional signaling.

[0205] The weights used to combine the predictions are described as follows (also in Fig.17 When DC or planar mode is selected or the width or height of the block is less than 4, equal weights are applied. For those blocks with width and height greater than or equal to 4, when horizontal / vertical mode is selected, a block is first split vertically / horizontally into four equal-area regions. Each weight set (denoted as (w_intra i ,w_inter i), where i is 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6)) is used for the area closest to the reference sample, and (w_intra4, w_inter4) is used for the area farthest from the reference sample. Then, the combined prediction can be calculated by adding the two weighted predictions and shifting right by 3 bits. In addition, the intra-frame prediction mode of the intra-frame hypothesis of the prediction value can be retained for reference by subsequent neighboring CUs.

[0206] In the proposed embodiment, CCLM is enabled to predict the chroma components of a mixed intra-inter CU. Therefore, a new mode, hybrid inter-CCLM, is introduced. The mode can be signaled per CU using a CU level flag. The flag indicates whether DM or CCLM mode is used.

[0207] Alternatively, instead of applying the DM model to chrominance as done in other methods, a CCLM model is applied instead of DM.

[0208] - The luminance component is decoded using a hybrid intra-inter mode.

[0209] -Perform the whole process of prediction and reconstruction of the luma component samples.

[0210] - predicting the chroma component samples of the block by using the reconstructed luma samples of the block, by using said CCLM mode, ie using linear parameters calculated from neighboring reconstructed luma and chroma samples of the block.

[0211] In the first version, there is no mixing of intra and inter prediction for chroma components, and the chroma blocks are fully predicted using the CCLM mode.

[0212] In a variant, as done when DM corresponds to horizontal or vertical mode, a weighted mix of intra and inter prediction is still applied to the chrominance components, which means that the final prediction of chrominance is a mix of inter prediction and CCLM. The mixing methods described in the prior art can be applied.

[0213] Alternatively, the same weights may be used for the entire chroma block, as is done in the prior art in the case where DM corresponds to DC and planar modes.

[0214] As in previous embodiments, with respect to pipelining of operations, processing of blocks coded in mixed Inter-CCLM mode may be delayed once all Intra, Inter, and mixed Intra-Inter luminance blocks have been processed.

[0215] Reduction of memory size used in CCLM

[0216] In its actual implementation, the CCLM process in contribution JVET-L0191 is implemented as follows (where B represents the bit depth of the luminance and chrominance signals).

[0217] Once the minimum and maximum brightness values ​​L A , L B and their associated chromaticity values ​​C A , C B is identified, and the linear parameters are derived as follows. Fig. 20 The process is illustrated in the block diagram using pseudocode to describe each step.

[0218] The variables a, b and shift_pred are derived as follows:

[0219] - The parameters shift, add, diff and k are derived as follows:

[0220] ■If (B>8), shift is set equal to (B-9), otherwise shift is set equal to 0 (step 501)

[0221] ■If (shift>0), add is set equal to (1<<(shift-1)), otherwise it is set equal to (step 502)

[0222] ■diff=(L B -L A +add)>>shift (step 503)

[0223] ■shift_pred=16

[0224] - If diff is greater than 0 (step 504), the following steps apply:

[0225] ■div=((C B -C A )×LUT_low[diff-1]+2 15 )>>16 (Step 505)

[0226] ■a=((C B -C A )×LUT_high[diff-1]+div+add)>>shift

[0227] (Step 506)

[0228] - Otherwise (step 504), the following steps apply:

[0229] ■a=0 (Step 507)

[0230] -b is exported as follows (step 508)

[0231] ■b=C A -((a×L A )>>shift_pred)

[0232] LUT_high and LUT_low are 2 lookup tables with 512 elements, each element is derived as follows.

[0233] LUT_high[x]=Floor(2 16 / diff)

[0234] LUT_low[x]=Floor(2 32 / diff)-Floor(2 16 / diff)x 2 16

[0235] Floor(x) is the largest integer less than or equal to x

[0236] For any p in the chroma block, the prediction sample Pcur(p) is derived as follows (step 509):

[0237] ■Pcur(p)=((pRef(p)×a)>>shift_pred)+b

[0238] Limiting is also applied to keep the signal within the allowed range defined by the signal bit depth.

[0239] The following issues were observed:

[0240] - Requires 2 lookup tables with 512 integers, namely, LUT_high and LUT_low

[0241] - For signals larger than 8 bits, a right shift (B-9) is applied to derive parameter a, which may result in a loss of precision

[0242] -When generating the prediction sample Pcur(p), a right shift of parameter k is applied to the first term of the formula, which may result in a loss of accuracy

[0243] The following embodiments are intended to solve these problems. They can be combined together.

[0244] Example 8 - Removal of one of the lookup tables

[0245] In one embodiment, the process is simplified by removing the lookup table LUT_low. Parameter a is derived as follows.

[0246] a=((CB -C A )×LUT_high[diff-1]+add)>>shift

[0247] In one variation, LUT_high[x] is derived as follows:

[0248] LUT_high[x]=Floor((2 16 +(diff / 2)) / diff)

[0249] This makes it possible to reduce the memory requirements by a factor of 2.

[0250] Fig.21 The modified process is shown in , where the changed boxes are indicated in bold. The new box is step 606, which replaces the previous step 506. The previous step 505 is removed.

[0251] Example 9 - Modifying access to a lookup table

[0252] In one embodiment, access to the lookup table is modified as follows.

[0253] shift=(L B -L A ) / 2 K

[0254] or equivalent

[0255] shift=(L B -L A )>>K

[0256] Where K is an integer value lower than B.

[0257] This makes:

[0258] - Reduced the size of the lookup table to 2 K When K = 8, this limits the table to 256 elements instead of 512 elements in the reference implementation of JVET-L0191.

[0259] -When (L B -L A ) is less than 2 K When, even 2 B This is not the case in the reference implementation of JVET-L0191, where once 2 B is higher than the actual lookup table size (512), then (L B -L A ) is divided by (B-9).

[0260] Fig. 22The modified process is shown in , where the changed blocks are indicated in bold. The new block is step 701 , which replaces the previous step 501 .

[0261] In one embodiment, an additional step 701a is introduced after step 701 and before step 502 to modify the shift value as follows.

[0262] -If shift>0, then shift=1+Floor(Log2(shift))

[0263] Where Log2(x) is the base 2 logarithm of x.

[0264] The changes are shown in Fig.23 middle.

[0265] For example, for K=8 (size 2 K =256 elements) and the input signal bit depth is B=10, the following results are obtained:

[0266] -If ((L B -L A ) is from 0 to 255, shift is set equal to 0

[0267] - Otherwise, if ((L B -L A ) from 256 to 511, the shift is set equal to 1

[0268] - Otherwise, if ((L B -L A ) from 512 to 1023, the shift is set equal to 2

[0269] This process ensures that the value of (diff-1) remains within the maximum table index value.

[0270] Example 10 - Adaptation of Linear Prediction

[0271] In one embodiment, in order to obtain higher accuracy in the calculation of the prediction signal, the parameter b is calculated as follows:

[0272] b=(C A < <shift_pred)-(a×L A )+(1<<(shift_pred-1))

[0273] And the linear prediction is performed as follows.

[0274] Pcur(p)=(pRef(p)×a+b)>>shift_pred

[0275] Fig.24The modified process is shown in , where the changed blocks are indicated in bold. The new blocks are step 808 replacing the previous step 508 and step 809 replacing the previous step 509.

[0276] Fig.18 An embodiment of a method 1800 according to the main aspects described herein is shown. The method starts at start block 1801 and control passes to block 1810 for predicting samples in a current block based on at least one of neighboring samples in the current block and based on a parameter model calculated from neighboring samples in the current block and reference samples in a reference frame. Control passes from block 710 to block 720 to encode the block using the predicted samples.

[0277] Fig.25 Another embodiment of a method 2500 under the main aspects described herein is shown. The method starts at block 2501 and control passes to block 2510 for predicting samples in a current block based on at least one of neighboring samples in the current block and based on a parameter model calculated from neighboring samples in the current block and reference samples in a reference frame. Control passes from block 2510 to block 2520 to decode the block using the predicted samples.

[0278] Fig.26 One embodiment of a device 2600 for encoding, decoding, compressing or decompressing video data using a simplification of a coding mode based on a model of neighboring sample correlation parameters is shown. The device includes a processor 2610 and may be interconnected to a memory 2620 through at least one port. Both the processor 2610 and the memory 2620 may also have one or more additional interconnections to external connections.

[0279] Processor 2610 is also configured to insert or receive information in a bitstream and compress, encode, or decode using any of the described aspects.

[0280] The present application describes a number of aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described as having specificity, and at least in order to illustrate individual characteristics, are usually described in a manner that may sound limited. However, this is for the purpose of describing clearly, and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide other aspects. In addition, these aspects can also be combined and interchanged with the aspects described in earlier documents.

[0281] The aspects described and contemplated in this application can be implemented in many different forms. Figure 5 , 6 19 provide some embodiments, but other embodiments are contemplated and are Figure 5, 6 The discussion of 19 does not limit the breadth of implementation. At least one of the aspects is generally related to video encoding and decoding, and at least one other aspect is generally related to transmitting the generated or encoded bitstream. These and other aspects can be implemented as methods, devices, computer-readable storage media having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having stored thereon bitstreams generated according to any of the described methods.

[0282] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0283] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0284] Various methods and other aspects described in this application can be used to modify modules, e.g. Figure 5 and Figure 6 Intra prediction, entropy coding and / or decoding modules (160, 260, 145, 230) are shown. In addition, the present invention is not limited to VVC or HEVC, and can be applied to, for example, other standards and proposals (whether pre-existing or developed in the future) and extensions of any such standards and proposals (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0285] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0286] Figure 5 An encoder 100 is shown. Variations of the encoder 100 are contemplated, but for clarity, the encoder 100 is described below without describing all contemplated variations.

[0287] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0288] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (102) and processed in units such as CUs. Each unit is encoded using, for example, intra or inter mode. When the unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the unit, and indicates the intra / inter decision by, for example, a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the prediction block from the original image block.

[0289] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may skip the transform and directly apply quantization to the untransformed residual signal. The encoder may bypass both transform and quantization, i.e., directly code the residual without applying the transform or quantization process.

[0290] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0291] Figure 6 2 shows a block diagram of a video decoder 200. In the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described below. Figure 5 The encoding process described in 1 is a decoding process that is the inverse of the encoding process described in 1. The encoder 100 also typically performs video decoding as part of encoding the video data.

[0292] In particular, the input to the decoder comprises a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors and other decoding information. Picture partition information indicates how the picture is partitioned. The decoder can therefore partition (235) the picture according to the decoded picture partition information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined (255) with a prediction block to reconstruct the image block. The prediction block may be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0293] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or performing an inverse remapping of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0294] Fig.19 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 1000 can be implemented as a device including various components described below, and is configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1000 can be implemented in a single integrated circuit (IC), multiple ICs and / or discrete components individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed on multiple ICs and / or discrete components. In various embodiments, the system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more aspects described herein.

[0295] The system 1000 includes at least one processor 1010, which is configured to execute instructions loaded therein, for implementing various aspects described herein, for example. The processor 1010 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical disk drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device (including a removable storage device and a non-removable storage device) and / or a network accessible storage device.

[0296] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding module and the decoding module. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated into the processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0297] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of the various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of the decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0298] In some embodiments, memory within the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store, for example, an operating system for a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as working memory for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by Joint Video Team experts JVET).

[0299] As shown in block 1130, input to the elements of system 1000 may be provided through various input devices. Such input devices include, but are not limited to: (i) an RF portion that receives a radio frequency (RF) signal transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input terminal (or a set of component input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Fig.19 Other examples not shown include composite video.

[0300] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band limiting the frequency band to a narrower frequency band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its relevant input processing element receive the RF signal that sends by wired (for example, cable) medium, and carry out the frequency selection to the desired frequency band by filtering, down-conversion and filtering again.Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example insert amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0301] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Solomon error correction) may be implemented as needed, for example, in a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed in a separate interface IC or in processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on output devices.

[0302] The various components of system 1000 may be disposed in an integrated housing in which the various components may be interconnected and transmit data using a suitable connection arrangement (e.g., an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and a printed circuit board).

[0303] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0304] In various embodiments, data is streamed or otherwise provided to the system 1000 using a wireless network (e.g., a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)). The Wi-Fi signals of these embodiments are received through a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to an external network including the Internet to allow streaming applications and other cloud communications. Other embodiments use a set-top box that transmits data through an HDMI connection of an input box 1130 to provide streaming data to the system 1000. Still other embodiments use an RF connection of an input box 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use a wireless network other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0305] The system 1000 can provide output signals to various output devices (including a display 1100, a speaker 1110, and other peripheral devices 1120). The display 1100 of various embodiments includes one or more of the following: for example, a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 1100 can also be integrated with other components (for example, as in a smart phone), or it can be separate (for example, an external monitor for a laptop computer). In various examples of various embodiments, the other peripheral devices 1120 include one or more of the following: a stand-alone digital video disk (or digital versatile disk) (DVR, for both), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functions based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0306] In various embodiments, control signals are transmitted between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. The output devices may be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices may be connected to the system 1000 via a communication interface 1050 using a communication channel 1060. The display 1100 and speaker 1110 may be integrated into a single unit in an electronic device (e.g., a television) along with other components of the system 1000. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0307] For example, if the RF portion of the input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more of the other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0308] These embodiments may be implemented by the processor 1010 or by computer software implemented by hardware or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, as a non-limiting example, one or more of the following: a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0309] Various implementations involve decoding. As used in the present application, "decoding" may include, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of the various implementations described in the present application.

[0310] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0311] Various implementations involve encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy decoding. In various embodiments, such processes also or alternatively include processes performed by encoders of the various implementations described in this application.

[0312] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy decoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally refer to a broader encoding process will become clear based on the context of the specific description and is believed to be fully understood by those skilled in the art.

[0313] Note that the syntax elements as used herein are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0314] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0315] Various embodiments relate to parameter models. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, usually given a constraint on computational complexity. It can be measured by rate-distortion optimization (RDO) metric, or by least mean square (LMS), mean absolute error (MAE) or other such measurements. The rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options, including all considered modes or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to save coding complexity, in particular, to calculate approximate distortion based on prediction or prediction residual signals rather than reconstructed signals. A mixture of these two methods can also be used, for example by using approximate distortion only for some possible coding options and full distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete assessment of both the coding cost and the associated distortion.

[0316] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). For example, the device can be implemented with appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0317] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations mean that a particular feature, structure, characteristic, etc. described in conjunction with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations in various places in this application do not necessarily all refer to the same embodiment.

[0318] Additionally, the present application may refer to “determining” various information. Determining the information may include, for example, one or more of: estimating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0319] In addition, the present application may involve "accessing" various information. Accessing the information may include, for example, one or more of: receiving the information, retrieving the information (e.g., retrieving the information from a memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0320] Additionally, the present application may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving the information may include, for example, one or more of: accessing the information or retrieving the information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is generally involved in one way or another.

[0321] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to multiple items listed, as will be apparent to one of ordinary skill in this and related arts.

[0322] In addition, as used herein, the word "signal" especially refers to a corresponding decoder indicating something. For example, in some embodiments, the encoder notifies a specific one of a plurality of decoding modes or marks with a signal. Thus, in one embodiment, the same parameters are used on the encoder side and the decoder side. Therefore, for example, the encoder can send (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. On the contrary, if the decoder already has the specific parameter and other parameters, signaling can be used without sending (implicitly signaling) to simply allow the decoder to know and select the specific parameter. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to send information to the corresponding decoder with a signal. Although the foregoing relates to the verb form of the word "signal", the word "signal" can also be used as a noun in this article.

[0323] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0324] We have described a number of embodiments. Features of these embodiments may be provided individually or in any combination. In addition, across various claim categories and types, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination:

[0325] ●Modify the decoding mode processing applied in the decoder and / or encoder.

[0326] ● Several advanced coding mode prediction methods are enabled in the decoder and / or encoder.

[0327] • Inserting syntax elements in the signaling that enable the decoder to identify the coding mode prediction method to be used.

[0328] Based on these syntax elements, the coding mode prediction method is selected to be applied to the decoder. The coding mode prediction method is applied at the decoder for deriving the mode.

[0329] The parameters are derived using the prediction process described above and removal using a lookup table.

[0330] The parameters are derived using the prediction process described above and modification using a lookup table.

[0331] Linear prediction was used to derive prediction parameters.

[0332] The residual is adapted at the encoder according to any of the embodiments discussed.

[0333] A bitstream or signal comprising one or more of the described syntax elements or variations thereof. A bitstream or signal comprising syntax conveying information generated according to any of the described embodiments.

[0334] According to any of the embodiments described, creating and / or sending and / or receiving and / or decoding.

[0335] A method, process, apparatus, medium storing instructions, medium storing data, or signal according to any described embodiments.

[0336] Syntax elements are inserted in the signaling that enable the decoder to determine the coding mode in a manner corresponding to the manner used by the encoder.

[0337] Creating and / or sending and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof.

[0338] A TV, set-top box, cell phone, tablet computer, or other electronic device that performs decoding mode determination according to any of the described embodiments.

[0339] A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs a decoding mode determination according to any of the described embodiments and displays (e.g., using a monitor, screen, or other type of display) the resulting image.

[0340] A TV, set-top box, cellular phone, tablet, or other electronic device that selects, band limits, tunes (e.g., using a tuner) a channel to receive a signal including an encoded image, and performs a decoding mode determination according to any of the described embodiments.

[0341] A TV, set-top box, cellular phone, tablet, or other electronic device receives over the air (eg, using an antenna) a signal including the encoded image and performs a decoding mode determination.

Claims

1. A method comprising: determining a CCLM intra prediction of samples in a current block based on at least one of neighboring samples in the current block and based on a parametric model calculated from the neighboring samples of the current block, wherein four reference samples are used for the prediction, including reference samples from a top neighboring row of reference samples and from a left neighboring row of reference samples, wherein two samples from the top neighboring row of reference samples and two samples from the left neighboring row of reference samples are used when available, otherwise when only the top neighboring row is available, samples in the top neighboring row are used, otherwise when only the left neighboring row is available, samples in the left neighboring row are used, wherein the limited range of indices of the single lookup table enables division with adaptable precision to derive parameters of the parameter model based on brightness differences; and Based on the prediction, the samples in the current block are encoded.

2. The method of claim 1, wherein the parametric model is derived from a linear model. 3 . The method of claim 1 , wherein the parameters of the parametric model are derived from at least two samples with a spatial distance constraint of neighboring samples.

4. The method according to claim 1, wherein the parameters of the parametric model are derived from at least three adjacent samples, wherein the three samples are respectively located at the rightmost side of the top row of adjacent samples above the block, the bottom of the left adjacent sample column, and the intersection of the top reference row and the left reference column.

5. The method of claim 1, wherein if the linear parameter derivation is well defined, a linear model based prediction is used, otherwise an alternative model is used. The method of claim 1 , wherein the derivation of parameters of the parametric model comprises calibration parameters.

7. The method of claim 1, wherein a cross-component linear model is enabled for predicting chrominance components of inter-coded blocks.

8. An apparatus comprising: A processor configured to: determining a CCLM intra prediction of samples in a current block based on at least one of neighboring samples in the current block and based on a parametric model calculated from the neighboring samples of the current block, wherein four reference samples are used for the prediction, including reference samples from a top neighboring row of reference samples and from a left neighboring row of reference samples, wherein two samples from the top neighboring row of reference samples and two samples from the left neighboring row of reference samples are used when available, otherwise when only the top neighboring row is available, samples in the top neighboring row are used, otherwise when only the left neighboring row is available, samples in the left neighboring row are used, wherein the limited range of indices of the single lookup table enables division with adaptable precision to derive parameters of the parameter model based on brightness differences; and Based on the prediction, the samples in the current block are encoded.

9. A method comprising: determining a CCLM intra prediction of samples in a current block based on at least one of neighboring samples in the current block and based on a parametric model calculated from the neighboring samples of the current block, wherein four reference samples are used for the prediction, including reference samples from a top neighboring row of reference samples and from a left neighboring row of reference samples, wherein two samples from the top neighboring row of reference samples and two samples from the left neighboring row of reference samples are used when available, otherwise when only the top neighboring row is available, samples in the top neighboring row are used, otherwise when only the left neighboring row is available, samples in the left neighboring row are used, wherein the limited range of indices of the single lookup table enables division with adaptable precision to derive parameters of the parameter model based on brightness differences; and Based on the prediction, the samples in the current block are decoded.

10. The method of claim 9, wherein the parametric model is derived from a linear model.

11. The method of claim 9, wherein the parameters of the parametric model are derived from at least two samples with a spatial distance constraint of neighboring samples.

12. The method of claim 9, wherein the parameters of the parametric model are derived from at least three adjacent samples, wherein the three samples are located at the rightmost side of a top row of adjacent samples above the block, at the bottom of a left adjacent sample column, and at the intersection of a top reference row and a left reference column, respectively.

13. The method according to claim 9, wherein: If the linear parameter derivation is well defined, the linear model based prediction is used, otherwise the alternative model is used.

14. The method of claim 9, wherein the derivation of parameters of the parametric model comprises calibration parameters.

15. The method of claim 9, wherein a cross-component linear model is enabled for predicting chrominance components of inter-coded blocks.

16. An apparatus comprising: A processor configured to: determining a CCLM intra prediction of samples in a current block based on at least one of neighboring samples in the current block and based on a parametric model calculated from the neighboring samples of the current block, wherein four reference samples are used for the prediction, including reference samples from a top neighboring row of reference samples and from a left neighboring row of reference samples, wherein two samples from the top neighboring row of reference samples and two samples from the left neighboring row of reference samples are used when available, otherwise when only the top neighboring row is available, samples in the top neighboring row are used, otherwise when only the left neighboring row is available, samples in the left neighboring row are used, wherein the limited range of indices of the single lookup table enables division with adaptable precision to derive parameters of the parameter model based on brightness differences; and Based on the prediction, the samples in the current block are decoded.

17. A device comprising: The device according to claim 16; as well as At least one of: (i) an antenna configured to receive a signal, the signal comprising a video block, (ii) a frequency band limiter configured to limit the received signal to a frequency band comprising the video block, and (iii) a display configured to display an output representing the video block.

18. A non-transitory computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 or 9.

19. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to perform the method according to any one of claims 1 or 9.

Images