Quantization constrained correction with individually enabled bound offsets

Abstract

Methods and apparatus are provided for performing the quantization constrained (QC) correction of a given transform coefficient c ƒ associated to a quantization level c q when the minimum and maximum bound offsets are individually enabled or disabled. In one embodiment, signaling methods are provided to save signaling cost of bound offset parameters if the corresponding bound offset is disabled. Another embodiment provides performing QC correction of a transform coefficient using individually enabled or disabled bound offsets. In another embodiment, computing both the minimum and the maximum bound offsets is performed using a single bound offset signaled in the bitstream.

Description

[0001] QUANTIZATION CONSTRAINED CORRECTION WITH INDIVIDUALLY ENABLED BOUND OFFSETS

[0002] CROSS REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of European Application No 24306978.8, filed on November 27, 2024 which is incorporated herein by reference in its entirety.

[0004] TECHNICAL FIELD

[0005] At least one of the present embodiments generally relates to a method or an apparatus for bound offset signaling with adaptive bitdepth for quantization constrained correction.

[0006] BACKGROUND

[0007] The general aspects described herein relate to approaches for video compression and decompression, and particularly to for bound offset signaling with adaptive bitdepth for quantization constrained correction.

[0008] SUMMARY

[0009] At least one of the present embodiments generally relates to a method or an apparatus for compressing or decompressing a set of images or videos using bound offset signaling with adaptive bitdepth for quantization constrained correction.

[0010] According to a first aspect, there is provided a method. The method comprises steps for parsing a bitstream for enabled bound offsets; selecting a correction function based on at least one enabled bound offset; applying the correction function to a prediction residual transform coefficient; and, decoding video data using the prediction residual transform coefficient after correction.

[0011] According to a second aspect, there is provided another method. The method comprises steps for determining a correction function to be used for quantization constrained correction of a video block based on at least one enabled bound offset; and, signaling in a bitstream at least one bound offset corresponding to the quantization constrained correction for encoding the video block.

[0012] According to another aspect, there is provided an apparatus. The apparatus comprises a processor and a memory. The processor can be configured to operate on video signals and data according to the aforementioned methods. According to another aspect, there is provided an apparatus. The apparatus comprises a processor and a memory. The processor can be configured to encode a video signal or decode video data by executing any of the aforementioned methods.

[0013] According to another general aspect of at least one embodiment, there is provided a device comprising an apparatus according to any of the decoding embodiments; and at least one of (i) an antenna configured to receive a signal, the signal including the video signal, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the video block, or (iii) a display configured to display an output representative of the video block.

[0014] According to another general aspect of at least one embodiment, there is provided a non-transitory computer readable medium containing data content generated according to any of the described encoding embodiments or variants.

[0015] According to another general aspect of at least one embodiment, there is provided a signal comprising video data generated according to any of the described encoding embodiments or variants.

[0016] According to another general aspect of at least one embodiment, video data or a bitstream is formatted to include data content generated according to any of the described encoding embodiments or variants.

[0017] According to another general aspect of at least one embodiment, there is provided a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the described decoding embodiments or variants.

[0018] These and other aspects, features and advantages of the general aspects will become apparent from the following detailed description of exemplary embodiments, which is to be read in connection with the accompanying drawings.

[0019] BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 illustrates quantization and dequantization of a transform coefficient c. Figure 2 illustrates quantization bounds depending on the sign of the quantization level.

[0021] Figure 3 illustrates reconstruction with filtering.

[0022] Figure 4 illustrates quantization constrained correction.

[0023] Figure 5 illustrates projection function onto the quantization constraint.

[0024] Figure 6 illustrates an example of smooth correction function. Figure 7 illustrates correction of a transform coefficient Cf associated with a quantization level cq.

[0025] Figure 8 illustrates correction of a transform coefficient Cf associated with a quantization level cqusing the symmetrized coefficient cfsymassociated with abs(cq).

[0026] Figure 9 illustrates QC correction with adaptive correction function based on the availability of bound offsets and the quantization level.

[0027] Figure 10 illustrates correction functions with clipping.

[0028] Figure 11 illustrates example of correction functions with smooth clipping. Figure 12 illustrates bounds that depend on the minimum bound offset.

[0029] Figure 13 illustrates bounds that depend on the maximum bound offset.

[0030] Figure 14 illustrates determination of enabling flags for inferior and superior correction bounds.

[0031] Figure 15 illustrates determination of enabling flags for inferior and superior correction bounds when QC correction is applied to a symmetrized transform coefficient csym.

[0032] Figure 16 illustrates selection of inferior or superior correction bound by disabling the non-selected correction bound.

[0033] Figure 17 illustrates decoding with signaling.

[0034] Figure 18 illustrates one embodiment of a first method under the described aspects.

[0035] Figure 19 illustrates one embodiment of a second method under the described aspects

[0036] Figure 20 illustrates one embodiment of an apparatus under the described aspects.

[0037] Figure 21 illustrates a standard, generic, video compression scheme.

[0038] Figure 22 illustrates a standard, generic, video decompression scheme.

[0039] Figure 23 illustrates a processor-based system for encoding / decoding under the general described aspects.

[0040] DETAILED DESCRIPTION

[0041] The embodiments described here are in the field of video compression, specifically focusing on in-loop or out-of-loop filters and is also related to quantization. In-loop filters are used to enhance reconstructed images before storing them in the reference buffer. In-loop filters form a whole family. Among them, deblocking filters (DBF) aim at reducing blocking artifacts occurring along block boundaries. Deblocking filters are usually designed to improve subjective quality, that is, the noticeability by the human visual system of such coding errors. In usual video coding standards such as HEVC and WC, deblocking filters are predetermined based on coding information (such as prediction modes, motion vectors, transform coefficients) and on local variations across block boundaries. On the other hand, adaptive loop filters (ALF) are learnt at encoder side in order to minimize the mean squared error with respect to source images, then the learned filter weights are written to the bitstream. Adaptive loop filters are applied at CTU-level, while deblocking filters are run along block borders.

[0042] The present embodiments propose computing the QC correction when the minimum and maximum bound offsets used in the correction process may be individually enabled or disabled.

[0043] To achieve high compression efficiency, image and video coding schemes usually employ block-based prediction, including motion vector prediction, and transform to leverage spatial and temporal redundancy in the video content. Generally, intra or inter prediction is used to exploit the intra or inter frame correlation, then the differences between the original block and the predicted block, often denoted as prediction errors or prediction residuals, are transformed, quantized, and entropy coded. To reconstruct the video, the compressed data are decoded by inverse processes corresponding to the entropy coding, quantization, transform, and prediction.

[0044] In the HEVC (High Efficiency Video Coding) video compression standard, motion compensated temporal prediction is employed to exploit the redundancy that exists between successive pictures of a video.

[0045] The embodiments described herein are in the context of video compression. The embodiments specifically focus on in-loop or out-of-loop filters and is also related to quantization.

[0046] In-loop filters are used to enhance reconstructed images before storing them in the reference buffer. In-loop filters form a whole family. Among them, deblocking filters (DBF) aim at reducing blocking artifacts occurring along block boundaries. Deblocking filters are usually designed to improve subjective quality, that is, the noticeability by the human visual system of such coding errors. In usual video coding standards such as HEVC and WC, deblocking filters are predetermined based on coding information (such as prediction modes, motion vectors, transform coefficients) and on local variations across block boundaries. On the other hand, adaptive loop filters (ALF) are learnt at encoder side in order to minimize the mean squared error with respect to source images, then the learned filter weights are written to the bitstream. Adaptive loop filters are applied at CTU-level, while deblocking filters are applied along block borders.

[0047] In another work, the quantization-constrained (QC) correction filter adds a complementary step to existing filters with the goal to prevent these filters from losing information carried by the quantized transform coefficients signaled in the bitstream, while still preserving the filtering effect.

[0048] Scalar quantization in High Efficiency Video Coding (HEVC)

[0049] HEVC and WC standards specify conventional independent scalar quantization with Uniform Reconstruction Quantizers (URQs). The reconstructed transform coefficients crof URQs are completely denoted by integer multiples of a quantization step size A (a.k.a. quantization step) which depends on a quantization parameter (QP). This integer specifies the associated transform coefficient level, which is transmitted in the bitstream as quantization levels cq. On the decoder side, given a quantization level cqand a quantization step size A, a reconstructed transform coefficient crmay be generated as follows:

[0050] cr= Cq× Δ (1) On the encoder side, given a coefficient c (a.k.a a transform coefficient or more precisely a prediction residual transform coefficient) and a quantization step size A, a quantization level cqmay be generated as follows:

[0051] c

[0052]

[0053] q = Q(c) = ⌊abs(c) / Δ + a⌋ × sgn(c), (2)

[0054] where [.] is the floor operation (i.e. rounding to the lower integer), abs is the

[0055] absolute value, sgn is the sign function (i.e. returns 1 if c>0, -1 if c<0 and 0 if c=0), and a is a parameter such that 0<a<1. With this quantization method, every quantization level cqis associated to quantization bounds infQ(cq) and supQ(cq) such that an original coefficient c between infQcq) and supQcq) is quantized to cq. Hence assuming this quantization method was used by the encoder

[0056] and given a quantization level cq, the original coefficient c associated with cqis known to be between infQ(

[0057]

[0058] cq) and supQ(cq~), as illustrated in Figure 1. In the case of uniform quantization with quantization step size A and parameter a, the inferior and superior quantization bounds are defined as follows:

[0059] , A ((ca ~ °max)Xif Ca< 0 inf (c} = J ^maxJq, (3) l(cQ+ Omin) X A if Cq > 0 r \ f (TQ Omin') X A if Cq0 sup0(ca) = \; ', (4)

[0060]

[0061] l(CQ + °max) if Cq> 0

[0062] with minimum offset ominand maximum offset omaxdefined as omin= -a and omax= 1 - a. This parameterization of the quantization bounds, and reconstruction values is illustrated in Figure 2 for the three cases where cq> 0, cq= 0 or cq< 0. It can be noted that from the definition in Equations 3 and 4, a symmetry property of the quantization bounds is always satisfied: i.e., infQ(

[0063]

[0064] cq) = -supQ-cq) a

[0065]

[0066] nd supQ(cq) = -infQ(-Cq}..

[0067] The encoder may use alternative strategies to select the quantization levels cq. For example, rate distortion optimized quantization (RDOQ) optimizes the quantization levels accounting not only for the distortion (i.e., MSE), but also for the rate. In this case, it is no longer guaranteed that the original coefficients c are between the boundsinfQcq) and supQcq).

[0068] In the quantization constrained correction, c is assumed to be bounded by i

[0069]

[0070] n / gCcg) and supQ(cq), where infQ(cq) and supQ(cq) can be determined on the decoder side knowing the quantization level cq, the scalar quantizer Q (a.k.a. scalar quantization operation) and its quantization parameter (QP). If the encoder uses quantization strategies that may break this assumption, such as RDOQ, the method remains applicable.

[0071] Quantization constrained correction filter

[0072] In the quantization-constrained correction filter, a quantization constraint is defined as the following equation: Q(. Cx) = Cq> (5) where Q is a scalar quantization operation (a.k.a. scalar quantizer), cqis the quantization level of a prediction residual transform coefficient c, and cxis a reconstructed value of the corresponding transform coefficient c. When no in-loop or out-of-loop filter is applied to a reconstructed picture, each coefficient cxis obtained as cx= cr. Since cr=

[0073]

[0074] its value is between the quantization bounds

[0075]

[0076] inf(cq)

[0077]

[0078] and sup(cq) (see Figure 1). Therefore, in this case, the quantization constraint Q(cr) = cqis satisfied. Assuming that each quantization level was determined by the encoder as cq= Q(c), where c is the original (unquantized) coefficient, then Q(cr) = Q(c), which is a necessary condition to reconstruct the original coefficient c without error. More generally, the absolute error |cr- c| between a reconstructed coefficient and the corresponding original coefficient is such that 0 < |

[0079]

[0080] cr- c| < supQ[cq) - infQ(cq).

[0081] However, in the case of filtering the reconstructed picture (e.g., with DBF, SAO, ALF, etc.), each coefficient cxis defined as cx= cf, where cfis the corresponding prediction residual transform coefficient of said reconstructed and filtered picture (cfmay not be computed explicitly in video coding). In this case, cxmight not satisfy the above quantization constraint anymore, resulting in a loss of information. For example, the DBF filter improves the subjective quality by removing block discontinuities, but it may also remove some details in the filtered area at the borders of the blocks. As a result, for some coefficients, we may have cf> supQ(cq~) or cf< infQ(

[0082]

[0083] cq~), and thus Q(

[0084]

[0085] cy) #= cq. Said otherwise, the quantization constraint is not necessarily satisfied. This results in a sub-optimal MSE of the filtered picture since the corresponding original coefficient c is expected to be in the range [ infQcq'),supQcq')]. Thus, cfis necessarily different from the original coefficient c (i.e. |c -c| > 0), and the error |cy - c| is not bounded by supQ(cq) - infQ(cq). Although the ALF filter is designed to reduce the MSE between the original and the reconstructed and filtered picture, it does not guarantee that the quantization constraint Q(

[0086]

[0087] cf) = cqis satisfied for every coefficient cf, hence potentially leading to sub-optimal MSE given that the quantization levels cqare known to both the encoder and the decoder.

[0088] The QC correction filter solves this problem by enforcing the quantization constraint on the prediction residual transform coefficients of the filtered picture. The QC correction is applied to an image block Xfobtained by filtering a block Xrpreviously obtained using conventional reconstruction steps in video coding, as shown in Figure 3. These steps include:

[0089] 1) Scalar dequantization of quantization levels cqto produce reconstructed coefficients cr=

[0090]

[0091] where the dequantization function Q-1is parameterized by a quantization parameter QP and where each quantization level cqis assigned to a position within the block.

[0092] 2) Inverse transform T-1applied to the coefficients cr, in order to obtain a prediction residual block.

[0093] 3) Reconstruction of the block Xras the sum between the prediction residual block and a prediction block pred.

[0094] After filtering Xrto produce Xf, the QC correction described in Figure 4 first converts Xfback to prediction residuals and compute its transform coefficients cfin order to correct the values of these coefficients using a correction step that ensures that the quantization constraint (

[0095]

[0096] ?(<^) = cqis satisfied for the corrected transform coefficient cf. The corrected transform coefficients are then converted back to obtain the corrected image block XQCFusing the inverse transform T-1and the summation with the prediction pred. The steps of the QC correction are further detailed hereafter: Prediction subtraction:

[0097] Xfis converted back to prediction residuals. To this aim, the prediction block pred is subtracted from the filtered image block Xfin order to obtain a filtered prediction residual block. The prediction pred is equal to the prediction used in the reconstruction stage. Therefore, the prediction block pred does not have to be computed again using the codec’s prediction mechanisms. Instead, the prediction block may be stored in a buffer during the reconstruction and reused for the QC correction.

[0098] Forward Transform:

[0099] The filtered prediction residual block is then converted back to the transform domain using the forward transform T. In the case where several types of transforms or inverse transforms may be applied (e.g., DCT-II, DST-7, DCT-8), the QC correction uses the forward transform T of the same type as the inverse transform T-1used during the reconstruction stage. For example, if a block X is reconstructed using the inverse of the DCT-II at S802, its corresponding block (i.e. the block of coefficients cfobtained from X) in the QC correction is transformed with forward DCT-II. The forward transform thus produces a new block of prediction residual transform coefficients cf, named more simply transform coefficients cf, each located at a given position within the block. Each transform coefficient cfcan thus be associated with the quantization level cqused during the reconstruction stage, and which is located at the same position within the block.

[0100] Projection to the quantization constraint:

[0101] Each transform coefficient cfof the block of prediction residual transform coefficients is corrected to obtain a block of corrected prediction residual transform coefficients cf, named more simply corrected transform coefficient cf. The corrected version is computed as cf= proj^=c^Cf), where proj[QQ=Cqiis a correction function

[0102]

[0103] (also called projection function onto the quantization constraint). This correction function ensures that the constraint (

[0104]

[0105] ?(<^) = cqis satisfied, where cqis the quantization level associated with cf, and where Q is the scalar quantizer associated to the scalar dequantizer Q-1used in the reconstruction stage. The correction function can be defined as follows:

[0106] p

[0107]

[0108] roj\Qu=Cq](cf) = min(max (cf, infQ(cq')),supQ(cq')) (6) It is illustrated in Figure 5. With this definition, each corrected transform coefficient cfis always between the quantization bounds in(3(c(?) and SUPQ^C^ of the quantization level cqfor the quantizer Q. Hence cfsatisfies (

[0109]

[0110] ?(<^) = cq. However, if cfis already within the quantization bounds, no correction is applied (i.e. cf= cf), which preserves the filtering effect.

[0111] The correction function in Equation 6 is defined to enforce the quantization constraint while also minimizing the squared error cf-

[0112]

[0113] between the corrected and the uncorrected transform coefficient. However, other functions may be used. For example, a more general clipping function can be defined as cf= min(max (cf> btnf) bsup) where the correction bounds binf, bsupmay be computed similarly t

[0114]

[0115] o in / (3(c(?) and SUPQ^C^ using Equations 3 and 4 respectively, but where the values of the parameters ominand omaxare set independently of the parameter a. For example, ominand omaxmay be pre-determined parameters known to both the encoder and decoder. Note that using ominand omaxsuch that -a < omin< omax< 1 - a ensures that i

[0116]

[0117] nfQ{cq^ < binf< bsup< supQ(cq'). Hence, assuming that cqwas computed with the quantization function in Equation 2 by the encoder, using such values of ominand omaxstill ensures that the quantization constraint (

[0118]

[0119] ?(<^) = cqis satisfied (since infQcq^ < binf< cf< bsup< supQ(cq)). However, if other quantization methods are used by the encoder (e.g., RDOQ), the original coefficient may not be between the bounds infQ(cq) and supQ(cq) defined in Equations 3 and 4. Hence, in such cases, it may be advantageous to compute the correction bounds binfand bsupusing other values of ominand omaxthat do not necessarily satisfy the constraint —a < omin< omax< 1 — a.

[0120] In another example, a smooth version of the correction function proj[Q^=Cqimay be used such as:

[0121] c̃f= s · tanh

[0122]

[0123] with a scale s =supQ(Cq) inWc^anqatranslation t =supQ^+inf^c<i This smooth 2 2

[0124] function is illustrated by Figure 6. Similarly to the previous example, the correction bounds binfand bsupmay be used instead of the inferior and superior bounds in(3(c(?) and supQ(cq) respectively for the computation of the scale and translation parameters s and t.

[0125] Inverse Transform and addition with prediction:

[0126] After enforcing the quantization-constraint in the residual transform domain, the corrected transform coefficients cfare converted back to the image domain to obtain a final corrected image block XQCF. This is performed by first applying the inverse transform again to obtain a block of corrected prediction residuals. The same inverse transform T-1is used as in the reconstruction stage. Finally, the prediction block pred is added back to the block of corrected prediction residuals to obtain a corrected image block XQCFwhich satisfies the quantization constraint. Integer bound offset parameterization for integer bound computation.

[0127] For practical computation of the bounds using the bound offsets ominand omax, an integer approximation of Equations 3 and 4 may be applied, based on an integer parameterization of the bound offsets of the form omin=

[0128]

[0129] and omax= where O0and Soare respectively the integer bound offset parameter and the integer shift parameter of the minimum bound offset, and Cf and S are respectively the integer bound offset parameter and the integer shift parameter of the maximum bound offset. Each integer bound offset parameter Oi (with i = 0 or i = 1) is represented by Ntbits and a starting value start i.e., Cf can take 2Niinteger values in the range [starti, starti + 2Ni- 1], For example, the values of Stand starttmay be fixed as S, = Nt- 1 and starti = ~Si sothat the corresponding bound offset has 2Nipossible values uniformly spaced in the range [— 1,1[. Alternatively, Stand starti may be fixed as Si = Nt- 1 and starti = l - 2Siso that the corresponding bound offset has 2Nipossible values uniformly spaced in the range ]-l,l].

[0130] In existing video coding standards such as HEVC and WC, the quantization step A is also represented with integer parameters scale&and shifty as A = scale& / 2shi t&in order to perform dequantization with integer-based computations.

[0131] Based on these integer parameterizations, a correction bound b(cq~) associated to a quantization level cqmay be computed using only operations on integers: additions, multiplications and bitshifts (i.e. leftshift «, or rightshift »). For example, using an integer bound offset parameter 0, its corresponding shift S, and the quantization step parameters scaleAand shifty, a correction bound

[0132]

[0133] b(cq) may be computed as:

[0134] ((num + add) » shift if shift > 0

[0135] (num « —shift if shift < 0

[0136]

[0137] where num = ([cq« S) + o) * scaleA, shift = shifty + S, and add is a value

[0138]

[0139] determined depending on the rounding strategy. For example, using add = 1 « (shift - 1), Equation 8 approximates the result of the division

[0140]

[0141] with the closest integer when shift > 0.

[0142] For each coefficient cfassociated to a quantization level cq, its inferior and superior correction bounds binfand bsupmay thus be computed with this method using values of O and S selected depending on the sign of cq. For example, considering the integer parameterization of omin(with O0and So) and omax(with O and Si), the inferior bound in Equation 3 may be approximated by an inferior correction bound binfcomputed with Equation 8 using S = S and 0 = -O if cq< 0, or S = Soand 0 = O0if cq> 0. Similarly, the superior bound in Equation 4 may be approximated by a superior correction bound bsupcomputed with Equation 8 using S = Soand O = -Ooif cq< 0 or S = Si and O = O if cq> 0.

[0143] Using both correction bounds binfand bsup, a correction function is finally applied to the transform coefficient cfto obtain the corrected transform coefficient cf.

[0144] Correction of symmetrized coefficient

[0145] In a variant of the QC correction detailed in Figure 8, the correction of a coefficient cfassociated with a quantization level cqis performed indirectly by exploiting the symmetry property of the quantization bounds illustrated in Figure 2 and by applying the correction to a symmetrized coefficient csymassociated with the absolute value

[0146]

[0147] abs(cq) of the quantization level. The symmetrized coefficient cfsymis equal to cfif cq> 0, or equal to -cfotherwise. The correction bounds of the symmetrized coefficient are then computed, and the correction function is applied to csymto obtain a corrected symmetrized coefficient csym. Finally, the corrected coefficient cfis equal to csymif cq>0 or equal to -csymotherwise. In this variant, the inferior bound binfof c^ymmay be computed with Equation 8 using S = S±and O = -O if cq= 0 or S = Soand O = O0otherwise, and by replacing the quantization level cqby its absolute value

[0148]

[0149] abs(cq~) in the numerator (i.e., num = ([abs(cq~) « S) + o) *

[0150]

[0151] scale / ). The superior bound may be computed similarly using S = S and O = O

[0152] Signaling of bound offsets

[0153] In order to adjust the QC correction for different regions to be corrected (e.g. slice, picture,...), the bound offsets ominand omaxcan be signaled in the bitstream thanks to their integer parameters (i.e., O0and Sofor ominand O and S for omax).

[0154] For each integer bound offset ( / (represented with Ntbits), its value is mapped to an index bound_offset_index which is signaled in the bitstream. The index bound_offset_index may be a Nrbit index. In this case, all the 2Nivalues of Oi in the range [starts startt+ 2Ni- 1] are allowed, and ( / is obtained from the index as ( / = bound_off set Jndex + starti. Alternatively, the index bound_offset_index may be a Mrbit index with M^Nt. In this case only a subset ofMlvalues of Otare allowed, and Oi is obtained from the index using a mapping function, e.g., a Lookup Table (LUT) or a linear mapping.

[0155] The values of the integer shift parameters Soand S±may be signaled in the bitstream or may be fixed values known to both the encoder and decoder.

[0156] Signaling for enabling or disabling QC correction

[0157] In the QC correction, a flag use_qc may be signaled per region (e.g., CTU, slice, picture,...) to indicate if the QC correction must be applied to the corresponding region. Alternatively, the flag may be inferred and not signaled explicitly. For example, the QC correction may be performed only if the QP is above a threshold.

[0158] Problem to be solved

[0159] In the quantization constrained correction both bound offsets ominand omax(or the corresponding integer parameters O0, So, O1and Si) are generally needed to compute the inferior and superior correction bounds binfand bsupwhich are used to apply a correction to the transform coefficient cf. However, in some cases, it may be preferable to perform QC correction with a single bound offset e.g., to save signaling cost of one of the bound offsets.

[0160] In the existing QC correction, it is possible to use only a single correction bound (i.e., either binfor bsup) e.g., by applying a one-sided correction function such as cf= max(c binfif only binfis used, or

[0161]

[0162] cf~ = mm(cf,bsup) if only bsupis used. However, given a single bound offset (i.e., either ominor omax) or its corresponding integer parameters, it is not known in advance which correction bounds can be obtained, due to the symmetry property of quantization bounds illustrated in Figure 2. The type of correction bound that can be computed (i.e., inferior or superior) depends on the quantization level cqassociated to each transform coefficient to be corrected. Depending on which bound offset is enabled and on whether cq< 0, cq= 0, or cq> 0, it may be possible to compute only the superior correction bound, only the inferior correction bound, both correction bounds or none of them. Hence, the correction function should also depend on the transform coefficient to be corrected.

[0163] The present embodiments describe how to perform the QC correction of a given transform coefficient cfassociated to a quantization level cqwhen the minimum and maximum bound offsets may be individually enabled or disabled. Corresponding signaling methods are also described to save signaling cost of bound offset parameters if the corresponding bound offset is disabled.

[0164] The embodiments describe:

[0165] 1) A general embodiment for performing QC correction of a transform coefficient using individually enabled or disabled bound offsets.

[0166] 2) A method for selecting adaptively the correction function based on either a single correction bound (inferior or superior) or both the inferior and the superior correction bounds.

[0167] 3) A method for enabling or disabling each correction bound based on the enabled or disabled bound offsets.

[0168] 4) Adaptations of the signaling to take advantage of the QC correction with individually enabled or disabled bound offsets.

[0169] 5) Another embodiment for computing both the minimum and the maximum bound offsets using a single bound offset signaled in the bitstream

[0170] QC Correction from individually enabled or disabled bound offsets

[0171] In an embodiment described in Figure 9, a coefficient cfassociated with a quantization level cqis corrected based on bound offset parameters (e.g. minimum bound offset ominor its integer parameter O0and So, maximum bound offset omaxor its integer parameters O1and Si) and flags min_enabled and max_enabled indicating if respectively the minimum and maximum bound offsets are enabled. For example, a flag value of 1 may indicate that the bound offset is enabled, and a flag value of 0 may indicate that the bound offset is disabled. Based on the min_enabled and max_enabled flags, and the quantization level cq, it is first determined if the inferior and the superior correction bounds are enabled (e.g., set values of flags inf_enabled and sup_enabled). If a correction bound is enabled, it is then computed using cqand the parameters of the enabled bound offsets, e.g., with Equation 3 or 4 (directly based on bound offset ominor omax) or with an integer based computation as in Figure 7 or Figure 8 (using integer bound offset parameters O0and Soor O and Si). If any of the inferior or superior correction bounds is enabled, the transform coefficient cfis then corrected using a correction function selected depending on which correction bounds (i.e., inferior, superior or both) are enabled. Otherwise (e.g., inf_enabled=0 and sup_enabled=0), the correction of the coefficient is skipped (i.e., cf= cf).

[0172] In a variant, at the beginning of the process, the inferior correction bound is initialized to a large negative value (e.g. -2A16), the superior correction bound is initialized to a large positive value (e.g. 2A16), and the correction function remains the generic one where both the inferior and superior bounds are checked. In this variant, the correction function does not depend on the inf_enabled and sup_enabled flags.

[0173] Selection of correction function

[0174] If only the inferior correction bound binfis enabled (e.g., inf_enabled=1 and sup_enabled=0), an inferior correction function is selected. The inferior correction function may be defined such that cf> binf. For example, this can be obtained by clipping values of cfthat are below binfas c = max(cy, binf).

[0175] If only the superior correction bound bsupis enabled (e.g., inf_enabled=0 and sup_enabled=1), a superior correction function is selected. The superior correction function may be defined such that - < bsup. For example, this can be obtained by clipping values of cfthat are above

[0176]

[0177] bsupas = min(cy, Z?sup).

[0178] If both the inferior and the superior correction bounds binfand bsupare enabled (e.g., inf_enabled=1 and sup_enabled=1), an inferior and superior correction function is selected. The inferior and superior correction function may be defined such that binf< cf< bsup. For example, this can be obtained by clipping values of cfthat are below binfor above bsupas cf= min man cf,binf),bsup)'.

[0179] These clipping correction functions are illustrated in Figure 10.

[0180] Note that other functions may be used. For example, a smooth clipping function as in Figure 6 may be used for the inferior and superior correction function. Similarly, a smooth inferior correction function and a smooth superior correction function may be used, as illustrated in Figure 11.

[0181] Enabling or disabling of correction bound(s)

[0182] Enabling of correction bounds when corresponding bound offset is enabled

[0183] Due to the symmetry property of quantization bounds illustrated in Figure 2, the type of correction bounds (i.e., inferior or superior) that can be computed from a single bound offset depends on the quantization level cqassociated to the transform coefficient cfto be corrected.

[0184] For each case cq> 0, cq= 0, or cq< 0, the types of bound that depend on the minimum bound offset are illustrated in Figure 12. The bounds associated to the absolute value of the quantization level abs(cq) (needed for the case where QC correction is applied to a symmetrized coefficient) are also shown in gray. Similarly, the type of bounds that depend on the maximum offset are illustrated in Figure 13.

[0185] As a result, if a correction bound depends on a disabled bound offset, this correction bound should be also disabled. For example, this may be performed by determining the inf_enabled and sup_enabled flags of the correction bounds based on the quantization level cqand the enabling flags min_enabled and max_enabled of the bound offsets (1st step of Figure 9) as described in Figure 14.

[0186] If cq= 0, both the inferior and superior correction bounds are disabled if the maximum offset is disabled (e.g., inf_enabled=sup_enabled=max_enabled).

[0187] If cq> 0, the inferior correction bound is disabled if the minimum offset is disabled (e.g., inf_enabled=min_enabled). Similarly, the superior correction bound is disabled if the maximum offset is disabled (e.g., sup_enabled=max_enabled).

[0188] If cq< 0, the inferior correction bound is disabled if the maximum offset is disabled (e.g., inf_enabled=max_enabled). Similarly, the superior correction bound is disabled if the minimum offset is disabled (e.g., sup_enabled=min_enabled).

[0189] In the variant of the QC correction in Figure 8, the correction function is applied to a symmetrized transform coefficient csymand the final corrected coefficient cfis obtained from the corrected symmetrized coefficient csym. In this case, the correction bounds binfand bsupused for correcting the symmetrized coefficient are computed based on the absolute value of the quantization level abs(

[0190]

[0191] cq) (instead of cq) as shown in gray in Figure 12 and Figure 13. Hence, in this variant, the enabling flags inf_enabled and sup_enabled of the correction may be determined as described in Figure 15.

[0192] If cq= 0, both the inferior and superior correction bounds are disabled if the maximum offset is disabled (e.g., inf_enabled=sup_enabled=max_enabled).

[0193] Otherwise (cq≠ 0), the inferior correction bound is disabled if the minimum offset is disabled (e.g, inf_enabled=min_enabled). Similarly, the superior correction bound is disabled if the maximum offset is disabled (e.g., sup_enabled=max_enabled).

[0194] Additional criteria for disabling correction bounds

[0195] In the examples of Figure 14 and Figure 15, a correction bound is only disabled if its corresponding bound offset is disabled. However, it is also possible to use additional criteria to disable a correction bound.

[0196] For example, one may force QC correction of a given transform coefficient to be performed with at most one correction bound by selecting which of the inferior or superior correction bound may be used and thus by disabling the other one. An example of selection is illustrated in Figure 16, based on the transform coefficient cfand the dequantized transform coefficient crobtained by dequantization of the quantization level cq(i.e., cr=

[0197]

[0198] In this case, either the inferior correction bound is disabled if cf> cr, or the superior correction bound is disabled otherwise. In the variant of the QC correction in Figure 8 where the correction function is applied to a symmetrized transform coefficient cfsym, the same criterion may be defined based on cfsymand a symmetrized dequantized transform coefficient crsymobtained by dequantization of the absolute value of quantization level, i.e., c,sym= Q-1(abs^c^.

[0199]

[0200] In this case, either the inferior correction bound is disabled if csym> crsymor the superior correction bound is disabled otherwise.

[0201] It may also be noted that even if a correction bound is not disabled according to this criterion, it may still be disabled for the QC correction if the corresponding bound offset is disabled.

[0202] In a variant, the disabling of one of the correction bounds illustrated in Figure 16 may be performed only if both the inferior correction bound binfand the superior correction bound bsupare enabled according to other criteria (e.g., correction bound enabled if the corresponding bound offsets is enabled, as in Figure 14 or Figure 15).

[0203] In another variant, the disabling of one of the correction bounds illustrated in Figure 16 may be performed only if both the inferior correction bound binfand the superior correction bound bsupare enabled according to other criteria, and are such that binf> bsup. Note that it is possible to determine if binf> bsupwithout computing explicitly binfand bsup. For example, according to Equations 3 and 4, if the quantization level cqis equal to 0 and if omax< 0, then it is known that binf> bsup. Similarly, if the quantization level cqis not equal to 0 and omax< omin, then it is known that binf> bsup.

[0204] In another example, both correction bounds may be disabled (i.e., QC correction may be disabled for the coefficient) if cqis equal to 0 and if omax< 0.

[0205] In another example, both correction bounds may be disabled (i.e., QC correction may be disabled for the coefficient) if cqis not equal to 0 and if omax< omin.

[0206] Early skipping of QC correction for a coefficient

[0207] Note that the condition for skipping QC correction in Figure 9 (i.e., if both correction bounds are disabled) may be directly evaluated using the quantization level cqand the flags min_enabled and max_enabled without explicitly determining if each correction bound is enabled.

[0208] For example, for both methods in Figure 14 and Figure 15, one may directly determine that both correction bounds are disabled (and thus skip QC correction of the coefficient) if “max_enabled=0 and (min_enabled=0 or cq=0)”.

[0209] If the additional criterion in Figure 16 is used for disabling a correction bound, then one may directly determine that both correction bounds are disabled if “

[0210]

[0211] (max_enabled=0 and (cf> cror cq=0)) or (min_enabled=0 and cf< crand cq#= 0)”.

[0212] Signaling or enabling or disabling bound offsets

[0213] Enabling or disabling each bound offset individually makes it possible to perform QC correction even if a single bound offset is available. Hence, in an embodiment described in Figure 17, the signaling of bound offset parameters in a bitstream is adapted to take advantage of the QC correction method based on individually enabled or disabled bound offsets.

[0214] For each region (e.g., CTU, slice, picture,...), a decoder decodes information indicating if the bound offset is enabled in the region. If a bound offset is enabled, the corresponding bound offset parameters (e.g., integer bound offset parameter Ot, integer shift parameter

[0215]

[0216] are then decoded. Finally, the QC correction is performed if at least one bound offset is enabled in the region, based on the information indicating if each bound offset is enabled in the region and on the parameters of the enabled bound offset(s) of the region.

[0217] Enabling or disabling flags In a first example, the information indicating if the bound offset is enabled is a flag signaled in the bitstream. For example, a flag min_enabled may be directly signaled in the bitstream to indicate if the minimum bound offset is enabled, and a flag max_enabled may be directly signaled in the bitstream to indicate if the maximum bound offset is enabled.

[0218] In a variant, only the max_enabled flag is signaled, and the min_enabled flag is always set to the same value (e.g., always enabled). Inversely, in another variant, only the min_enabled flag is signaled, and the min_enabled flag is always set to the same value.

[0219] Disabling using a specific value of the bound offset index

[0220] In a second example, a specific value of the bound_offset_index used in the signaling of an integer bound offset parameter (e.g., O0or O ) is reserved to indicate that the corresponding bound offset is disabled. For example, if a / V—bit integer bound offset Oi is signaled with Ntbits, the corresponding bound_offset_index can take values in the range [0, 2W*-1]. Hence, a value in this range (e.g., 0 or 2W*-1) may be used to indicate that the bound offset is disabled. Similarly, if a reduced bitdepth Mt(with Mi< Ni) is used for the signaling of Ot, a value in the range [0, 2M*-1] (e.g., the value 0 or 2M*-1) may be used to indicate that the bound offset is disabled.

[0221] This method avoids the signaling cost of a flag max_enabled or min_enabled (i.e., 1 bit) when the corresponding bound offset is enabled. However, it has a higher signaling cost than the direct signaling of the flag max_enabled or min_enabled when the bound offset is disabled (i.e.,

[0222]

[0223] or Mtbits instead of 1 bit). Hence, this method may be advantageous compared to the direct signaling of the flags max_enabled or min_enabled, only in situations where the bound offset is likely to be enabled. In practice, the likelihood of enabling a bound offset may depend on the type of bound offset (e.g., higher image quality may be obtained with QC correction only with a minimum bound offset than only with a maximum bound offset). Hence, this method may be used for enabling or disabling one of the bound offsets, and the signaling of a flag (max_enabled or min_enabled) may be used for the other bound offset.

[0224] Interaction with existing QC disabling mechanism

[0225] The QC correction may be disabled in a region (e.g., CTU, slice, picture,...) with the signaling of a flag use_qc. Hence, if QC correction is disabled in a region, the signaling of bound offset information (e.g., enabling flags max_enabled and min_enabled, bound offset parameters) should be skipped forthat region.

[0226] Inversely, if QC correction is enabled in a region, one may infer that at least one of the bound offsets is enabled. Hence, in a variant of the method in Figure 17, the decoder first decodes information indicating if the minimum bound offset is enabled. Then, if the minimum bound offset is disabled, the decoder infers that the maximum bound offset is enabled. Otherwise, the decoder decodes information indicating if the maximum bound offset is enabled.

[0227] In a similar variant, the decoder first decodes information indicating if the maximum bound offset is enabled. Then, if the maximum bound offset is disabled, the decoder infers that the minimum bound offset is enabled. Otherwise, the decoder decodes information indicating if the minimum bound offset is enabled.

[0228] Encoder decision for enabling or disabling

[0229] On the encoder side, the decision to either enable or disable a bound offset may be performed based on a rate distortion criterion, where a rate-distortion cost of a given mode may be obtained as:

[0230] rd_cost = R + A * D

[0231] where R is the rate of the mode (e.g. number of bits that need to be signaled in the bitstream), D is the distortion (e.g., sum of squared errors (SSE) between the original and the reconstructed image) and A is a Lagrangian parameter.

[0232] For example, for the minimum bound offset in a given region, two rate-distortion costs may be computed either without performing QC correction or by performing QC correction with the minimum bound offset enabled and the maximum bound offset disabled (e.g., the smallest rate-distortion cost of QC correction obtained with each allowed value of the integer bound offset O0may be used). If the rate distortion cost without QC correction is smaller, the minimum bound offset is disabled in the region, otherwise it is enabled.

[0233] Similarly, for the maximum bound offset in a given region, two rate-distortion costs may be computed either without performing QC correction or by performing QC correction with the maximum bound offset enabled and the minimum bound offset disabled (e.g., the smallest rate-distortion cost of QC correction obtained with each allowed value of the integer bound offset O may be used). If the rate-distortion cost without QC correction is smaller, the maximum bound offset is disabled in the region, otherwise it is enabled.

[0234] If all the bound offsets are disabled for the same region, the encoder may only signal in the bitstream a value of the flag use_qc indicating that QC correction is disabled without signaling further information of individual bound offsets for that region. Otherwise, the value of the flag use_qc signaled in the bitstream is set to a value indicating that QC correction is enabled. Information of each bound offset is finally signaled in the bitstream (i.e., information indicating if a bound offset is enabled, and parameters of the enabled bound offset(s)).

[0235] In order to reduce the computational complexity when computing the distortions with QC correction either with only the minimum bound offset enabled or with only the maximum bound offset enabled, a transform domain computation of the distortion may be performed. For example, for each block, the original transform coefficients c may be obtained by applying the transform to the difference between the original image block and the prediction block. The SSE is then computed between the corrected transform coefficients cfand the original transform coefficients c. Using this method, the distortions with QC correction are thus obtained without the need to fully reconstruct the QC corrected blocks from the corrected transform coefficients.

[0236] Computation of both bound offsets from a single bound offset

[0237] In another embodiment, the QC correction is performed by signaling a single bound offset o and by computing both the minimum and maximum bound offset using the single bound offset o. For example, the minimum and maximum bound offset ominand omaxmay be obtained as omin= -o and omax= o (or alternatively as omin= o and omax= — o). In another example, the bound offset o corresponds to the parameter a in Equation 2. In this case, the minimum and maximum bound offsets are obtained as omin= —o and omax= 1 — o.

[0238] In this embodiment, the single bound offset o may be constrained to take a positive value in order to have omin< omax. For example, in the integer representation of o as o = O / 2S, with an integer shift parameter S and a N-bit integer bound offset parameter 0 in the range [start, start + 2W- 1], the integer bound offset parameter 0 may be constrained to take positive values by using the starting value start = 0. In this case, the integer bound offset parameter 0 may be directly signaled in the bitstream (i.e., the bound_offset_index signaled in the bitstream is equal to 0). Note that this embodiment may be used along with the process described in Figure 9 to enable or disable the minimum and maximum bound offsets individually. In this case, the minimum bound offset ominmay be computed from the bound offset o only if min_enabled=1. Similarly, the maximum bound offset ominmay be computed from the bound offset o only if max_enabled=1. Additionally, if both bound offsets are disabled (i.e., min_enabled=0 and max_enabled=0) or equivalently if the QC correction is disabled (i.e., use_qc=0), the signaling of the bound offset o may be skipped.

[0239] In a variant, the minimum and maximum bound offsets are computed from a single bound offset o signaled in the bitstream only if both bound offsets are enabled (i.e., min_enabled=1 and max_enabled=1). Otherwise, the minimum bound offset ominis signaled if it is enabled (i.e., if min_enabled=1) or the maximum bound offset omaxis signaled if it is enabled (i.e., if max_enabled=1).

[0240] One embodiment of a method 1800 under the general aspects described here is shown in Figure 18. The method commences at start block 1801 and control proceeds to block 1810 for parsing a bitstream for enabled bound offsets. Control proceeds from block 1810 to block 1820 for selecting a correction function based on at least one enabled bound offset. Control proceeds from block 1820 to block 1830 for applying the correction function to a prediction residual transform coefficient. Control proceeds from block 1830 to block 1840 for decoding video data using the quantized transform coefficient after correction.

[0241] One embodiment of a method 1900 under the general aspects described here is shown in Figure 19. The method commences at start block 1901 and control proceeds to block 1910 for determining a correction function to be used for quantization constrained correction of a video block based on at least one enabled bound offset. Control proceeds from block 1910 to block 1920 for signaling in a bitstream at least one bound offset corresponding to the quantization constrained correction for encoding the video block.

[0242] Figure 20 shows one embodiment of an apparatus 2000 for encoding, decoding, compressing or decompressing, or filtering of video data using the aforementioned methods. The apparatus comprises Processor 2010 and can be interconnected to a memory 2020 through at least one port. Both Processor 2010 and memory 2020 can also have one or more additional interconnections to external connections.

[0243] Processor 2010 is also configured to either insert or receive information in a bitstream and, either compressing, encoding, decoding, or quantizing using any of the described aspects.

[0244] The embodiments described here include a variety of aspects, including tools, features, embodiments, models, approaches, etc. Many of these aspects are described with specificity and, at least to show the individual characteristics, are often described in a manner that may sound limiting. However, this is for purposes of clarity in description and does not limit the application or scope of those aspects. Indeed, all of the different aspects can be combined and interchanged to provide further aspects. Moreover, the aspects can be combined and interchanged with aspects described in earlier filings as well.

[0245] The aspects described and contemplated in this application can be implemented in many different forms. Figures 21, 22, and 23 provide some embodiments, but other embodiments are contemplated and the discussion of Figures 21, 22, and 23 does not limit the breadth of the implementations. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a bitstream generated or encoded. These and other aspects can be implemented as a method, an apparatus, a computer readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the methods described, and / or a computer readable storage medium having stored thereon a bitstream generated according to any of the methods described.

[0246] In the present application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, the terms “image,” “picture” and “frame” may be used interchangeably. Usually, but not necessarily, the term “reconstructed” is used at the encoder side while “decoded” is used at the decoder side.

[0247] Various methods are described herein, and each of the methods comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Various methods and other aspects described in this application can be used to modify modules, for example, the intra prediction, entropy coding, and / or decoding modules (160, 260, 145, 230), of a video encoder 100 and decoder 200 as shown in Figure 21 and Figure 22. Moreover, the present aspects are not limited to WC or HEVC, and can be applied, for example, to other standards and recommendations, whether pre-existing or future-developed, and extensions of any such standards and recommendations (including WC and HEVC). Unless indicated otherwise, or technically precluded, the aspects described in this application can be used individually or in combination.

[0248] Various numeric values are used in the present application. The specific values are for example purposes and the aspects described are not limited to these specific values.

[0249] Figure 21 illustrates an encoder 100. Variations of this encoder 100 are contemplated, but the encoder 100 is described below for purposes of clarity without describing all expected variations.

[0250] Before being encoded, the video sequence may go through pre-encoding processing (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to get a signal distribution more resilient to compression (for instance using a histogram equalization of one of the color components). Metadata can be associated with the pre-processing and attached to the bitstream.

[0251] In the encoder 100, a picture is encoded by the encoder elements as described below. The picture to be encoded is partitioned (102) and processed in units of, for example, CUs. Each unit is encoded using, for example, either an intra or inter mode. When a unit is encoded in an intra mode, it performs intra prediction (160). In an inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which one of the intra mode or inter mode to use for encoding the unit, and indicates the intra / inter decision by, for example, a prediction mode flag. Prediction residuals are calculated, for example, by subtracting (110) the predicted block from the original image block.

[0252] The prediction residuals are then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the non-transformed residual signal. The encoder can bypass both transform and quantization, i.e., the residual is coded directly without the application of the transform or quantization processes.

[0253] The encoder decodes an encoded block to provide a reference for further predictions. The quantized transform coefficients are de-quantized (140) and inverse transformed (150) to decode prediction residuals. Combining (155) the decoded prediction residuals and the predicted block, an image block is reconstructed. In-loop filters (165) are applied to the reconstructed picture to perform, for example, deblocking / SAO (Sample Adaptive Offset) filtering to reduce encoding artifacts. The filtered image is stored at a reference picture buffer (180).

[0254] Figure 22 illustrates a block diagram of a video decoder 200. In the decoder 200, a bitstream is decoded by the decoder elements as described below. Video decoder 200 generally performs a decoding pass reciprocal to the encoding pass as described in Figure 21. The encoder 100 also generally performs video decoding as part of encoding video data.

[0255] In particular, the input of the decoder includes a video bitstream, which can be generated by video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. The picture partition information indicates how the picture is partitioned. The decoder may therefore divide (235) the picture according to the decoded picture partitioning information. The transform coefficients are de-quantized (240) and inverse transformed (250) to decode the prediction residuals. Combining (255) the decoded prediction residuals and the predicted block, an image block is reconstructed. The predicted block can be obtained (270) from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275). In-loop filters (265) are applied to the reconstructed image. The filtered image is stored at a reference picture buffer (280).

[0256] The decoded picture can further go through post-decoding processing (285), for example, an inverse color transform (e.g. conversion from YcbCr 4:2:0 to RGB 4:4:4) or an inverse remapping performing the inverse of the remapping process performed in the pre-encoding processing (101). The post-decoding processing can use metadata derived in the pre-encoding processing and signaled in the bitstream.

[0257] Figure 23 illustrates a block diagram of an example of a system in which various aspects and embodiments are implemented. System 1000 can be embodied as a device including the various components described below and is configured to perform one or more of the aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, the system 1000 is communicatively coupled to one or more other systems, or other electronic devices, via, for example, a communications bus or through dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more of the aspects described in this document.

[0258] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing, for example, the various aspects described in this document. Processor 1010 can include embedded memory, input output interface, and various other circuitries as known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device, and / or a nonvolatile memory device). System 1000 includes a storage device 1040, which can include non-volatile memory and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash, magnetic disk drive, and / or optical disk drive. The storage device 1040 can include an internal storage device, an attached storage device (including detachable and non-detachable storage devices), and / or a network accessible storage device, as non-limiting examples.

[0259] System 1000 includes an encoder / decoder module 1030 configured, for example, to process data to provide an encoded video or decoded video, and the encoder / decoder module 1030 can include its own processor and memory. The encoder / decoder module 1030 represents module(s) that can be included in a device to perform the encoding and / or decoding functions. As is known, a device can include one or both of the encoding and decoding modules. Additionally, encoder / decoder module 1030 can be implemented as a separate element of system 1000 or can be incorporated within processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0260] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to perform the various aspects described in this document can be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. In accordance with various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 can store one or more of various items during the performance of the processes described in this document. Such stored items can include, but are not limited to, the input video, the decoded video or portions of the decoded video, the bitstream, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0261] In some embodiments, memory inside of the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and to provide working memory for processing that is needed during encoding or decoding. In other embodiments, however, a memory external to the processing device (for example, the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory can be the memory 1020 and / or the storage device 1040, for example, a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, an external non-volatile flash memory is used to store the operating system of, for example, a television. In at least one embodiment, a fast external dynamic volatile memory such as a RAM is used as working memory for video coding and decoding operations, such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by JVET, the Joint Video Experts Team).

[0262] The input to the elements of system 1000 can be provided through various input devices as indicated in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives an RF signal transmitted, for example, over the air by a broadcaster, (ii) a Component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a n High Definition Multimedia Interface (HDMI) input terminal. Other examples, not shown in Figure 23, include composite video.

[0263] In various embodiments, the input devices of block 1130 have associated respective input processing elements as known in the art. For example, the RF portion can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select (for example) a signal frequency band which can be referred to as a channel in certain embodiments, (iv) demodulating the downconverted and bandlimited signal, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF portion of various embodiments includes one or more elements to perform these functions, for example, frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion can include a tuner that performs various of these functions, including, for example, downconverting the received signal to a lower frequency (for example, an intermediate frequency or a near-baseband frequency) or to baseband. In one set-top box embodiment, the RF portion and its associated input processing element receives an RF signal transmitted over a wired (for example, cable) medium, and performs frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements performing similar or different functions. Adding elements can include inserting elements in between existing elements, such as, for example, inserting amplifiers and an analog-to-digital converter. In various embodiments, the RF portion includes an antenna.

[0264] Additionally, the USB and / or HDMI terminals can include respective interface processors for connecting system 1000 to other electronic devices across USB and / or HDMI connections. It is to be understood that various aspects of input processing, for example, Reed-Solomon error correction, can be implemented, for example, within a separate input processing IC or within processor 1010 as necessary. Similarly, aspects of USB or HDMI interface processing can be implemented within separate interface ICs or within processor 1010 as necessary. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 1010, and encoder / decoder 1030 operating in combination with the memory and storage elements to process the datastream as necessary for presentation on an output device.

[0265] Various elements of system 1000 can be provided within an integrated housing, Within the integrated housing, the various elements can be interconnected and transmit data therebetween using suitable connection arrangement, for example, an internal bus as known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0266] The system 1000 includes communication interface 1050 that enables communication with other devices via communication channel 1060. The communication interface 1050 can include, but is not limited to, a transceiver configured to transmit and to receive data over communication channel 1060. The communication interface 1050 can include, but is not limited to, a modem or network card and the communication channel 1060 can be implemented, for example, within a wired and / or a wireless medium.

[0267] Data is streamed, or otherwise provided, to the system 1000, in various embodiments, using a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The WiFi signal of these embodiments is received over the communications channel 1060 and the communications interface 1050 which are adapted for Wi-Fi communications. The communications channel 1060 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet for allowing streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 1000 using a set-top box that delivers the data over the HDMI connection of the input block 1130. Still other embodiments provide streamed data to the system 1000 using the RF connection of the input block 1130. As indicated above, various embodiments provide data in a nonstreaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, for example a cellular network or a Bluetooth network.

[0268] The system 1000 can provide an output signal to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or another device. The display 1100 can also be integrated with other components (for example, as in a smart phone), or separate (for example, an external monitor for a laptop). The other peripheral devices 1120 include, in various examples of embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide a function based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0269] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV. Link, Consumer Electronics Control (CEC), or other communications protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be connected to system 1000 using the communications channel 1060 via the communications interface 1050. The display 1100 and speakers 1110 can be integrated in a single unit with the other components of system 1000 in an electronic device such as, for example, a television. In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0270] The display 1100 and speaker 1110 can alternatively be separate from one or more of the other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments in which the display 1100 and speakers 1110 are external components, the output signal can be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0271] The embodiments can be carried out by computer software implemented by the processor 1010 or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. The memory 1020 can be of any type appropriate to the technical environment and can be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 1010 can be of any type appropriate to the technical environment, and can encompass one or more of microprocessors, general purpose computers, special purpose computers, and processors based on a multi-core architecture, as non- limiting examples.

[0272] Various implementations involve decoding. “Decoding”, as used in this application, can encompass all or part of the processes performed, for example, on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by a decoder of various implementations described in this application.

[0273] As further examples, in one embodiment “decoding” refers only to entropy decoding, in another embodiment “decoding” refers only to differential decoding, and in another embodiment “decoding” refers to a combination of entropy decoding and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or generally to the broader decoding process will be clear based on the context of the specific descriptions and is believed to be well understood by those skilled in the art.

[0274] Various implementations involve encoding. In an analogous way to the above discussion about “decoding”, “encoding” as used in this application can encompass all or part of the processes performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, for example, partitioning, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, such processes also, or alternatively, include processes performed by an encoder of various implementations described in this application.

[0275] As further examples, in one embodiment “encoding” refers only to entropy encoding, in another embodiment “encoding” refers only to differential encoding, and in another embodiment “encoding” refers to a combination of differential encoding and entropy encoding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or generally to the broader encoding process will be clear based on the context of the specific descriptions and is believed to be well understood by those skilled in the art.

[0276] Note that the syntax elements as used herein are descriptive terms. As such, they do not preclude the use of other syntax element names.

[0277] When a figure is presented as a flow diagram, it should be understood that it also provides a block diagram of a corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow diagram of a corresponding method / process.

[0278] Various embodiments may refer to parametric models or rate distortion optimization. In particular, during the encoding process, the balance or trade-off between the rate and distortion is usually considered, often given the constraints of computational complexity. It can be measured through a Rate Distortion Optimization (RDO) metric, or through Least Mean Square (LMS), Mean of Absolute Errors (MAE), or other such measurements. Rate distortion optimization is usually formulated as minimizing a rate distortion function, which is a weighted sum of the rate and of the distortion. There are different approaches to solve the rate distortion optimization problem. For example, the approaches may be based on an extensive testing of all encoding options, including all considered modes or coding parameters values, with a complete evaluation of their coding cost and related distortion of the reconstructed signal after coding and decoding. Faster approaches may also be used, to save encoding complexity, in particular with computation of an approximated distortion based on the prediction or the prediction residual signal, not the reconstructed one. Mix of these two approaches can also be used, such as by using an approximated distortion for only some of the possible encoding options, and a complete distortion for other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete evaluation of both the coding cost and related distortion.

[0279] The implementations and aspects described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. The methods can be implemented in, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.

[0280] Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment.

[0281] Additionally, this application may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

[0282] Further, this application may refer to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0283] Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0284] It is to be appreciated that the use of any of the following “ / ”, “and / or”, and “at least one of”, for example, in the cases of “A / B”, “A and / or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.

[0285] Also, as used herein, the word “signal” refers to, among other things, indicating something to a corresponding decoder. For example, in certain embodiments the encoder signals a particular one of a plurality of transforms, coding modes or flags. In this way, in an embodiment the same transform, parameter, or mode is used at both the encoder side and the decoder side. Thus, for example, an encoder can transmit (explicit signaling) a particular parameter to the decoder so that the decoder can use the same particular parameter. Conversely, if the decoder already has the particular parameter as well as others, then signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select the particular parameter. By avoiding transmission of any actual functions, a bit savings is realized in various embodiments. It is to be appreciated that signaling can be accomplished in a variety of ways. For example, one or more syntax elements, flags, and so forth are used to signal information to a corresponding decoder in various embodiments. While the preceding relates to the verb form of the word “signal”, the word “signal” can also be used herein as a noun.

[0286] As will be evident to one of ordinary skill in the art, implementations can produce a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal can be formatted to carry the bitstream of a described embodiment. Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

[0287] The preceding sections describe a number of embodiments, across various claim categories and types. Features of these embodiments can be provided alone or in any combination. Further, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across various claim categories and types:

[0288] At least one embodiment comprises performing an analysis of block vectors neighboring a current video block and determining a prediction by blending predictions of at least two predictors using weights based on said analysis.

[0289] At least one embodiment comprises the above embodiment and further comprising encoding or decoding the video block using the determined prediction.

[0290] At least one embodiment comprises a bitstream or signal that includes one or more of the described syntax elements, or variations thereof.

[0291] At least one embodiment comprises a bitstream or signal that includes syntax conveying information generated according to any of the embodiments described.

[0292] At least one embodiment comprises creating and / or transmitting and / or receiving and / or decoding according to any of the embodiments described.

[0293] At least one embodiment comprises a method, process, apparatus, medium storing instructions, medium storing data, or signal according to any of the embodiments described.

[0294] At least one embodiment comprises inserting in the signaling syntax elements that enable the decoder to determine decoding information in a manner corresponding to that used by an encoder.

[0295] At least one embodiment comprises creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described syntax elements, or variations thereof.

[0296] At least one embodiment comprises a TV, set-top box, cell phone, tablet, or other electronic device that performs transform method(s) according to any of the embodiments described.

[0297] At least one embodiment comprises a TV, set-top box, cell phone, tablet, or other electronic device that performs transform method(s) determination according to any of the embodiments described, and that displays (e.g., using a monitor, screen, or other type of display) a resulting image.

[0298] At least one embodiment comprises a TV, set-top box, cell phone, tablet, or other electronic device that selects, bandlimits, or tunes (e.g., using a tuner) a channel to receive a signal including an encoded image, and performs transform method(s) according to any of the embodiments described. At least one embodiment comprises a TV, set-top box, cell phone, tablet, or other electronic device that receives (e.g., using an antenna) a signal over the air that includes an encoded image, and performs transform method(s).

Claims

1. CLAIMS1. A method, comprising:3.parsing a bitstream for a first syntax element indicating whether a minimum bound offset is enabled and a second syntax element indicating whether a maximum bound offset is enabled;4.selecting a correction function based on the first and second syntax elements; and5.applying the correction function on a transform coefficient of a video signal.

2. An apparatus, comprising a memory and a processor configured for: parsing a bitstream for a first syntax element indicating whether a minimum bound offset is enabled and a second syntax element indicating whether a maximum bound offset is enabled;7.selecting a correction function based on the first and second syntax elements; and8.applying the correction function on a transform coefficient of a video signal.

3. A method, comprising:10.signaling in a bitstream a first syntax element indicating whether a minimum bound offset is enabled and a second syntax element indicating whether a maximum bound offset is enabled;11.selecting a correction function based on the first and second syntax elements; and12.applying the correction function on a transform coefficient of a video signal.

4. An apparatus, comprising a memory and a processor configured for: signaling in a bitstream a first syntax element indicating whether a minimum bound offset is enabled and a second syntax element indicating whether a maximum bound offset is enabled;14.selecting a correction function based on the first and second syntax elements; and15.applying the correction function on a transform coefficient of a video signal.

5. The method of claim 1 or 3, or the apparatus of claim 2 or 4, wherein selecting a correction function based on the first and second syntax elements comprises:16.determining whether an inferior correction bound is enabled based on a quantization level associated with the transform coefficient and the first and second syntax elements;17.determining whether a superior correction bound is enabled based on the quantization level associated with a transform coefficient and the first and second syntax elements; and18.selecting the correction function responsive to the determining.

6. The method of claim 5, or the apparatus of claim 5, wherein selecting the correction function responsive to the determining comprises:20.selecting an inferior correction function responsive to determining that the inferior correction bound is enabled and the superior correction bound is disabled;21.selecting a superior correction function responsive to determining that inferior correction bound is disabled and superior correction bound is enabled; and selecting an inferior and superior correction function responsive to determining that both inferior and superior correction bounds are enabled.

7. The method of claim 6, or the apparatus of claim 6, wherein applying the correction function on a transform coefficient comprises:23.computing the inferior correction bound responsive to determining that the inferior correction bound is enabled;24.computing the superior correction bound responsive to determining that the superior correction bound is enabled; and25.applying the correction function on the transform coefficient based on the enabled inferior and superior correction bounds.

8. A device comprising:27.an apparatus according to Claim 2; and28.at least one of (i) an antenna configured to receive a signal, the signal including a video signal, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes a video block, and (iii) a display configured to display an output representative of the video block.

9. A non-transitory computer readable medium containing data content generated according to the method of any one of claims 3, or 5 through 7, or by the apparatus of any one of claims 4, or 5 through 7, for playback using a processor.

10. A signal comprising video data generated according to the method of any one of claims 3, or 5 through 7, or by the apparatus of any one of claims 4, or 5 through 7, for playback using a processor.

11. A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of claims 3 or 5 through 7.

12. A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of claims 1 or 5 through 7.

Images