Determining coefficients of adaptive loop filter using mantissa and exponent

Abstract

A method for performing a video coding. The method comprises determining an allowed set of coefficient values of adaptive loop filter, ALF, wherein each coefficient value included in the allowed set of ALF coefficient values is m · 2 b where m is a mantissa, b is an exponent. The method further comprises determining a plurality of ALF coefficient values to use for the video coding, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values. The method further comprises performing the video coding using the plurality of ALF coefficient values.

Description

DETERMINING COEFFICIENTS OF ADAPTIVE LOOP FILTER USING MANTISSA AND EXPONENT TECHNICAL FIELD

[0001] This disclosure relates to determining coefficients of adaptive loop filter (ALF) using mantissa and exponent. The ALF is for video coding (e.g., video encoding and decoding). BACKGROUND

[0002] Enhanced Coding Model (ECM)

[0003] Versatile Video Coding (VVC), also known as H.266, is a block-based video codec which was developed by the Joint Video Experts Team (JVET), a team of experts from ITU-T VCEG and ISO / IEC MPEG, and which was firstly published in 2020.

[0004] JVET continues study on future video codec providing coding capabilities beyond VVC. The exploratory codec is called Enhanced Coding Model (ECM). The current ECM version is ECM-13.0.

[0005] Video Sequence, Picture, Color Components

[0006] ECM is used for compression of video sequences that consist of successive pictures. Each picture consists of 2-dimensional arrays of color components. Usually, each picture has three components: one luma component (Y) and two chroma components (Cb, Cr). Instead of Y, Cb, Cr, other color spaces such as I, Cp, Ct can also be used. Usually, the chroma components are half the size in both the x- and y- dimensions as compared to the luma component.

[0007] Coding Tree Unit (CTU), Coding Unit (CU), and Transform Unit (TU)

[0008] In VVC and ECM, each picture component is split into square top-level blocks called Coding Tree Units (CTUs). The maximum supported size of a CTU is 256x256 in ECM. Each CTU is split into rectangular Coding Units (CUs). The size of a CU varies from 4x4 to 256x256 in ECM.

[0009] It is common that a CU is associated with a Transform Unit (TU) and is then transform coded. However, if the size of the CU is higher than the maximum supported transform length, then the CU is split into several TUs.

[0010] Parameter Sets and Headers

[0011] ECM specifies three types of parameter sets: Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). The PPS contains data that is common for a whole picture, the SPS contains data that is common for a coded layer video sequence (CLVS), and the VPS contains data that is common for multiple CLVSs.

[0012] During coding, a picture is split into slices each of which consists of several CTUs. Decoding of one slice in the picture is independent of decoding of other slices in the same picture. Pictures, slices, and CTUs have headers comprising syntax elements used by the decoder. Coding parameters in the picture header are common for all slices in the picture and coding parameters in the slice header are common for all CTUs in the slice.

[0013] Intra Prediction and Inter Prediction

[0014] Each CU is either intra predicted or inter predicted. If the CU is intra predicted, it is predicted using previously decoded blocks in the same picture. An intra picture is a picture in which all CUs must be intra predicted.

[0015] In inter prediction, the CU is predicted using previously decoded pictures. A picture that allows inter predicted blocks is called an inter picture.

[0016] Transform, Quantization, and Entropy Coding

[0017] Difference between the original samples and predicted samples of a CU is called residual. The residual is transformed into the frequency domain using one of the transforms supported by ECM. Examples of such transforms include but are not limited to the Discrete Cosine Transform (DCT), the Discrete Sine Transform (DST), etc.

[0018] The resulting transform coefficients are quantized and entropy coded, and the resulting bits are put into a bitstream. The level of quantization depends on quantization parameter (“QP parameter”) used for the CU. In some settings, all CUs belonging to the same picture may have the same QP parameter value.

[0019] Rate-Distortion Optimization (RDO)

[0001] Encoder in ECM extensively utilizes the rate-distortion theory to make decisions such as how to split a CTU into CUs, what type of prediction should be used, what area of the reference picture should be used for inter prediction, etc. To make such decisions, rate-distortion (RD) cost is calculated as (MSE ^ ^^ ⋅ Rate) and the RD cost is minimized, whereMSE is the mean square error (MSE) between the original picture and the reconstructed picture, Rate is the number of bits required to be signaled into the bitstream, and ^^ is the rate distortion optimization (RDO) parameter derived by the encoder.

[0002] In-Loop Filters

[0001] Since transform coefficients are quantized during video encoding, the video encoding produces losses. To reduce some quantization artifacts and improve video quality, in-loop filters are introduced. In-loop filtering is done on the decoder side as well as on the encoder side. In the in-loop filtering, filtered pictures are used for inter-prediction. There are four in- loop filters in ECM-13.0: deblocking filter (DBF), sample adaptive offset (SAO), bilateral filter (BIF), and adaptive loop filter (ALF). SAO and ALF can be supplemented by cross-component (CC) counterparts. A flow of the in-loop filtering is shown in FIG.6.

[0002] Adaptive Loop Filter (ALF)

[0003] Adaptive loop filtering is designed to reduce quantization errors of the output pictures as well as of the reference pictures used for inter prediction, as disclosed in Reference [1] listed at the end of this disclosure. ALF is based on the Wiener filter, i.e. a linear filter that minimizes mean square error (MSE) between the original picture and the noised picture. In using ALF for video coding, the filter coefficients are signaled into a bitstream so that the decoder receiving the bitstream can apply the filter and improve quality of the reconstructed picture.

[0004] Let ^^^^^,^^^ be an output sample of the decoding stage preceding ALF, i.e. the sample at position ^^^,^^^ outputted by the combination of SAO, CC-SAO, and BIF (see FIG.6). TheALF output ^^^^^^,^^^ is defined as^ ^^^^^^,^^^ ൌ ^^^^^,^^^ ^ ^ ^^^^^^^^^,^^^,^1^ ^ୀ^ where ^^^is an ^^-th ^^^,^^^.

[0005] As disclosed infor luma samples, ECM-13.0 utilizes ALF with the filtering shape shown in FIG. 7 with 40 taps, where the first 20 taps (taps 0-19) are samples in the neighboring pixels (spatial taps) and the other taps have more complex nature and include taps based on the output of the fixed filters, taps based on the input for deblocking filter (DBF), and taps based on the reconstructed residuals.

[0006] Coefficients ^^^are derived by the encoder. The derivation used in ECM-13.0 can be split into several parts. First, the optimal real valued coefficients are obtained by solving the Wiener-Hopf equations to minimize the mean square error (MSE) between the original picture and the filtered picture. Then, the coefficients are quantized, namely, multiplied by a constant2^^ైూି^, rounded to integers, and clipped into the interval ^െ2^^ైూି^ ^ 1, 2^^ైూି^ െ 1^. Here,the parameter ^^^^^is called “precision.” Finally, a descent-based procedure is launched: ateach iteration, each coefficient is tried to be increased or decreased by 1, the best (by MSE) change of the coefficient is accepted, and the iterations continue while there is an improvement of MSE. The actual filtering is done by formula (1) in integers and right bit shifting is utilizedto properly scale the ALF offset ∑^ ^ୀ^ ^^^^^^^^^,^^^.

[0007] The quantized coefficients ^^^are signaled in a special header called adaptation parameter set (APS). In ECM-13.0, the coefficients are split into two groups (0-33 and 34-39) and different Golomb-Rice codesdifferent groups are used to convert the coefficients into the APS bitstream. For each group, four parameters of Golomb-Rice coding are iterated, and the best (by total required bits) choice is signaled in APS.

[0008] In ECM-13.0, the precision parameter ^^^^^is fixed to 8 for chroma component but is adaptive for luma component. Namely, for luma components, the values 6, 7, 8 and 9 are iterated for the parameter, and the best (by RD-cost) choice is signaled in APS. SUMMARY

[0009] Certain challenges presently exist. For example, in ECM, ALF performs dozens of multiplications for coding of each pixel, which make hardware implementation of ECM unfeasible. Therefore, there is a need to replace the multiplications performed by ALF with simpler operations, thereby reducing the complexity of ALF and possibly improving the performance of ALF.

[0010] Accordingly, in one aspect of the embodiments of this disclosure, there is provided a method for performing a video coding. The method comprises determining an allowed set of coefficient values of adaptive loop filter, ALF, wherein each coefficient value included in theallowed set of ALF coefficient values is ^^ ∙ 2^ where ^^ is a mantissa, ^^ is an exponent. Themethod further comprises determining a plurality of ALF coefficient values to use for the video coding, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values. The method further comprises performing the video coding using the plurality of ALF coefficient values.

[0011] In another aspect, there is provided a computer program comprising instructions which when executed by processing circuitry cause the processing circuitry to perform the method of any one of the above embodiments.

[0012] In a different aspect, there is provided a carrier containing the computer program of the above embodiment, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

[0013] In a different aspect, there is provided an apparatus for performing a video coding. The apparatus is configured to determine an allowed set of coefficient values of adaptive loop filter, ALF, wherein each coefficient value included in the allowed set of ALF coefficient values is^^ ∙ 2^ where ^^ is a mantissa, ^^ is an exponent. The apparatus is configured to determine aplurality of ALF coefficient values to use for the video coding, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values. The apparatus is configured to perform the video coding using the plurality of ALF coefficient values.

[0014] In a different aspect, there is provided an apparatus comprising processing circuitry and a memory. Said memory contains instructions executable by said processing circuitry, whereby the apparatus is operative to perform the method of any one of the above embodiments.

[0015] Embodiments of this disclosure allow reducing the complexity of hardware implementation of ALF as well as improving compression efficiency (which is, for example, measured as Bjontegaard delta (BD) rate) as compared to the existing ALF. In one implementation of the embodiments of this disclosure, bit-rate savings of 0.03% / 0.08% / 0.11% for AI / RA / LDB are achieved with the same run time for CPU implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments.

[0017] FIG.1 illustrates a system according to some embodiments.

[0018] FIG.2 is a schematic block diagram of an encoder according to some embodiments.

[0019] FIG.3 is a schematic block diagram of a decoder according to some embodiments.

[0020] FIG.4 shows a process according to some embodiments.

[0021] FIG.5 shows an apparatus according to some embodiments.

[0022] FIG.6 shows a flow of in-loop filtering.

[0023] FIG.7 shows ALF shapes in ECM-13.0.

[0024] FIG. 8 shows an exemplary hardware implementation of ALF operation according to some embodiments.

[0025] FIG.9 shows a process according to some embodiments.DETAILED DESCRIPTION

[0026] FIG.1 illustrates a system 100 according to an embodiment. System 100 includes an encoder 102 and a decoder 104. In some embodiments, encoder 102 is in communication with decoder 104 via a network 110 (e.g., the Internet or other network). Encoder 102 encodes a source video sequence 101 into a bitstream comprising an encoded video sequence and may transmit the bitstream to decoder 104 via network 110.

[0027] In some embodiments, instead of or in addition to transmitting bitstream to decoder 104, the bitstream may be stored in a data storage unit 190 and decoder 104 may retrieve from the data storage unit 190 the bitstream containing the encoded video sequence. Data storage unit 190 may be co-located with encoder 102 or may be remote from encoder 102.

[0028] Decoder 104 decodes the pictures included in the encoded video sequence to produce video data (e.g., reconstructed pictures) for display. In some embodiments, decoder 104 may be part of a device 103 having a display device 106 (or display 106 for short). Device 103 may be a mobile device, a set-top device, a head-mounted display, or any other device, and may be a component of an extended reality (XR) system.

[0029] FIG. 2 illustrates functional components of encoder 102 according to some embodiments. It should be noted that encoders may be implemented differently so implementation other than this specific example can be used. Encoder 102 employs a subtractor 241 to produce a residual block which is the difference in sample values between an input block and a prediction block (i.e., the output of a selector 251, which is either an inter prediction block output by an inter predictor 250 (a.k.a., motion compensator) or an intra prediction block output by an intra predictor 249). Then a forward transform 242 is performed on the residual block to produce a transformed block comprising transform coefficients. A quantization unit 243 quantizes the transform coefficients based on a quantization parameter (QP) value (e.g., a QP value obtained based on a picture QP value for the picture in which the input block is a part and a block specific QP offset value for the input block), thereby producing quantized transform coefficients which are then encoded into the bitstream by entropy encoder 244 (e.g., an entropy encoder) and the bitstream with the encoded transform coefficients is output from encoder 102. Next, encoder 102 uses the quantized transform coefficients to produce a reconstructed block. This is done by first applying inverse quantization 245 and inverse transform 246 to the transform coefficients to produce a reconstructed residual block and using an adder 247 to add the prediction block to the reconstructed residual block, therebyproducing the reconstructed block, which is stored in the reconstructed picture buffer (RPB) 266. Loop filtering by an in-loop filter (LF) stage 267 is applied and the final decoded picture is stored in a decoded picture buffer (DPB) 268, where it can then be used by the inter predictor 250 to produce an inter prediction block for the next picture to be processed. LF stage 267 may include three sub-stages: i) a DBF, ii) an SAO filter and a BIF, and iii) an ALF.

[0030] FIG. 3 illustrates functional components of decoder 104 according to some embodiments. It should be noted that decoder 104 may be implemented differently so implementations other than this specific example can be used. Decoder 104 includes a decoder module 361 (e.g., an entropy decoder) that decodes from the bitstream quantized transform coefficient values of a block. Decoder 104 also includes a reconstruction stage 398 in which the quantized transform coefficient values are subject to an inverse quantization process 362 and inverse transform process 363 to produce a residual block. This residual block is input to adder 364 that adds the residual block and a prediction block output from selector 390 to form a reconstructed block. Selector 390 either selects to output an inter prediction block or an intra prediction block. The reconstructed block is stored in a RPB 365. The inter prediction block is generated by inter prediction module 350 and the intra prediction block is generated by intra prediction module 369. Following the reconstruction stage 398, an in-loop filter stage (LF) 367 applies one or more filters to the reconstructed blocks and the final decoded picture may be stored in a decoded picture buffer (DPB) 368. Like LF stage 267 shown in FIG. 2, LF stage 367 may include three sub-stages: i) a DBF, ii) an SAO filter and a BIF, and iii) an ALF.

[0031] Note that, as explained above, during a video coding, a coefficient of an ALF may be quantized, rounded to an integer, and clipped to a given interval. In the remaining disclosure, “coefficient” may mean a coefficient after the quantization, being rounded to an integer, and / or being clipped to a given interval.

[0032] As mentioned above, in video coding, the operation of ALF involves performing the following mathematic calculation: ^ ^^^^^^,^^^ ൌ ^^^^^,^^^ ^ ^^^^^^^^^^,^^^^ୀ^where ^^^^^^,^^^ is the output ^^^ is an ^^-th coefficient, and^^^^^^,^^^ is an ^^-th tap for

[0033] However, calculating ∑^ ^ୀ^ ^^^^^^^^^,^^^for each pixel may require more than dozens of multiplications, and thus may be computationally expensive and complex. This is especially true when the number of bits indicating the value of ^^^is large (e.g., 9 bits). Therefore, there is a need to replace the multiplications performed by ALF with simpler operations, which would result in reducing the complexity of ALF and possibly improving the performance of ALF.

[0034] Accordingly, in some embodiments of this disclosure, the number of bits that aremultiplied by ^^^^^, ^^^ is ^^ reduced by limiting the value of each ALF coefficient ^^^ to ^^ ∙ 2where ^^ is a mantissa and ^^ is an exponent. The mantissa ^^ holds the main digit in bits and the exponent ^^ indicates where the decimal point should be placed within the main digit.

[0035] The concept of mantissa and exponent is as follows: Let’s assume that in binaryrepresentation ^^ ൌ 00001 and ^^ ൌ 0001. In this case, ^^ indicates +0001 (i.e.1) because thevery first digit “0” of the binary representation indicates that the mantissa is positive. On the contrary, if the first digit is “1,” then the mantissa is negative. ^^ indicates an amount of bitshifts to be applied to the absolute value of mantissa. Here because ^^ ൌ 0001 which is equalto 1, a single bit shifting is applied to the mantissa ^^, thereby changing +0001 (i.e.1) to +0010(i.e. 2). Thus, in this example, ^^ ∙ 2^ indicates the value of 2.

[0036] The complexity of ALF operation can be reduced by reducing the number of bits that are multiplied by ^^^^^^,^^^, and the number of bits multiplied by ^^^^^^,^^^ can be reduced bylimiting the value of each ALF coefficient ^^ ^^ to ^^ ∙ 2 where ^^ is a mantissa and ^^ is anexponent.

[0037] For example, let’s assume that the number of bits indicating the value of an ALF coefficient (^^^) is 9 bits and the number of bits indicating ^^^^^^,^^^ is 12 bits. Then calculating ^^^^^^^^^,^^^ would require performing a multiplication of 9-bit integer and 12-bit integer. On the contrary, in some embodiments of this disclosure, because the value of an ALF coefficient (^^^)is limited to ^^^ ൌ ^^^ ൈ 2^^, calculating ^^^^^^^^^,^^^ would be same as calculating ^^^^ ൈ 2^^^ ൈ^^^^^^,^^^ which is same as ^^^^ ൈ ^^^^^^,^^^^ ൈ 2^^. Here because performing the operation of“ൈ 2^^ ” corresponds to a bit shifting, the only multiplication required for calculating^^^^ ൈ ^^^^^^,^^^^ ൈ 2^^ would be ^^^ ൈ ^^^^^^,^^^. Because the number of bits indicating ^^^ isgenerally much smaller (e.g., 2) than the number of bits indicating ^^^(e.g., 9), the complexityof the multiplication needed for calculating ^^^^^^^^^,^^^ in case ^^ ^^ is limited to ^^^ ൌ ^^^ ൈ 2 ^would be much less than the complexity of the multiplication needed for calculating ^^^^^^^^^, ^^^in case ^^^is a regular digit in a binary representation.

[0038] As explained above, the mathematical operation of ^^ ൈ 2^ is simply left bit shifting ofthe binary representation, i.e. ^^ ≪ ^^ . Similarly, the mathematical operation of ^^^^ ൈ^^^^^^,^^^^ ൈ 2^^ is ൫^^^ ൈ ^^^^^^,^^^൯ ≪ ^^^. Let ^^^ be ^^ெ-bit number, i.e. ^^ெ bits are enough forbinary representation of ^^^, then instead of multiplication of ^^^^^^,^^^ by ^^^ெ ^ ^^^^-bit number^^^ ൈ needs to multiply ^^^^^^,^^^ by ^^ெ -bit number what significantly decrease thehardware complexity of the multiplication ^^^^^^^^^,^^^, especially if ^^^ ^ 3, because ^^ ൈ 0 ൌ 0,^^ ൈ 1 ൌ ^^, ^^ ൈ 2 ൌ ^^^ ≪ 1^ and ^^ ൈ 3 ൌ ^^^ ≪ 1^ ^ ^^. Accordingly, by limiting the value ofeach ALF coefficient ^^ ^^ to ^^^ ൈ 2 ^ with a reasonable set of allowed mantissa values (e.g.െ3 ^ ^^^ ^ 3), multiplications needed for calculating ∑^ ^ୀ^ ^^^^^^^^^,^^^can be replaced with hardware-friendly multiplications that can be implemented using only bit shifting and summation operations, thereby reducing the complexity ofoperation.

[0039] FIG. 8 illustrates an example of how the representation -- ^^^^ ൈ ^^^^^^,^^^^ ൈ 2^^ -- canbe efficiently implemented in hardware with very little resources. As mentioned above, instead of performing a full multiplier between the coefficient ^^^and the sample value ^^^^^^,^^^ using^^^ ൈ ^^^^^^,^^^, some embodiments of this disclosure make use of the fact that the coefficient ^^^is equal to ^^ ^^ ൈ 2 , and that ^^^ ∈ ^െ3,3^. Let’s assume that the value of ^^^^^^,^^^ is 12-bitvalue -- ^^^^^^^^^^^^^ଽ… ^^^^where ^^^^^is the sign bit and ^^^^is the least significant bit.

[0040] The 12-bit sample value is input to both bit-wise AND units 807 and 808 as inputs 805and 806.then ^^^ can be written as a three-bit signed two-complementnumber ^^^^^^^^^^^^, where ^^^is the least significant bit. The least si^^^ ^ ^gnificant bit ^^^803 will be used to selectively zero out the output of the bit wise AND unit 808; if ^^^^ is zero, the bit-wise AND unit0 everywhere, but if ^^^^is 1, the bit-wise AND unit 808 will output its input 806. The unit 810 extends the sign bit so that the resultfrom 12 bits to 13 bits.

[0041] Likewise, bit ^^^^ 804 will be used to selectively zero out the output of the bit wise AND unit 807. The output of the unit 807 will be shifted one step to the left in the unit 809, effectively multiplying the result by 2, to produce a 13-bit output. The output of the units 809 and 810 will be added together using the adder 811. The sign bit ^^^^^^^ 802 will be used toconditionally negate the output of the adder 811; if ^^^^^^^ is zero, no negation will happen, whereas if ^^^^^^^ is one, the output of the negator 812 will be the negative of the output of 811.

[0042] Finally, a variable bit shifter 813 will shift the result -- the output of the negator 812 --bi (801) the left. The output 814 of the shifter 813 will be equal to ൫^^^ ൈ ^^^^^^, ^^^൯ ≪^^^ which is the same as ^^^ ൈ ^^^^^^,^^^. In the hardware implementation shown in FIG. 8,significantly fewer gates (smaller silicon surface area) than a general multiplication between a 9-bit coefficient and an 12-bit number are used.

[0043] By limiting the value of each ALF coefficient ^^ ^^ to ^^^ ൈ 2 ^, the set of ALF coefficients^^^ಲ^ಷ,^ಾmay be expressed as follows: ^^^ಲ^ಷ,^ಾ ൌ ^^^ ∙ 2^^ where ^^^^ிcorresponds to ALF coefficient precision and ^^ெcorresponds to mantissa precision. Note that, as further explained below, the mantissa precision ^^^may be adaptive - - i.e., it can be signaled in the APS or hard-coded for a different ALF configuration.

[0044] The above expression means that the set ^^^ಲ^ಷ,^ಾis a set of coefficient values each ofwhich value is ^^ ∙ 2^. ^^ may be an integer having a value that is within a range defined bythe mantissa precision ^^ ^ெ. In some embodiments, ^^ ∈ ^െ2 ^ ^ 1, 2^^ െ 1^. ^^ may be aninteger having a value that is within a range defined by the ALF coefficient precision ^^^^ி. Insome embodiments, ^^ ∈ ^0,^^^^ி െ 1^. Like ^^, the value of ^^ ∙ 2^ may set to limited within arange defined the ALF coefficient precision ^^ ^ ^^^ி. For example, 2 ∈ ^െ2 ^ైూି^, 2^^ైూି^^.

[0045] For example, in case ^^^^ி ൌ 6 and ^^ெ ൌ 1, the coefficient set ^^^,^ ൌ^െ32,െ16,െ8,െ4,െ2,െ1,0,1,2,4,8,16,32^. In another example, in case ^^^^ி ൌ 7 and^^ெ ൌ 2, the coefficient set ^^^,ଶ ൌ^െ64,െ48,െ32,െ24,െ16,െ12,െ8,െ6,െ4,െ3,െ2,െ1,0,1,2,3,4,6,8,12,16,24,32,48,64^.

[0046] For each configuration of ALF, the range of allowed values of the mantissa precisionmay be hardcoded. For example, for chroma component, it may be enough to utilize ^^^ ൌ 1but, for luma component, it may require more bits for precision. For example, for lumacomponents, ^^^ ൌ 2 or ^^^ ∈ ^1,2^. In case the value of ^^^ is selected from the set of ^1,2^,1-bit in APS of a bitstream may indicate which one of the values is used as the value of the mantissa precision.

[0047] In some embodiments, a set of prefix-free binary codes, for example, taken from a family of Huffman codes, may be constructed to cover a full range of acceptable configurations of ALF. This set of codes may be used to adaptively signal ALF coefficients. The set of binary codes may be defined by the following tuple: (luma / chroma, ^^^^^, ^^^, coefficients group, filter cluster), which in this particular example is a 5-tuple, meaning that the tuple consists of 5 elements.

[0048] The first element “luma / chroma” may indicate whether the set is for a luma component or a chroma component. The second element ^^^^^is the ALF coefficient precision, and the third element ^^^is the mantissa precision. The fourth element “coefficient group” may identify the group to which the coefficients belong. In some embodiments, the coefficients may be split into various groups in the same way as it is done in ECM-13.0. In some embodiments, coefficients 0-33 may belong to group 0 and coefficients 34-39 may belong to group 1. The fifth element “filter cluster” is an additional adaptation parameter. The cluster number may be signaled (e.g., from the encoder to the decoder) if the number of allowed clusters for (luma / chroma, ^^^^^, ^^^, coefficients group) is more than 1.

[0049] Huffman code is completely defined by the alphabet of symbols (e.g., allowed ALF coefficient values) and the probabilities or, equivalently, frequencies of the alphabet symbols.For example, for ^^^,^ ൌ ^െ32,െ16,െ8,െ4,െ2,െ1,0,1,2,4,8,16,32^ one can constructHuffman code based on the array of frequencies^ 0,0,51,1077,4368,4739,11316,3863,5710,2169,224,6,0, ^, where the frequency of െ32 is0, of െ16 is 0, of െ8 is 51, of െ4 is 1077 and so on. The full family of Huffman codes for thevideo coding system with hard-coded ^^^^^ ∈ ^6, 7, 8, 9^ , ^^^ ∈ ^1, 2^ for luma; ^^^^^ ൌ 8 ,^^^ ൌ 1 for chroma; 2 coefficient groups; 1 filter cluster can be represented, for example, bythe following arrays of frequencies (luma, 6, 1, 0, 0) : { 0, 0, 51, 1077, 4368, 4739, 11316, 3863, 5710, 2169, 224, 6, 0, }, (luma, 6, 1, 1, 0) : { 6, 25, 135, 406, 581, 642, 957, 525, 1385, 1002, 507, 375, 12, }, (luma, 6, 2, 0, 0) : { 0, 0, 1, 1, 59, 327, 1124, 1875, 3422, 4408, 12492, 4995, 3705, 2200, 1558, 591, 167, 24, 4, 0, 0, }, (luma, 6, 2, 1, 0) : { 5, 8, 11, 27, 105, 219, 251, 336, 440, 466, 1181, 669, 653, 562, 502, 503, 480, 396, 253, 95, 10, },(luma, 7, 1, 0, 0) : { 0, 0, 54, 893, 3666, 4760, 3607, 6709, 3803, 4400, 4441, 2308, 371, 2, 0, }, (luma, 7, 1, 1, 0) : { 16, 33, 194, 467, 556, 449, 336, 525, 400, 639, 1065, 1139, 680, 262, 32, (luma, 7, 2, 0, 0) : { 0, 0, 0, 28, 70, 258, 950, 1722, 3390, 2442, 2887, 3477, 6757, 3731, 2976, 3383, 2517, 2018, 802, 263, 49, 2, 0, 0, }, (luma, 7, 2, 1, 0) : { 4, 7, 18, 68, 174, 223, 383, 362, 294, 289, 305, 272, 531, 352, 437, 414, 651, 689, 727, 510, 364, 280, 176, 54, 23, }, (luma, 8, 1, 0, 0) : { 25, 3, 110, 831, 3103, 3688, 3516, 3064, 7035, 3529, 3629, 3873, 3334, 1636, 344, 6, 25, }, (luma, 8, 1, 1, 0) : { 19, 68, 237, 363, 474, 376, 379, 256, 448, 298, 366, 633, 886, 990, 1000, 590, 45, }, (luma, 8, 2, 0, 0) : { 0, 0, 3, 9, 53, 152, 699, 1414, 3055, 3117, 4172, 3128, 3742, 4496, 5012, 4399, 3726, 3190, 4363, 3597, 3922, 2383, 1760, 739, 236, 24, 2, 0, 0, }, (luma, 8, 2, 1, 0) : { 12, 12, 36, 81, 192, 259, 370, 385, 394, 354, 280, 224, 228, 257, 341, 294, 339, 351, 553, 538, 737, 688, 856, 745, 736, 523, 416, 158, 35, }, (luma, 9, 1, 0, 0) : { 5, 2, 64, 933, 3190, 3915, 3730, 3305, 3182, 5676, 3437, 3305, 3892, 4593, 3678, 1653, 206, 6, 5, }, (luma, 9, 1, 1, 0) : { 28, 54, 199, 382, 500, 368, 313, 214, 167, 226, 171, 254, 460, 800, 1008, 1199, 1338, 729, 66, }, (luma, 9, 2, 0, 0) : { 0, 0, 7, 14, 63, 161, 756, 1530, 3224, 3173, 4383, 3541, 4208, 3432, 4037, 5090, 5555, 5000, 3991, 3570, 4443, 4063, 5211, 3875, 4276, 2432, 1608, 512, 149, 20, 6, 0, 0, }, (luma, 9, 2, 1, 0) : { 6, 10, 46, 98, 230, 288, 383, 418, 479, 381, 368, 257, 206, 171, 160, 236, 289, 208, 221, 263, 403, 469, 730, 656, 958, 849, 1067, 1055, 1240, 996, 693, 226, 39, }, (chroma, 8, 0, 0, 0) : { 0, 0, 10, 202, 1088, 2317, 2867, 3038, 3968, 3001, 2635, 2262, 1810, 700, 94, 0, 0, }, (chroma, 8, 0, 1, 0) : { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 14, 49, 260, 528, 137, }.

[0050] In a summary, when encoder 102 encodes a video, encoder 102 generates values of ALF coefficients to use for video encoding. Then encoder 102 determines a combination of a tuple and a codeword which collectively indicates each of the values of the ALF coefficients. After obtaining the tuple and the codewords, encoder 102 transmits a video bitstream including the encoded video, the tuple, and the codewords to decoder 104. Upon receiving the video bitstream, decoder 104 derives the values of ALF coefficients based on the tuple and the codewords, and decodes the encoded video using the derived values of ALF coefficients.

[0051] FIG. 4 shows a process 400 for performing a video coding according to some embodiments. Process 400 may begin with step s402. Step s402 comprises determining an allowed set of coefficient values of adaptive loop filter, ALF, wherein each coefficient valueincluded in the allowed set of ALF coefficient values is ^^ ∙ 2^ where ^^ is a mantissa, ^^ is anexponent. Step s404 comprises determining a plurality of ALF coefficient values to use for the video coding, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values. Step s406 comprises performing the video coding using the plurality of ALF coefficient values.

[0052] In some embodiments, the mantissa ^^ is an integer that is limited to be within a rangebetween െ2^ಾ ^ 1 and 2^ಾ െ 1 where ^^ெ is an integer. That is, in this embodiment, m isgreater than or equal to െ2^ಾ ^ 1 and less than or equal to 2^ಾ െ 1.

[0053] In some embodiments, ^^ெ ൌ 2 for luma components, and ^^ெ ൌ 1 for chromacomponents.

[0054] In some embodiments, ^^ெ ∈ ^1,2^ for luma components, and ^^ெ ൌ 1 for chromacomponents.

[0055] In some embodiments, the exponent ^^ is an integer that is limited to be within a rangebetween 0 and ^^^^ி െ 1 where ^^^^ி is an integer. That is, in this embodiment, b is greater thanor equal to 0 and less than or equal to BALF-1.

[0056] In some embodiments, the value of ^^ ∙ 2^ is limited to be within a range betweenെ2^ಲ^ಷି^ and 2^ಲ^ಷି^ . That is, in this embodiment, ^^ ∙ 2^ is greater than or equal toെ2^ಲ^ಷି^and less than or equal to 2^ಲ^ಷି^.

[0057] In some embodiments, process 400 is performed by a video encoder, and performing the video coding comprises encoding a video using the plurality of ALF coefficient values.

[0058] In some embodiments, process 400 further comprises transmitting a video bitstream, and the video bitstream includes a set of codewords and a tuple.

[0059] In some embodiments, the tuple indicates the allowed set of ALF coefficient values, a combination of a codeword included in the set of codewords and one or more parameters included in the tuple collectively indicates a certain ALF coefficient value to use for the video coding, and the tuple comprises a plurality of parameters each of which indicates one of: (1) whether the tuple is for a luma sample or a chroma sample, (2) ^^^^ி, (3) ^^ெ, (4) a coefficient group, and (5) a filter cluster.

[0060] In some embodiments, in case the tuple indicates that the tuple is for a chroma sample, a combination of (i) a codeword included in the set of codewords and (ii) a parameter of the tuple, which indicates a coefficient group collectively indicates a certain ALF coefficient value to use for the video coding.

[0061] In some embodiments, in case the tuple indicates that the tuple is for a luma sample, a combination of (i) a codeword included in the set of codewords, (ii) a parameter of the tuple, which indicates ^^^^ி, (iii) a parameter of the tuple, which indicates ^^ெ, and (iv) a parameter of the tuple, which indicates a coefficient group collectively indicates a certain ALF coefficient value to use for the video coding.

[0062] In some embodiments, process 400 is performed by a video decoder, determining the plurality of ALF coefficient values to use for the video coding comprises: receiving from a video encoder a video bitstream including a set of codewords and a tuple, wherein the tuple indicates the allowed set of ALF coefficient values; and determining the plurality of ALF coefficient values based on the set of codewords and the tuple, and performing the video coding comprises decoding an encoded video using the plurality of ALF coefficient values.

[0063] In some embodiments, a combination of a codeword included in the set of codewords and one or more parameters included in the tuple collectively indicates a certain ALF coefficient value to use for the video coding, and the tuple comprises a plurality of parameters each of which indicates one of: (1) whether the tuple is for a luma sample or a chroma sample, (2) ^^^^ி, (3) ^^ெ, (4) a coefficient group, and (5) a filter cluster.

[0064] In some embodiments, in case the tuple indicates that the tuple is for a chroma sample, determining the plurality of ALF coefficient values comprises deriving a certain ALFcoefficient value based on a combination of (i) a codeword included in the set of codewords and (ii) a parameter of the tuple, which indicates a coefficient group.

[0065] In some embodiments, in case the tuple indicates that the tuple is for a luma sample, determining the plurality of ALF coefficient values comprises deriving a certain ALF coefficient value based on a combination of (i) a codeword included in the set of codewords, (ii) a parameter of the tuple, which indicates ^^^^ி, (iii) a parameter of the tuple, which indicates ^^ெ, and (iv) a parameter of the tuple, which indicates a coefficient group.

[0066] In some embodiments, the set of codewords is determined using Huffman coding.

[0067] FIG. 5 is a hardware block diagram of device 103, according to some embodiments. As shown in FIG.5, device 103 may comprise: processing circuitry (PC) 502, which includes one or more processors (P) 555 (e.g., one or more general purpose microprocessors and / or one or more other processors, such as an application specific integrated circuit (ASIC), field- programmable gate arrays (FPGAs), and the like), which processors may be co-located in a single housing or in a single data center or may be geographically distributed (i.e., encoder device 103 may be a distributed computing apparatus comprising multiple computers or may be a monolithic device comprising a single computer); at least one network interface 548 (e.g., a physical interface or air interface) comprising a transmitter (Tx) 545 and a receiver (Rx) 547 for enabling device 103 to transmit data to and receive data from other nodes connected to a network 110 (e.g., an Internet Protocol (IP) network) to which network interface 548 is connected (physically or wirelessly) (e.g., network interface 548 may be coupled to an antenna arrangement comprising one or more antennas for enabling encoder device 103 to wirelessly transmit / receive data); and a storage unit (a.k.a., “data storage system”) 508, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 502 includes a programmable processor, a computer readable storage medium (CRSM) 542 may be provided. CRSM 542 may store a computer program (CP) 543 comprising computer readable instructions (CRI) 544. CRSM 542 may be a non- transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the CRI 544 of computer program 543 is configured such that when executed by PC 502, the CRI causes encoder device 103 to perform steps described herein (e.g., steps described herein with reference to the flow charts). In other embodiments, encoder device 103 may be configured to perform steps described herein without the need for code. That is, forexample, PC 502 may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and / or software.

[0068] Summary of Embodiments

[0069] A1. A method for performing a video coding, the method comprising: determining an allowed set of coefficient values of adaptive loop filter, ALF, wherein each coefficient valueincluded in the allowed set of ALF coefficient values is ^^ ∙ 2^ where ^^ is a mantissa, ^^ is anexponent; and determining a plurality of ALF coefficient values to use for the video coding, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values; and performing the video coding using the plurality of ALF coefficient values.

[0070] A2. The method of embodiment A1, wherein the mantissa ^^ is an integer that islimited to be within a range between െ2^ಾ ^ 1 and 2^ಾ െ 1 where ^^ெ is an integer.

[0071] A3. The method of embodiment A2, wherein ^^ெ ൌ 2 for luma components, and^^ெ ൌ 1 for chroma components.

[0072] A4. The method of embodiment A2, wherein ^^ெ ∈ ^1,2^ for luma components, and^^ெ ൌ 1 for chroma components.

[0073] A5. The method of any one of embodiments A2-A4, wherein the exponent ^^ is aninteger that is limited to be within a range between 0 and ^^^^ி െ 1 where ^^^^ி is an integer.

[0074] A6. The method of embodiment A5, wherein the value of ^^ ∙ 2^ is limited to bewithin a range between െ2^ಲ^ಷି^and 2^ಲ^ಷି^.

[0075] A7. The method of any one of embodiments A1-A6, wherein the method is performed by a video encoder, and performing the video coding comprises encoding a video using the plurality of ALF coefficient values.

[0076] A8. The method of embodiment A7 when embodiment A7 depends on embodiment A5 or A6, wherein the method further comprises transmitting a video bitstream, and the video bitstream includes a set of codewords and a tuple.

[0077] A9. The method of embodiment A8, wherein the tuple indicates the allowed set of ALF coefficient values, a combination of a codeword included in the set of codewords and one or more parameters included in the tuple collectively indicates a certain ALF coefficient value to use for the video coding, and the tuple comprises a plurality of parameters each ofwhich indicates one of: (1) whether the tuple is for a luma sample or a chroma sample, (2) ^^^^ி, (3) ^^ெ, (4) a coefficient group, and (5) a filter cluster.

[0078] A10. The method of embodiment A9, wherein in case the tuple indicates that the tuple is for a chroma sample, a combination of (i) a codeword included in the set of codewords and (ii) a parameter of the tuple, which indicates a coefficient group collectively indicates a certain ALF coefficient value to use for the video coding.

[0079] A11. The method of embodiment A9, wherein in case the tuple indicates that the tuple is for a luma sample, a combination of (i) a codeword included in the set of codewords, (ii) a parameter of the tuple, which indicates ^^^^ி, (iii) a parameter of the tuple, which indicates ^^ெ, and (iv) a parameter of the tuple, which indicates a coefficient group collectively indicates a certain ALF coefficient value to use for the video coding.

[0080] A13. The method of any one of embodiments A1-A6, wherein the method is performed by a video decoder, determining the plurality of ALF coefficient values to use for the video coding comprises: receiving from a video encoder a video bitstream including a set of codewords and a tuple, wherein the tuple indicates the allowed set of ALF coefficient values; and determining the plurality of ALF coefficient values based on the set of codewords and the tuple, and performing the video coding comprises decoding an encoded video using the plurality of ALF coefficient values.

[0081] A14. The method of embodiment A13, wherein a combination of a codeword included in the set of codewords and one or more parameters included in the tuple collectively indicates a certain ALF coefficient value to use for the video coding, and the tuple comprises a plurality of parameters each of which indicates one of: (1) whether the tuple is for a luma sample or a chroma sample, (2) ^^^^ி, (3) ^^ெ, (4) a coefficient group, and (5) a filter cluster.

[0082] A15. The method of embodiment A14, wherein in case the tuple indicates that the tuple is for a chroma sample, determining the plurality of ALF coefficient values comprises deriving a certain ALF coefficient value based on a combination of (i) a codeword included in the set of codewords and (ii) a parameter of the tuple, which indicates a coefficient group.

[0083] A16. The method of embodiment A14, wherein in case the tuple indicates that the tuple is for a luma sample, determining the plurality of ALF coefficient values comprises deriving a certain ALF coefficient value based on a combination of (i) a codeword includedin the set of codewords, (ii) a parameter of the tuple, which indicates ^^^^ி, (iii) a parameter of the tuple, which indicates ^^ெ, and (iv) a parameter of the tuple, which indicates a coefficient group.

[0084] A17. The method of any one of embodiment A8-A16, wherein the set of codewords is determined using Huffman coding.

[0085] B1. A computer program comprising instructions which when executed by processing circuitry cause the processing circuitry to perform the method of any one of embodiments A1-A17.

[0086] B2. A carrier containing the computer program of embodiment B1, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

[0087] C1. An apparatus for performing a video coding, the apparatus being configured to: determine an allowed set of coefficient values of adaptive loop filter, ALF, wherein eachcoefficient value included in the allowed set of ALF coefficient values is ^^ ∙ 2^ where ^^ is amantissa, ^^ is an exponent; and determine a plurality of ALF coefficient values to use for the video coding, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values; and perform the video coding using the plurality of ALF coefficient values.

[0088] C2. The apparatus of embodiment C1, wherein the apparatus is configured to perform the method of any one of embodiments A2-A17.

[0089] D1. An apparatus comprising: processing circuitry; and a memory, said memory containing instructions executable by said processing circuitry, whereby the apparatus is operative to perform the method of any one of embodiments A1-A17.

[0090] Abbreviations ITU-T The International Telecommunication Union Telecommunication Standardization Sector VCEG Video Coding Experts Group ISO / IEC International Organization for Standardization / International Electrotechnical CommissionMPEG Moving Picture Experts Group RD Rate Distortion AI All Intra RA Random Access LDB Low-Delay CPU Central Processing Unit

[0091] Additional Disclosure

[0092] 1. Intro

[0093] This disclosure proposes to restrict allowed values of ALF coefficients to the elementsof set ^^^ ⋅ 2^^, where ^^ ∈ ^െ1, 0, 1^ or ^^ ∈ ^െ3,െ2,െ1, 0, 1, 2, 3^, and ^^ is a non-negativeinteger. The coefficients are derived using Simulated Annealing approach and signaled into APS using Huffman coding.

[0094] The proposed algorithm is implemented on the top of ECM-13.0. According to simulation results under common test conditions (CTC), BD-rate PSNR impact is reported to be -0.03% / -0.08% / -0.11% for AI / RA / LDB configuration for Luma. The average encoding run time is reported to be 100.3% / 100.0% / 100.5% for AI / RA / LDB configuration. The average decoding run time is reported to be 100.6% / 100.9% / 100.8% for AI / RA / LDB configuration. The proposed solution eliminates multiplications by 3 or more-bit numbers when Adaptive Loop Filter (ALF) is applied. The hardware complexity is claimed to be reduced.

[0095] Adaptive loop filtering may be applied in video coding. ALF is designed to reduce quantization errors of the output pictures as well as of the reference pictures used for inter- prediction (see reference [1]). ALF is included into Versatile Video Coding (VVC), also known as H.266, developed by the Joint Video Experts Team (JVET). ALF is also included into ECM-13.0 (reference [2]) as one of the stages in loop filtering as showed in FIG.6. The proposed algorithm provides (hardware) complexity reduction of ALF model in ECM-13.0 as well as provides bitrate savings.

[0096] 2. Background

[0097] Let ^^^^^,^^^ be an output sample of the decoding stage preceding ALF, i.e. sample at position ^^^,^^^ outputted by combination of SAO, CC-SAO and BIF (see FIG.6), the ALFoutput ^^^^^^,^^^ is defined as:^ ^^^^^^,^^^ ൌ ^^^^^,^^^ ^ ^^^^^^^^^^,^^^,^1^ ^ୀ^ where ^^^is an ^^-th ^^^,^^^. For luma samples, ECM-13.0 [2] utilizeswith 40 taps, where the first 20 taps (taps 0-19) are samples in the neighboring pixels (spatial taps) and the other taps have more complex nature and include taps based on the output of the fixed filters, taps based on the input for deblocking filter (DBF), taps based on the reconstructed residuals.

[0098] Coefficients ^^^are derived by the encoder. The derivation used in ECM-13.0 can be split into several parts. Firstly, the optimal real-valued coefficients are obtained by solving Wiener-Hopf equations to minimize the mean square error (MSE) between the original picture and the filtered picture. Secondly, the coefficients are quantized, namely, multiplied by a constant 2^^ైూି^, rounded to integers and clipped into the interval ^െ2^^ైూି^^1, 2^^ైూି^ െ 1^. Parameter ^^^^^ is called a precision. Finally, a descent-based procedure islaunched: at each iteration, each coefficient is tried to be increased or decreased by 1, the best (by MSE) change of a coefficient is accepted, the iterations continue while there is an improvement of MSE. The actual filtering is done by formula (1) in integers and right bitshifting is utilized to properly scale the ALF offset ∑^ ^ୀ^ ^^^^^^^^^,^^^.

[0099] Quantized coefficients ^^^are signaled in a special header called adaptation parameter set (APS). In ECM-13.0, the coefficients are split into two groups (the group number is defined by index ^^, and forfor luma, the group number equals the results ofcomparison ^^ ^ 34) and different Golomb-Rice codes for different groups are used to convertthe coefficients into the APS bitstream. For each group, four parameters of Golomb-Rice coding are iterated, and the best (by total number of required bits) choice is signaled in APS.

[0100] In ECM-13.0, the precision parameter ^^^^^is fixed to be 8 for chroma component but is adaptive to luma component. Namely, the values 6, 7, 8 and 9 are iterated, the best (by RD-cost) choice is signaled in APS.

[0101] 3. Set of Allowed Coefficient Values

[0102] Let ^^^^^be ALF coefficients precision and denote the mantissa precision by ^^^. In the proposed solution, the quantized ALF coefficients are limited to the elements of the set ^^^^ైూ,^^ ൌ ^^^ ⋅ 2^^, where ^^ and b are integers, ^^ ∈ ^െ2^^ ^ 1, 2^^ െ 1^ and ^^ ∈^0,^^ െ 1^, moreo ^ ^^ైూି^ ^^ైూି^^^ி ver, ^^ ⋅ 2 belongs to the interval ^െ2 , 2 ^. For example,^^^,ଶ ൌ^െ64,െ48,െ32,െ24,െ16,െ12,െ8,െ6,െ4,െ3,െ2,െ1,0,1,2,3,4,6,8,12,16,24,32,48,64^.

[0103] For each configuration, the range of allowed mantissa precisions is hardcoded. For chroma component, mantissa precision ^^^ ൌ 1 is utilized, but for luma component, mantissaprecision is configured as ^^^ ∈ ^1,2^. For luma, encoder signals 1 bit in APS to indicatewhich mantissa precision is used. The choice of ^^^value for luma is based on RD-cost.

[0104] The purpose of such limitation of the coefficient values is to decrease (hardware) complexity of ALF implementation. Similar ideas were presented to JVET, for example, in [3-4], however, resulting in coding loses. In this contribution, opposite to [3-4], bit-rate savings are provided.

[0105] 4. Signaling of Coefficients

[0106] The family of prefix-free binary codes is constructed to cover the full range of acceptable configurations defined by the following tuple: (luma / chroma, ^^^^^, ^^^, coefficients group). Coefficients are split into the groups in the same way as it is done in ECM-13.0: for example, for luma component, coefficients 0-33 belong to group 0 and coefficients 34-39 belong to group 1.

[0107] The constructed binary codes are based on Huffman coding. Huffman code is completely defined by the alphabet of symbols (e.g., allowed ALF coefficient values) and the probabilities or, equivalently, frequencies of the alphabet symbols. For example, in the proposed algorithm, for ^^^,^ ൌ ^െ32,െ16,െ8,െ4,െ2,െ1,0,1,2,4,8,16,32^ Huffman codebased on the array of frequencies ^0, 0, 51, 1077, 4368, 4739, 11316, 3863, 5710, 2169, 224, 6, 0, ^ is built, where thefrequency of െ32 is 0, of െ16 is 0, of െ8 is 51, of െ4 is 1077 and so on. In total, 18 Huffman codes (16 for luma and 2 for chroma) are included into proposal.

[0108] Note that an efficient Huffman coding might be unfeasible for ECM-13.0 without changes introduced in section 3 since the number of allowed coefficient values is much higher than in the proposal.

[0109] 5 Coefficient Derivation

[0110] To improve the search of the ALF coefficients, especially in the case when ECM-13.0 is combined with the coefficient values limitation proposed in section 3, Simulated Annealing based search is introduced. A practical implementation of ALF, such as the implementation in ECM-13.0, works with quantized coefficients and moreover limited with a certain range of values, that is why solving Wiener-Hopf equations to find the optimal real-valued coefficients does not provide a reference for optimal quantized coefficients. There is a variety of discrete optimization methods to approximate the optimal coefficients. In ECM-13.0, deterministic descent approach is used, however, probabilistic methods such as Simulated Annealing demonstrate advantage in search of a solution in reasonable time.

[0111] Let ^^ ൌ ^^^^, ^^^, … , ^^^^ denote the filter coefficients and ^^^^^^ is an objective function.Simulated annealing approach is used to minimize ^^^^^^, and ^^ is a subject for optimization. It starts from the initial solution ^^^and continues for a number of iterations. At ^^-th iteration, a random change to the input solution ^^^is applied, resulting in a temporary solution ^^^ᇱ. For example, it may be done in such a way: 3 random coefficients ^^^భ , ^^^మ , ^^^య are chosen, andeach of them is randomly increased or decreased by 1, i.e. ^^ᇱ^ೕ ൌ ^^^ೕ ^ rand^^െ1,1^^. Then,objective function ^^^^^^^ is calculated and one needs to decide whether to accept this random change or not. For example, the acceptance probability can be calculated as follows: ^^൫^^,^^′1 ^ ,^^^൯ ൌ1ାexp^ି^^ା1^2⋅^ி^^^^^ିி^൫^^′൯^^, solutions which decreases the value of objectivefirst iterations (in particular, ^^ ൌ 0^, solutions whichincreases the value of objective function might be accepted. Then, random generator is used to make a decision: ^^ ൌ ^^^^ᇱwith probability ^^^^^,^^^,^^^ᇱ^, ^ା^ ^^^ , otherwise.

[0112] In state-of-the-art procedure from ECM-13.0, the objective function forthe deterministic descent MSE between the original picture and the filtered picture. In the proposed solution, ^^^^^^^ ൌ MSE^^^^ ^ ^^^ ⋅ Rate^^^^, where MSE^^^^ is the same MSEbetween the original picture and the filtered picture, Rate^^^^ is the number of bits required to be signaled in APS. ^^^may be defined in different ways, for example,^^ ൌ ^^ ⋅2 ^^1ା 1 െ 1^,10^^శ1^ (^^ → ∞^, the objective function tends to RD-cost, but at the firstfunction is closer to MSE. If the number of iterations exceeds a preconfigured limit ^^^^௫or if there are no newly accepted solutions for a while (for example, for 8 ⋅ ^^^ ^ 1^ iterations in a row), the Simulated Annealing procedure is stopped, and thelatest solution is taken as output of the optimization procedure. The process of the simulated annealing is depicted in FIG.9.

[0113] The simulated annealing process may be launched several times with different initial solutions. For example, one can start from the solution obtained by the state-of-the-art optimization method from ECM-13.0 and run the simulated annealing 10 times, moreover, ^^- th launch takes the output of ^^^ െ 1^-th launch as the input.

[0114] Random number generator is extensively utilized in the simulated annealing approach. There is a variety of pseudo-random generators that can be used, for example, Xorshift. To provide an encoder reproducibility, one may use the same hardcoded pseudo-random number generator with a fixed seed for each simulated annealing launch.

[0115] 6. ALF Complexity Reduction

[0116] In general, hardware implementation of multiplication of ^^^-bit integer and ^^ଶ-bit integer takes ^^^^^^^^ଶ^logical gates, however if the values of ^^ଶ-bit integers are limited to pure powers of two, then multiplication becomes left bit shifting which can be implemented using the barrel shifter and resulting in complexity of ^^^^^^logଶ^^ଶ^ logical gates.

[0117] In ECM-13.0, the maximal bit width of ALF coefficient ^^^is 9 bits and suppose that the bit width of the tap ^^^^^^,^^^ is 12 bits. The complexity of formula (1) is roughly estimated to be 28K logical gates (per pixel). In the proposed algorithm, ^^ ^^ ൌ ^^^ ⋅ 2 ^, and socalculating ^^^^^^^^^, ^^^ is same as calculating ^^^ ^^ ⋅ 2 ^^ ⋅ ^^^^^^,^^^ ൌ ^^^^ ⋅ ^^^^^^,^^^^ ⋅ 2^^ . Themultiplication by 2^^ is simply left bit shifting, moreover, since െ3 ^ ^^^ ^ 3 in theproposed solution, all multiplications can be2-bit positive integer can be implemented with one summation and AND gates. The complexity of formula (1) is roughly estimated to be 16K logical gates, i.e.57% of the baseline complexity.

[0118] 7. Simulation Results

[0119] The proposal is implemented on top of ECM-13.0 and evaluated under the CTC. In the tables below, all reported runtimes (both EncT and DecT) are reliable.

[0120] The following tables show the results for PSNR-based BD-rate and encoding / decoding run time when all normative and encoder changes from Sections 3-5 are applied. All Intra Main 10Normative & encoder changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 -0.03% 0.00% 0.04% 100.7% 100.8% 102.3% 100.1% Class A2 -0.01% -0.02% 0.09% 100.4% 100.5% 102.7% 99.9% Class B -0.02% 0.03% -0.07% 100.0% 100.7% 102.9% 100.2% Class C -0.03% 0.06% 0.04% 100.0% 100.4% 99.9% 100.0% Class E -0.05% 0.10% -0.07% 100.7% 100.7% 100.5% 100.0% Overall -0.03% 0.04% 0.00% 100.3% 100.6% 101.7% 100.1% Class D -0.07% -0.27% -0.40% 99.5% 99.7% 100.3% 100.0% Class F -0.08% -0.11% -0.16% 100.4% 100.1% 101.9% 100.0% Class TGM -0.02% 0.01% 0.02% 100.7% 100.5% 98.7% 100.0% Random Access Main 10 Normative & encoder changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 -0.10% -0.03% -0.15% 99.6% 101.4% 100.0% 99.7% Class A2 -0.10% -0.11% -0.04% 99.7% 101.2% 99.9% 100.1% Class B -0.11% -0.03% -0.22% 100.1% 101.1% 100.0% 100.1% Class C -0.02% -0.14% -0.15% 100.5% 100.0% 100.3% 100.0% Class EOverall -0.08% -0.08% -0.15% 100.0% 100.9% 100.1% 100.0% Class D -0.10% 0.00% -0.16% 100.6% 100.2% 100.9% 100.0% Class F -0.11% -0.07% -0.24% 100.8% 101.0% 100.1% 100.4% Class TGM 0.06% 0.10% 0.05% 100.3% 99.6% 100.2% 101.1% Low delay B Main 10Normative & encoder changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeakClass A1 Class A2Class B -0.20% -0.75% -0.66% 100.1% 101.0% 100.2% 100.2% Class C -0.04% -0.55% -0.54% 100.7% 101.0% 100.5% 100.0% Class E -0.06% -0.53% -0.12% 100.9% 100.4% 101.1% 100.2% Overall -0.11% -0.63% -0.49% 100.5% 100.8% 100.5% 100.1% Class D -0.17% -1.02% -1.11% 99.2% 100.8% 101.0% 100.1% Class F -0.17% -0.64% -0.22% 100.3% 101.4% 100.0% 100.1% Class TGM -0.04% 0.13% 0.19% 100.4% 100.3% 100.6% 103.1%

[0121] The following tables show the results for PSNR-based BD-rate and encoding / decoding run time when only normative changes from Sections 3-4 are applied. All Intra Main 10Only normative changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 -0.02% 0.01% 0.06% 100.4% 100.7% 100.9% 100.1% Class A2 0.02% 0.04% 0.19% 100.2% 99.9% 100.5% 100.0% Class B 0.00% 0.07% 0.05% 100.0% 100.6% 105.6% 99.9% Class C -0.01% 0.11% 0.08% 99.7% 99.7% 100.0% 100.0% Class E -0.01% 0.15% -0.08% 100.4% 100.2% 98.4% 100.0% Overall -0.01% 0.08% 0.06% 100.1% 100.2% 101.5% 100.0% Class D -0.05% -0.23% -0.37% 99.8% 100.2% 100.5% 100.0% Class F -0.05% -0.07% -0.12% 100.3% 100.4% 101.9% 100.0% Class TGM 0.01% 0.07% 0.10% 100.1% 100.4% 99.9% 100.1% Random Access Main 10Only normative changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 -0.06% 0.00% -0.26% 99.7% 101.2% 100.2% 99.7% Class A2 -0.05% 0.03% 0.13% 99.8% 101.1% 100.4% 100.1% Class B -0.05% 0.05% -0.07% 99.9% 101.0% 100.3% 100.3% Class C -0.01% -0.09% 0.04% 100.0% 100.8% 100.3% 100.1% Class EOverall -0.04% 0.00% -0.04% 99.9% 101.0% 100.3% 100.1%Class D -0.03% -0.04% -0.19% 100.1% 100.3% 100.6% 100.0% Class F -0.03% -0.13% -0.17% 100.4% 100.1% Class TGM -0.09% 0.03% 0.07% 100.0% 99.7% 99.8% 99.9% Low delay B Main 10Only normative changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 Class A2Class B -0.11% -0.63% -0.49% 99.9% 101.7% 100.2% 100.2% Class C -0.02% -0.31% -0.61% 99.6% 100.9% 101.0% 100.0% Class E -0.01% 0.69% -0.01% 100.1% 99.2% 101.5% 100.2% Overall -0.05% -0.19% -0.41% 99.9% 100.8% 100.8% 100.1% Class D -0.07% -0.71% -0.27% 100.2% 100.2% 101.4% 100.2% Class F 0.02% 0.36% 0.02% 99.7% 102.2% 99.2% 100.2% Class TGM -0.03% 0.10% 0.14% 100.2% 100.9%

[0122] The following tables show the results for PSNR-based BD-rate and encoding / decoding run time when only encoder optimizations from Section 5 are applied All Intra Main 10Only encoder changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 -0.01% 0.00% -0.02% 100.8% 101.0% 101.2% 99.9% Class A2 -0.01% -0.05% 0.00% 100.1% 101.0% 97.1% 100.1% Class B -0.01% 0.01% -0.05% 99.9% 100.4% 101.6% 100.0% Class C -0.01% 0.00% -0.01% 100.0% 100.0% 99.9% 100.0% Class E -0.02% 0.00% -0.03% 100.4% 100.8% 101.5% 100.0% Overall -0.01% 0.00% -0.02% 100.2% 100.6% 100.4% 100.0% Class D -0.02% -0.01% -0.01% 100.4% 99.9% 100.7% 100.0% Class F -0.02% -0.01% -0.02% 99.9% 99.9% 99.1% 100.0% Class TGM -0.02% -0.03% -0.02% 100.5% 100.4% 100.2% 100.0% Random Access Main 10Only encoder changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 0.01% 0.04% -0.06% 100.0% 99.8% 100.2% 99.9% Class A2 -0.02% -0.09% 0.07% 100.1% 100.5% 100.3% 99.9% Class B -0.01% 0.03% -0.22% 100.4% 100.0% 99.9% 100.1% Class C -0.02% -0.04% -0.01% 100.4% 99.5% 100.6% 100.0% Class E Overall -0.01% -0.01% -0.07% 100.2% 99.9% 100.2% 100.0% Class D -0.05% -0.06% 0.04% 99.8% 99.5% 100.9% 100.0% Class F 0.06% 0.10% -0.05% 100.5% 100.4% 101.0% 100.1% Class TGM 0.02% -0.02% 0.00% 100.4% 99.5% 99.1% 100.0% Low delay B Main 10 Only encoder changes over ECM-13.0 Y U V EncT DecT EncVmPeak DecVmPeak Class A1 Class A2Class B -0.06% -0.36% -0.37% 100.3% 100.1% 100.2% 100.0% Class C -0.13% 0.02% -0.41% 99.7% 100.3% 100.5% 100.0% Class E -0.01% 0.58% 0.39% 100.5% 99.1% 100.3% 100.2% Overall -0.07% 0.00% -0.19% 100.2% 99.9% 100.3% 100.1% Class D -0.12% -0.56% -0.38% 99.3% 100.0% 101.2% 100.0% Class F -0.33% 0.02% -0.21% 99.6% 100.0% Class TGM -0.12% 0.01% -0.18% 100.3% 98.9%

[0123] Conclusion

[0124] While various embodiments are described herein, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

[0125] As used herein transmitting a message “to” or “toward” an intended recipient encompasses transmitting the message directly to the intended recipient or transmitting themessage indirectly to the intended recipient (i.e., one or more other nodes are used to relay the message from the source node to the intended recipient). Likewise, as used herein receiving a message “from” a sender encompasses receiving the message directly from the sender or indirectly from the sender (i.e., one or more nodes are used to relay the message from the sender to the receiving node). Further, as used herein “a” means “at least one” or “one or more.”

[0126] Additionally, while the processes described above and illustrated in the drawings are shown as a sequence of steps, this was done solely for the sake of illustration. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be re-arranged, and some steps may be performed in parallel.

[0127] List of References

[0128] 1. C. -Y. Tsai et al., “Adaptive Loop Filtering for Video Coding,” in IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 6, pp. 934-945, Dec. 2013, doi: 10.1109 / JSTSP.2013.2271974.

[0129] 2. Coman, M., et. al., “Algorithm description of Enhanced Compression Model 13 (ECM 13)”, JVET-AH2025, 34th Meeting: Rennes, FR, 17–24 April 2024.

[0130] 3. Esenlik, S., et. al., "CE2: Alf with Multiplication Replaced by Bit-Shifting (Test 2.5.1)," JVET-L0162, 12th Meeting: Macao, CN, 3–12 Oct.2018.

[0131] 4. Ström, J., et. al., "Non-CE5: Multiplication simplification for ALF and CC-ALF," JVET-Q0167, 17th Meeting: Brussels, BE, 7–17 January 2020.

Claims

CLAIMS 1. A method (400) for video processing, the method comprising: determining (s402) an allowed set of adaptive loop filter, ALF, coefficient values for an ALF, wherein each coefficient value included in the allowed set of ALF coefficient valuesis ^^ ∙ 2^ where ^^ is a mantissa, ^^ is an exponent; anddetermining (s404) a plurality of ALF coefficient values to use for the video processing, wherein each one of the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values; and performing (s406) the video processing using the plurality of ALF coefficient values.

2. The method of claim 1, wherein the mantissa ^^ is an integer that is limited to bewithin a range between െ2^ಾ ^ 1 and 2^ಾ െ 1 where ^^ெ is an integer.

3. The method of claim 2, wherein ^^ெ ൌ 2 for luma components, and^^ெ ൌ 1 for chroma components.

4. The method of claim 2, wherein ^^ெ ∈ ^1,2^ for luma components, and^^ெ ൌ 1 for chroma components.

5. The method of any one of claims 2-4, wherein the exponent ^^ is an integer that islimited to be within a range between 0 and ^^^^ி െ 1 where ^^^^ி is an integer.

6. The method of claim 5, wherein the value of ^^ ∙ 2^ is limited to be within a rangebetween െ2^ಲ^ಷି^and 2^ಲ^ಷି^.

7. The method of any one of claims 1-6, wherein the method is performed by a video encoder, and performing the video processing comprises encoding a video using the plurality of ALF coefficient values.

8. The method of claim 7 when claim 7 depends on claim 5 or 6, wherein the method further comprises generating a video bitstream, and the video bitstream includes a set of codewords and a tuple.

9. The method of claim 8, wherein the tuple indicates the allowed set of ALF coefficient values, a combination of a codeword included in the set of codewords and one or more parameters included in the tuple collectively indicates a certain ALF coefficient value to use for the video processing, and the tuple comprises a plurality of parameters each of which indicates one of: (1) whether the tuple is for a luma sample or a chroma sample, (2) ^^^^ி, (3) ^^ெ, (4) a coefficient group, and (5) a filter cluster.

10. The method of claim 9, wherein in case the tuple indicates that the tuple is for a chroma sample, a combination of (i) a codeword included in the set of codewords and (ii) a parameter of the tuple, which indicates a coefficient group, collectively indicates a certain ALF coefficient value to use for the video processing.

11. The method of claim 9, wherein in case the tuple indicates that the tuple is for a luma sample, a combination of (i) a codeword included in the set of codewords, (ii) a parameter of the tuple, which indicates ^^^^ி, (iii) a parameter of the tuple, which indicates ^^ெ, and (iv) a parameter of the tuple, which indicates a coefficient group, collectively indicates a certain ALF coefficient value to use for the video processing.

12. The method of any one of claims 1-6, wherein the method is performed by a video decoder, determining the plurality of ALF coefficient values to use for the video processing comprises:receiving from a video encoder a video bitstream including a set of codewords and a tuple, wherein the tuple indicates the allowed set of ALF coefficient values; and determining the plurality of ALF coefficient values based on the set of codewords and the tuple, and performing the video processing comprises decoding an encoded video using the plurality of ALF coefficient values.

13. The method of claim 12, wherein a combination of a codeword included in the set of codewords and one or more parameters included in the tuple collectively indicates a certain ALF coefficient value to use for the video processing, and the tuple comprises a plurality of parameters each of which indicates one of: (1) whether the tuple is for a luma sample or a chroma sample, (2) ^^^^ி, (3) ^^ெ, (4) a coefficient group, and (5) a filter cluster.

14. The method of claim 13, wherein in case the tuple indicates that the tuple is for a chroma sample, determining the plurality of ALF coefficient values comprises deriving a certain ALF coefficient value based on a combination of (i) a codeword included in the set of codewords and (ii) a parameter of the tuple, which indicates a coefficient group.

15. The method of claim 13, wherein in case the tuple indicates that the tuple is for a luma sample, determining the plurality of ALF coefficient values comprises deriving a certain ALF coefficient value based on a combination of (i) a codeword included in the set of codewords, (ii) a parameter of the tuple, which indicates ^^^^ி, (iii) a parameter of the tuple, which indicates ^^ெ, and (iv) a parameter of the tuple, which indicates a coefficient group.

16. The method of any one of claim 8-15, wherein the set of codewords is determined using Huffman coding.

17. A computer program (543) comprising instructions (544) which when executed by processing circuitry (502) cause the processing circuitry to perform the method of any one of claims 1-16.

18. A carrier containing the computer program of claim 17, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

19. An apparatus (500) for performing a video processing, the apparatus being configured to: determine (s402) an allowed set of coefficient values of adaptive loop filter, ALF,wherein each coefficient value included in the allowed set of ALF coefficient values is ^^ ∙ 2^where ^^ is a mantissa, ^^ is an exponent; and determine (s404) a plurality of ALF coefficient values to use for the video processing, wherein the plurality of ALF coefficient values is included in the allowed set of ALF coefficient values; and perform (s406) the video processing using the plurality of ALF coefficient values.

20. The apparatus of claim 19, wherein the apparatus is configured to perform the method of any one of claims 2-16.

21. An apparatus (500) comprising: processing circuitry (502); and a memory (542), said memory containing instructions executable by said processing circuitry, whereby the apparatus is operative to perform the method of any one of claims 1- 16.

Images