Cross-component adaptive loop filter for chroma

Abstract

To provide a method for executing a cross-component adaptive loop filter.SOLUTION: A method includes: acquiring a plurality of sample values of a first component corresponding to a plurality of samples surrounding a reference sample; acquiring a plurality of filter coefficients from a bitstream; and generating output of a cross-component adaptive loop filter for a second component different from the first component, by using the plurality of filter coefficients and the plurality of sample values of the first component. The plurality of filter coefficients are represented without prefix in the bitstream. The values of the filter coefficients are regulated within a range represented by a value of power of two defined by using a prescribed number of bits. Acquiring the plurality of filter coefficients includes determining a filter coefficient absolute value in the plurality of filter coefficients by using the prescribed number of bits.SELECTED DRAWING: None

Description

[Technical Field]

[0001] The present invention relates to the encoding or decoding of blocks of video components. Embodiments of the invention find particular (but not exclusive) application in controlling filters for filtering samples of such components, particularly but not exclusively in controlling adaptive loop filters. [Background technology]

[0002] Video coding involves coding images (where an image corresponds to one frame of video). In video coding, coding tools such as quantization of transform coefficients and motion compensation (often performed using interpolation filters) can introduce distortion biases or artifacts (distortions that appear systematic, or at least non-random in a given context). To correct these biases and artifacts and improve (or at least maintain a good level of) coding efficiency, several specific coding tools, known as post-filters or in-loop filters, are used. Deblocking filters (DBFs), sample-adaptive offset (SAO) filters, and adaptive loop filters (ALFs) are examples of such coding tools. In-loop filters are applied within the coding loop so that the image quality improvements provided to the current frame allow for improved coding efficiency for subsequent frames that are coded based on the current frame. For example, quantization of DCT coefficients is efficient for video compression, but can introduce blocking artifacts (biases) at the boundaries of compressed sample blocks. Deblocking filters reduce the undesirable effects resulting from such artifacts. Deblocking (using DBFs) a decoded frame within the coding loop (before it serves as a reference frame for motion compensation of another frame) significantly improves the coding efficiency of motion compensation compared to deblocking a frame outside the coding loop (e.g., just before displaying it).

[0003] The present invention particularly relates to an adaptive loop filter (ALF), which can also be applied as an in-loop filter to reduce undesirable compression artifacts in decoded frames / images. The ALF is being studied by the Video Coding Experts Group / Moving Picture Experts Group (VCEG / MPEG) standardization group and is being considered for use in the Versatile Video Coding (VVC) standard, for example, in the fifth version of the VVC Test Model software (VTM-5.0 or VVC Draft Version 5). Cross-component ALF (CCALF), proposed at JVET-O0636 (the 15th Meeting of the Joint Video Experts Team (JVET), held July 3-12, 2019, in Gothenburg, Sweden), is an auxiliary filter to the ALF that uses sample values from the luma channel to estimate additional error correction of artifacts to be performed on sample values from the chroma channels. The CCALF may be applied to each chroma channel. Summary of the Invention

[0004] The present invention relates to improving the coding performance of a portion of the CCALF process by simplifying the cross-component filter.

[0005] Embodiments of the present invention relate to using a reduced bit-depth representation (compared to the internal bit-depth) of either or both of the filter coefficients and / or the cross-component input samples, which may be performed by bit-shifting operations and / or ignoring certain bits of the filter coefficients / input samples.

[0006] In one aspect of the present invention, there is provided a method for filtering an image portion, the method comprising: receiving a plurality of sample values of a first component and a plurality of filter coefficients corresponding to a plurality of samples surrounding a reference sample; and inputting the plurality of filter coefficients and the plurality of sample values of the first component to a cross-component filter to generate a cross-component filter output, the cross-component filter generating the filter output using a limited number of bits to represent the plurality of filter coefficients and / or the plurality of sample values of the first component.

[0007] In this way, the filter calculations are simplified and most of the improvements of the CCALF coding are surprisingly preserved.

[0008] Optionally, the limited number of bits is lower than the number of bits defined by the internal bit depth.

[0009] For efficient implementation, the limited number of bits for representing the filter coefficients and / or sample values of the first component allows the filter output to be represented by 16 bits or less.

[0010] Optionally, the method further comprises combining the cross-component filter output with an output of a filter corresponding to a different component, Optionally, the first component is a luma component and the different component is a chroma component.

[0011] For efficient implementation, the method includes performing a bit shifting operation to reduce the number of bits before the filter multiplication operation, which can simplify the filter calculation.

[0012] Multiple sample value limit In one aspect of the present invention, the number of bits used to represent a sample value is reduced.

[0013] Optionally, according to a third variant, said restricting comprises reducing the accuracy of the sample value by considering only a specific plurality of bits.

[0014] Optionally, a predetermined number of least significant bits are ignored; for example, the least significant bit, the least significant 2 bits, the least significant 3 bits or the least significant 4 bits.

[0015] Optionally, a predetermined number of most significant bits are retained. For example, the most significant 8, 7, 6, 5 or 4 bits.

[0016] To further simplify, this method may include rounding the sample value using the removed most significant bits.

[0017] Optionally, according to a fourth variant, said inputting the plurality of sample values into said filter comprises inputting a numerical value representing the difference in value between the sample value of said first component and a reference sample value.

[0018] Optionally, the method further comprises clipping said difference based on a clipping function defined by a fixed clipping parameter C.

[0019] Said clipping function K may be represented as follows: K(d,C)=max(-C,min(C,d)) Here, C is the clipping parameter and d is the numerical value representing the difference. Thus, the input to the clipping function is the numerical value representing the difference and the clipping parameter, and the output is the maximum value of -C or the smaller of C and d.

[0020] In an advantageous embodiment, C is 2 (IBitDepth-n) -1, IBitDepth is the number of bits used to represent the plurality of sample values, and n is an integer satisfying n < IBitDepth. Optionally, n = 2 or 3.

[0021] Multiple coefficient value restrictions One aspect of the present invention is to reduce the number of bits used to represent multiple filter coefficient values.

[0022] Optionally, according to a second variant, the limiting comprises limiting the range of possible values of the plurality of filter coefficients, which allows the coefficients to be represented using fewer bits, making subsequent calculations simpler.

[0023] Optionally, according to a first variant, said limiting comprises using reduced fixed-point fractional precision for the input coefficient values.

[0024] Particularly advantageous embodiments include where the reduced fixed-point decimal precision is one of 8 bits, 7 bits, or 6 bits.

[0025] In one optional embodiment, limiting the range includes not using any bits above a threshold, for example, the threshold being the 7th, 6th, 5th, 4th or 3rd bit of fixed-point decimal precision.

[0026] In one advantageous embodiment of the present invention, each multiplication is replaced by a bitwise shift and add operation, which is easier to implement in both hardware and software. Thus, the filter operation includes multiplication of a plurality of sample values by one filter coefficient, and the multiplication is performed via a bitwise shift and / or combination with another filter coefficient, such that the multiplication operation is replaced by a bitwise shift and add operation.

[0027] Filter Shape Another aspect of the present invention proposes a modification of the filter shape, which involves reducing the number of samples to be considered (and thus simplifying the resulting calculation) while maintaining the gain provided by CCALF.

[0028] Optionally, each sample value of the first component is one sample value at a defined position relative to a reference sample, said position of the sample being defined by the filter shape.

[0029] For ease of implementation, the filter shape is limited to contain 16 samples or less.

[0030] In one embodiment, the filter shape comprises a pattern in which samples of the first component have positions (x+u, y+v) relative to the reference samples, where u and v belong to the [-2,2] integer interval.

[0031] In another embodiment, the filter shape comprises a pattern in which all samples have a position (x+u, y+v) relative to the reference sample, with u in [-2,2] and v in [-1,2]. In a variant, the filter is "diamond-shaped", e.g., when |u|=2, v in [0,1].

[0032] In one embodiment, the filter shape comprises a pattern in which all samples have a position (x+u, y+v) relative to the reference sample, with u in [-1,1] and v in [-1,2]. In a variant, the filter is "diamond-shaped", e.g., when |u|=1, v in [0,1].

[0033] In another embodiment, the filter shape comprises a pattern in which all samples have a position (x+u, y+v) relative to the reference sample, where u, v are in [0,1].

[0034] signaling Embodiments of the present invention also relate to signaling multiple CCALF filter coefficients in a bitstream. An object of the present invention is to improve the signaling in a way that minimizes the bit rate cost associated with CCALF.

[0035] Optionally, the method further comprises signaling the reduced number of bits for said filter coefficients and / or sample values in a bitstream, thereby allowing for consistent processing across different images and flexibility in processing where required.

[0036] According to one aspect of the invention, the method further includes determining whether a cross-component adaptive loop filter (CCALF) is indicated for a given component and decoding the plurality of filter coefficients from the bitstream.

[0037] According to another aspect of the present invention, there is provided a method for decoding an image portion coded using CCALF, the method comprising: determining whether CCALF is indicated for a given component; decoding a plurality of CCALF coefficients from a bitstream; and decoding the image portion using the plurality of CCALF coefficients, the plurality of CCALF coefficients being coded in the bitstream without a prefix.

[0038] This aspect represents a simplification over the prior art, and especially when combined with reducing the number of bits used to represent multiple filter coefficients as described herein, results in lower bit rates without significantly compromising the performance of CCALF.

[0039] Optionally, determining whether CCALF is indicated for a given component comprises decoding a flag.

[0040] According to another aspect of the present invention, there is provided a method for encoding an image portion using a cross-component adaptive loop filter (CCALF), the method comprising: determining whether CCALF is indicated for a given component; encoding a plurality of CCALF coefficients into a bitstream; and encoding the image portion using the plurality of CCALF coefficients, wherein the plurality of CCALF coefficients are encoded into a slice header using a fixed representation.

[0041] Optionally, determining whether CCALF is indicated for a given component comprises encoding a flag in a slice header.

[0042] In a first variant, for ease of implementation, the CCALF coefficients may be coded directly in the slice header.

[0043] The CCALF coefficients may be coded into the bitstream using signed unary coding, which is simple to code and decode and does not depend on other parameters (e.g., prefixes) to do so.

[0044] Optionally, according to a third variant, said signed unary coding comprises one bit indicating the sign of said coefficient and a series of bits indicating the magnitude of said coefficient.

[0045] Optionally, according to a second variant, the signed unary coding comprises a first bit in the bitstream indicating whether the coefficient is zero or not, and if the coefficient is not zero, one bit indicating the sign of the coefficient and a bit sequence equal to 1 indicating the magnitude of the coefficient.

[0046] Optionally, said first bit equal to 1 indicates that said coefficient is zero.

[0047] Optionally, the sign bit equal to 1 indicates that the coefficient is negative.

[0048] Optionally, to facilitate parsing, said signed unary encoding further comprises one bit equal to zero at the end of the sequence indicating the end of the signaling of said coefficient.

[0049] Yet another aspect of the invention relates to an apparatus for filtering images as defined by claim 48.

[0050] Yet another aspect of the invention relates to an encoder and a decoder as defined by claims 49 and 50 respectively.

[0051] Yet another aspect of the present invention relates to a program as defined by claim 51. The program may be provided on its own, on a carrier medium, by or within the carrier medium. The carrier medium may be non-transitory, for example a storage medium, in particular a computer-readable storage medium. The carrier medium may also be transitory, for example a signal or other transmission medium. The signal may be transmitted via any suitable network, including the Internet.

[0052] Further features of the invention are characterized by the other independent and dependent claims.

[0053] Any feature of one aspect of the invention may be applied to other aspects of the invention in any appropriate combination, in particular method aspects may be applied to apparatus aspects and vice versa.

[0054] Furthermore, features implemented in hardware may be implemented in software and vice versa, and any references herein to software and hardware features should be construed accordingly.

[0055] Apparatus features as described herein may also be provided as method features, and vice versa. As used herein, means plus function features may alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and associated memory.

[0056] It is also to be understood that specific combinations of the various features described and defined in any embodiment of the present invention can be implemented and / or provided and / or used independently. [Brief explanation of the drawings]

[0057] Embodiments of the present invention will now be described, by way of example only, with reference to the following drawings:

[0058] [Figure 1] Figure 1 shows where ALF takes place in a typical decoding loop in VTM-5.0. [Figure 2] FIG. 2 is a flowchart outlining the ALF filter of VTM-5.0. [Figure 3] Figure 3 is a flowchart showing an overview of the CCALF filter in addition to the ALF in VTM-5.0. [Figure 4-a] FIG. 4-a provides the filter shape and coefficient arrangement of a CCALF filter according to an embodiment of the present invention. [Figure 4-b] Figure 4-b shows a typical bit-wise memory representation of the CCALF filter coefficients. [Figure 4-c] Figure 4-c shows a typical bit-wise memory representation of sample values used in a CCALF filter with an internal bit depth of 10 bits. [Figure 4-d] Figure 4-d shows a typical bit-wise memory representation of the output of the multiplication of the CCALF filter coefficients and sample values during the CCALF filter implementation. [Figure 4-e]FIG. 4-e shows an exemplary bit-wise memory representation of the output of an addition operation performed to sum the results of multiplications of multiple CCALF filter coefficients and multiple sample values during the implementation of a CCALF filter. [Figure 5-a] FIG. 5-a shows a bitwise memory representation according to a first variant of the invention that reduces the number of bits of the CCALF filter coefficients by reducing the precision of the coefficient values. [Figure 5-b] FIG. 5-b shows a bit-wise memory representation according to a second variant of the invention that reduces the number of bits for the CCALF filter coefficients by restricting the range of possible coefficient values. [Figure 5-c] FIG. 5-c shows a bit-wise memory representation of the CCALF filter coefficients that reduces the number of bits by reducing the precision of the coefficient values and limiting the range of possible coefficient values, according to a combined embodiment of the first and second variants of the present invention. [Figure 5-d] FIG. 5-d shows a bitwise memory representation according to a third variant of the invention, which reduces the number of bits of the sample values used inside the filtering process by ignoring the least significant bits. [Figure 6-a] FIG. 6-a shows a bit-by-bit memory representation of the difference between multiple sample values used inside the filter processing according to a fourth variant of the present invention. [Figure 6-b] FIG. 6-b shows a bit-wise memory representation of the difference of multiple sample values when a clipping operation is used to reduce the number of bits inside the filtering process according to a fourth variant of the present invention. [Figure 6-c] FIG. 6-c shows a bit-wise memory representation of the difference of multiple sample values when a clipping operation is used to reduce the number of bits within the filter processing and the least significant bits are ignored, according to an embodiment combining the third and fourth variants of the present invention. [Figure 7-a] FIG. 7-a shows an exemplary bit-wise memory representation of the output of the multiplication of the CCALF filter coefficients and sample values during the implementation of a CCALF filter according to a variant of the present invention. [Figure 7-b]FIG. 7-b shows an exemplary bit-wise memory representation of the output of an addition operation performed to sum the results of multiplications of CCALF filter coefficients by sample values during implementation of a CCALF filter according to a variant of the present invention. [Figure 7-c] FIG. 7-c provides the filter shape and coefficient arrangement of an example of a CCALF filter that uses 16 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variant of the present invention. [Figure 7-d] FIG. 7-d provides the filter shape and coefficient arrangement of an example of a CCALF filter that uses 16 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variant of the present invention. [Figure 7-e] FIG. 7-e provides the filter shape and coefficient arrangement of an example of a CCALF filter that uses 12 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variant of the present invention. [Figure 7-f] FIG. 7-f provides the filter shape and coefficient arrangement of an example of a CCALF filter that uses eight samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variant of the present invention. [Figure 7-g] FIG. 7-g provides the filter shape and coefficient arrangement of an example of a CCALF filter that uses four samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variant of the present invention. [Figure 8] FIG. 8 is a flow chart illustrating the steps of an encoding method according to an embodiment of the invention. [Figure 9] FIG. 9 is a flow chart illustrating the steps of a decoding method according to an embodiment of the present invention. [Figure 10] FIG. 10 is a block diagram that schematically illustrates a data communications system in which one or more embodiments of the present invention may be implemented. [Figure 11] FIG. 11 is a block diagram illustrating components of a processing device in which one or more embodiments of the present invention may be implemented. [Figure 12] FIG. 12 is a diagram illustrating a network camera system in which one or more embodiments of the present invention may be implemented. [Figure 13] FIG. 13 is a diagram illustrating a smartphone in which one or more embodiments of the present invention may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0059] The embodiments of the present invention described below relate to improvements in image encoding and decoding.

[0060] As used herein, "signaling" may refer to inserting into (providing / including / encoding) or extracting / obtaining from (decoding) a bitstream information regarding one or more parameters for controlling a filter (e.g., whether a mode / scheme is in use, not in use, enabled, or disabled, or other filter control related information).

[0061] The term "slice" is used herein as an example of an image portion (other examples of such an image portion are a tile or a tile group (which is a group / set of tile(s))). It will also be understood that embodiments of the present invention can be implemented based on image portions (e.g., tiles or tile groups) instead of slices, and on appropriately modified parameters / values / syntax, such as image portion / tile / tile group headers (instead of slice headers), image portion / tile / tile group types (instead of slice types), and image portion / tile / tile group statistics (instead of slice statistics). It will also be understood that instead of slice headers or sequence parameter sets (SPS), adaptive parameter sets (APS) or tile (group) headers may be used to signal ALF or CCALF parameters (or information for using a (CC)ALF filter). When an APS is used to signal (CC)ALF parameters (or information for using a (CC)ALF filter), the slice header or tile group header may be used to indicate which APS should be used to obtain the (CC)ALF parameters (or information for using a (CC)ALF filter), for example, by indicating an adaptive set identifier (aps_id). It will also be understood that any of a slice, tile group, tile, coding tree unit (CTU) / largest coding unit (LCU), coding tree block (CTB), coding unit (CU), prediction unit (PU), transform unit (TU), or block of pixels / samples may be referred to as an image part.

[0062] It is also understood that when a filter or tool is described as "active", the filter / tool is "enabled" or "available" or "used", and when described as "inactive", the filter / tool is "disabled" or "disabled" or "not used"; and a "class" means a group, grouping, category, or classification of one or more elements. It is further understood that when a flag is described as "active", the flag is meant to indicate that the associated filter / tool is "active".

[0063] Adaptive Loop Filter (ALF) Figure 1 shows where the ALF process takes place in a typical VTM-5.0 decoding loop. At 101, an image portion (e.g., a slice) is decoded in coding tree units (CTUs, the largest coding unit in VVC, typically 128x128 samples / pixels in size). CTUs are then divided into rectangular blocks, or coding units (CUs), which are coded using a specific prediction scheme / mode, often with lossy coding of residual blocks. The use of block-based coding can result in visible blocking artifacts at the boundaries between coded blocks. At 102, the decoded image portion is processed by DBF to reduce or remove these artifacts. Typically, to code the residual (block) for block prediction, the residual values are transformed using a DCT-like transform (to compress the residual energy into a small number of coefficients), and the transformed coefficients are quantized to reduce the coding cost. This quantization often introduces ringing artifacts in the reconstructed block (i.e., the block in the reference frame stored in frame buffer 106). At 103, the output image portion of the DBF is then processed by an SAO filter, which is useful for reducing some of these artifacts at low computational cost. At 104, the output image portion of the SAO filter is then processed by an ALF. The ALF can further reduce artifacts such as "ringing." The ALF has higher-order error modeling capabilities, but at a higher computational cost. The output image portion of the ALF is then sent to an output (e.g., a display or a communication interface for communicating with a display) 105. It can also be placed in a frame buffer 106 (as part of a reference frame stored there) for use in temporal prediction (if a temporal prediction tool is used). This is why the DBF, SAO filter, and ALF are referred to as "in-loop" filters. The encoder can disable some of the in-loop filters so that they are bypassed during decoding (i.e., the filter is not executed and the output of the step corresponding to the disabled tool is the same as its input).Also, in some cases, the image portion being processed is not limited to a slice, but may be a full frame containing one or more slices, and filters may be applied across slice boundaries (if there are multiple) to reduce artifacts on those boundaries. For multi-component images (e.g., images in YCrCb format), DBF, SAO filter, and ALF processing are applied to each component separately, possibly in a different manner (e.g., using different filtering parameters than the other components).

[0064] FIG. 2 shows an overview of the ALF filter in VTM-5.0. For each channel (luma channel: Y, chroma channels: Cb, Cr), input image portions 201, 202, 203, along with filter parameters 204, 205, 206, are fed to adaptive loop filter (ALF) processes 207, 208, 209. The ALF processes generate processed output image portions for each channel 210, 211, 212. The ALF chroma processing in 208, 209 is the same for both chroma channels but can be configured to operate differently for each channel via specific parameters provided in 205, 206. The ALF luma processing performed in 207 is often more complex than the ALF chroma processing performed in 208. There are two reasons for this. The first reason is that the content of the luma channel is often more complex to process / encode than the chroma channels because the luma channel contains more structure (contours, texture, etc.) than the chroma channels, which are often smoother. The second reason is that the human eye is more sensitive to errors in the luma channel than to errors in the chroma channels, so more advanced and complex (i.e., more power-consuming and / or slower) processing is preferred for the luma channel to provide better coding gain and better visual quality.

[0065] The input parameter information provided by 204, 205, and 206 is carried in an adaptation parameter set (APS) and / or slice header, and 204, 205, and 206 may share some common information.

[0066] The ALF processes 207, 208, and 209 can be performed in parallel on each channel because the results of each task are independent of the results of the others. This parallel processing opportunity can reduce the overall processing time and can be implemented efficiently in either hardware or software.

[0067] The input parameter information provided in 204, 205, and 206 may also include internal bit depth information. The internal bit depth is the maximum number of bits that can be used to represent a sample value (i.e., it indicates the precision of the sample value). VTM-5.0 allows the use of an internal bit depth for the luma channel that is different from the internal bit depth for the chroma channels (bit depth information is signaled in the Sequence Parameter Set (SPS) for both luma and chroma channel types). In the following description, "IBDL" means the internal bit depth for luma, and "IBDC" means the internal bit depth for chroma. When "IBD" is used for the internal bit depth, it means that IBDL is considered equivalent to IBDC. If there is no mention or no specification, it means that the internal bit depth does not affect the processing being discussed or is omitted because the internal bit depth is treated as orthodox.

[0068] FIG. 3 shows an extension of the ALF filter of FIG. 2 to use cross-component ALF (CCALF) processing.

[0069] For each chroma channel 202, 203, the input image portion of the luma channel 201 is provided to a cross-component adaptive loop filtering (CCALF) process 308, 309 along with filter parameters 305, 306. The output of the CCALF filtering is added to the output of the ALF chroma filtering process 208, 209 at 311, 312 to generate processed output image portions 313, 314 for each chroma channel. The CCALF processing at 308, 309 is the same for both chroma channels but can be configured to operate differently for each channel by specific parameters provided at 305 and 306. For example, each chroma channel may be processed using specific filter coefficients.

[0070] In some variations, the chroma channels are subsampled compared to the luma channel, for example for video using a YUV 4:2:0 color representation. In such variations, the CCALF filtering 308, 309 is not applied to each luma sample position, but only to integer luma sample positions obtained when upsampling the chroma sample positions. Thus, in a YUV 4:2:0 representation, the filter is applied to only 1 / 4 of the luma sample positions.

[0071] The ALF filter processes 207, 208, and 209 and the CCALF filter processes 308 and 309 are independent and can be performed in parallel. Compared to Figure 2, Figure 3 introduces latency due to the addition operations performed in 311 and 312 to combine the results of the two chroma filters for each channel. However, because the CCALF and chroma ALFs are generally less complex than the luma ALF, it should be possible in VTM-5.0 to achieve overall processing of the three channels without additional latency (i.e., the rate determination step is considered to be the luma ALF, so the additional latency in the other two channels does not affect the overall processing latency).

[0072] The input parameter information provided at 305 and 306 is carried in an adaptation parameter set (APS) and / or slice header. 305 and 306 may also share some common information.

[0073] In one variation, 305, 306 contain filter coefficients for a linear convolution filter (see, e.g., FIG. 4). In one variation, 305 and 306 contain filter coefficients and clipping parameters for a nonlinear filter, similar to, for example, those used in VTM-5.0 for the nonlinear ALF filter. Both linear and nonlinear filters have been found to provide coding performance gains, although linear filters are less complex and as such may be preferred in certain implementations.

[0074] In some variations, the input parameter information provided at 305 and 306 may include IBDL and IBDC for inferring some left or right bit-shifting operation parameters (or multiplication / division operation parameters) for "normalizing" (i.e., using equivalent precision values) data (e.g., sample values) between channels.

[0075] It will be appreciated that in some variations, CCALF can be applied even when the ALF filter is not implemented. It will also be appreciated that the luma channel or other types of channels can be used to correct types of channels other than chroma channels. For example, the red channel can be used to correct the green and / or blue channels in video using an RGB color representation.

[0076] It will also be appreciated that in some variations, more than one channel may be used to reduce errors in another channel. For example, input image portions of both chroma channels 202 and 203 may both be filtered with a CCALF filter, and the filter results for each image portion may be added to the output of ALF luma filter 207 to correct remaining errors in the luma channel. Such variations may be particularly applicable when the chroma channels are subsampled compared to the luma channel. For example, in a YUV4:2:0 color representation, four different filters may be used on the chroma samples, one filter for each of the four luma sample positions that are downsampled / associated with one chroma sample position. In other words, for a 2x2 block of luma samples, a first filter is applied to the corresponding chroma position to correct the luma sample at position (0,0) of the 2x2 block, and a second, third, and fourth filters are applied to the same chroma sample position to correct the luma samples at positions (0,1), (1,0), and (1,1), respectively. In a variant, only one CCALF filter is used for four luma sample positions, but the filtered chroma sample values are obtained by upsampling the chroma samples to obtain chroma samples aligned on the same grid as the luma samples.

[0077] Figure 4-a shows an example of a filter shape 401 that can be used to perform CCALF with a linear convolution filter. This filter shape is suitable for filtering the luma channel to correct errors in the chroma channels of video using the YUV4:2:0 color representation, but can also be used with other types of color representations. Each box corresponds to multiple samples arranged around a reference sample location (the box with a gray dotted background). The number in each box corresponds to the index of the filter coefficient to apply to the sample at the box's location.

[0078] position(x c ,y c) Cb component chroma samples are filtered / corrected by adding the following amount to their values:

[0079]

number

[0080] where f Cb i is the filter coefficient of the filter index i for the Cb chrominance component.

[0081] The same filtering is performed for the Cr component, but f Cr i The filter coefficients are used.

[0082] To implement a similar filter, the following formula notation is used to simplify the notation:

[0083]

number

[0084] where f Cb (ui,vi) is the reference sample (the box with the gray dotted background at 401) with position (x,y) and the offset (ui,vi) of sample I in the filter shape. (x+ui,y+vi) indicates the filter coefficients to be applied.

[0085] The implementation of equation (2) is typically done using integer arithmetic to approximate real arithmetic using fixed-point arithmetic, and equation (2) can be rewritten as follows:

[0086]

number

[0087] where << is an integer left bit shift operation, >> is an integer right bit shift operation, and B is the filter coefficient f' Cb (ui,vi)(See Figure 4-b) The decimal precision of the fixed-point representation (usually 10 bits), I (x+ui,y+vi) is an integer sample value with the same integer precision as the IBD of the codec (usually 10 bits, see Figure 4-c).

[0088] In YUV4:2:0 color representation, the luma sample position (x,y) is the upsampled chroma position (x,y)=(2.x c ,2.y c ) corresponds to the nearest integer value.

[0089] In some variations, when IBDL is greater than IBDC, equation (3) can be rewritten as:

[0090]

number

[0091] or

[0092]

number

[0093] or

[0094]

number

[0095] and when IBDL is smaller than IBDC, equation (3) can be rewritten as:

[0096]

number

[0097] Or, equivalently, when B+IBDL is strictly greater than IBDC:

[0098]

number

[0099] and B1=B+IBDL-IBDC.

[0100] By using the bitwise shift operations in equations (4), (5), (6), and (7) or by changing the bitwise shift parameters in equation (8), the filter results can be normalized to maintain the same precision as the chroma sample values. To achieve this, equation (4) normalizes the luma sample values, equation (5) normalizes the multiplication results of the luma sample values and coefficient values, or equations (6), (7), and (8) normalize the sum of the multiplication outputs (i.e., the filter result).

[0101] Moreover, the same result can be obtained by rewriting equation (6) as equation (8).

[0102] In equation (7), we intentionally do not enclose the bitwise shift in parentheses because the result is the same no matter where the left shift occurs when IBDL is less than IBDC. This is a matter of hardware or software design choice. It is also possible to round the shift value in equation (4) by adding 1 << (IBDC-IBDL-1) before the right shift by (IBDL-IBDC).

[0103] From a hardware / software design perspective, when IBDL is larger than IBDC, an implementation according to the formulation of Equation (8) may be preferable because it requires fewer operations. In the specific case where IBDL is always larger than IBDC, Equation (4) may be preferable to limit the number of bits in the multiplication input. The number of bits obtained in a system implementation using Equation (4) is determined by the maximum possible IBDC and the maximum possible IBDL in the system (since it is the worst case that must be handled). When IBDC is larger than IBDL, it may be desirable to use Equation (8) to avoid increasing the input bit depth of the multiplication, especially when the maximum possible IBDC is always larger than the maximum possible IBDL. When the maximum possible IBDC and the maximum possible IBDL are equal, Equation (8) is the best compromise.

[0104] If B+IBDL is not strictly greater than IBDC, then equation (8) can be rewritten as follows:

[0105]

number

[0106] Here, B2 = IBDC - IBDL - B.

[0107] Figure 4-b shows a typical bitwise memory representation of the CCALF filter coefficients in equation (3). The bit representation 402 of the CCALF filter coefficients typically uses a 16-bit signed integer representation (the complement of the tow representation) to store a real fixed-point approximation. 10 bits of precision are used for the fractional part, and the other bits are used to store the integer part and the sign.

[0108] Figure 4-c shows a typical bitwise memory representation of the sample values of Equation (3) with an internal bit depth of 10 bits in VTM-5.0 when stored in a 16-bit integer memory space 403 (or register). The crossed boxes represent unused bits.

[0109] Figure 4-d shows the result of multiplication of sample values with 10-bit internal bit depth and CCALF filter coefficients (e.g., f' for a given i) when the CCALF filter coefficient representation corresponds to that used in Figure 4-b. Cb (ui,vi) I (x+ui,y+vi) ) is a typical bitwise memory representation of a 32-bit signed integer. As can be seen, we need 24 bits of precision (10 bits for 403 and 14 bits for 402) and 1 bit for the sign (i.e., a 25-bit signed integer representation).

[0110] Figure 4-e shows the accumulation (sum) of the results of multiplication of 10-bit internal bit-depth sample values by CCALF filter coefficients (e.g., Σif') when the CCALF filter coefficient representation corresponds to that used in Figure 4-b and uses the filter pattern of Figure 4-a. Cb (ui,vi) I (x+ui,y+vi) ) is shown in Figure 4-b. This corresponds to the bitwise memory required to store the sum of eighteen 25-bit signed integers, as in Figure 4-c. As can be seen, we need 29 bits of precision (10 bits of precision for 403 + 14 bits of precision for 402 + 5 bits to store the worst-case results of the 18 additions) and 1 bit for the sign (i.e., a 30-bit signed integer representation).

[0111] According to an embodiment of the present invention, the filter operation of equation (3) for the CCALF filter is significantly simplified compared to the typical coefficient representation of FIG. 4-b, especially in a hardware-specific design where the number of logic gates has a cost and the number of gates for multiplication depends on the maximum number of bits of the input. It has been surprisingly found that significant simplifications can be achieved with significant impacts on coding efficiency when significant simplifications of the design are involved. In general, the simplifications disclosed below are based on the filter coefficients (f' Cb (ui,vi) ) or cross-component input samples (I (x+ui,y+vi)) or both. The reduced bit depth representation is limited compared to the internal bit depth of the first component samples, or limited compared to the bit depth of typical filter coefficients (a common example of both is 10 bits).

[0112] It will be appreciated that certain combinations of bit-depth reductions are particularly advantageous (e.g., when they lead to calculations that can be performed on a 16-bit architecture). In any case, calculations using a limited number of bits are fast, meaning that the additional cost (either in time or increased computational power) of CCALF (compared to just ALF) is low.

[0113] A first variant to reduce the number of bits in the multiplication input of Equation (3) is to reduce the fixed-point fractional precision. This is possible by removing some of the least significant bits in Figure 4b. For example, using 7 bits instead of 10 bits has been found to represent a good compromise (e.g., when using a 10-bit sample internal bit depth) without significantly reducing the coding gain of CCALF. In some variants, using 6 bits instead of 10 bits reduces the coding gain of CCALF a little, but still represents an interesting hardware design compromise.

[0114] Figure 5-a shows an example of a bitwise memory representation of the CCALF filter coefficients of equation (3) when the fixed-point fractional precision is limited to 7 bits instead of 10. The bitwise representation in 402 is then reduced to 13 bits (12 bits of precision, 1 bit of sign) in 501.

[0115] A second modification to reduce the number of bits in the multiplication input of Equation (3) is to restrict the range of possible values of the filter coefficients. This can be achieved by deleting some of the most significant bits shown in Figure 4-b. Surprisingly, we find that CCALF still operates efficiently even when the range of possible values of the coefficients is significantly restricted. This effect is particularly surprising because it is specific to CCALF and is not reproduced in, for example, ALF. The surprisingly low impact of such simplification on coding efficiency can be explained by the inventors' discovery that cross-component ALF is most useful where the gradient of the luma samples is strong (e.g., near edges), and chroma errors at these locations are generally strong and correlated with the gradient of the luma samples. Furthermore, CCALF outputs at such locations are typically small / limited (while being nearly zero elsewhere). This means that mathematically optimal filter coefficients (in the real domain) are generally small in amplitude and can be adequately approximated by a small-bit integer representation (fixed-point decimal). Therefore, most of the coding performance improvement can be achieved from a subset of multiple coefficients. Embodiments of the present invention can exploit this property to simplify the calculations without significantly impacting coding performance. This also explains why the third and fourth variants below work: reducing the precision of the samples / coefficients creates a kind of "rounding error noise" that does not significantly affect the estimation of small values in the CCALF output.

[0116] In summary, the following became clear: i) Restricting coefficients to the range -0.03125 to 0.03125 (e.g., not using more than the sixth decimal point bit) is an acceptable coding impact. ii) Restricting the coefficients to the range -0.0625 to 0.0625 (e.g., not using more than the 5th decimal point bit) does not significantly affect the coding. iii) Restricting the coefficients to the range -0.125 to 0.125 (e.g., not using more than the fourth decimal point bit) has little effect on the encoding. iv) Restricting the coefficients to the range -0.25 to 0.25 (e.g., not using more than three decimal points) has negligible effect on the coding. v) Restricting coefficients to the range of -0.015625 to 0.015625 (e.g., not using bits higher than the 7th decimal point) has a coding impact that may be acceptable in a particular implementation.

[0117] The limits of the ranges may or may not be inclusive of the coefficient values that can be used. Examples using these three ranges are detailed below, but alternative ranges may also be used.

[0118] The following discussion provides results. These results were obtained using the "JVET Common Test Conditions and Software Reference Configuration for SDR Video" defined in the JVET-N01010 document, but restricting the test conditions to apply to the first 17 frames of a video sequence. Thus, a variation of the present invention is implemented in the VTM-5.0 software (or an intermediate development version during the development process to provide the VTM-6.0 software). Average BD-rate coding gain results for the "All Intra" (AI), "Random Access" (RA), "Low Latency B" (LB), and "Low Latency P" (LP) configurations for the luma and both chroma channels are compared to a reference (e.g., VTM-5.0). These average channel coding gains are combined by adding 14 times the luma coding gain to the coding gain of each chroma channel and dividing the sum by 16 to obtain the average YUV coding gain. This is the conventional method for combining the results of the luma and chroma channels, since the chroma channel is easier to code than the luma channel. The average YUV coding gains obtained with the AI, RA, LB, and LP configurations are averaged together, and these final average coding gain values are compared together to provide a coding gain comparison. The coding gain comparison is expressed as a percentage relative to the reference software (i.e., VTM-5.0 or a later intermediate development version) specified in the JVET-O0636 contribution, which implements CCALF but does not include the invention to simplify it (unless explicitly specified otherwise). The compared variant is implemented using a simplified version of that reference (i.e., using the same VTM-5.0 or a later intermediate development version, but implementing the simplified CCALF of the invention). The percentage is obtained by calculating the ratio between the coding gain of the variant (which is the average BD rate gain, expressed as a percentage of rate reduction for the same quality) and the coding gain of the reference (also the average BD rate gain). Note that using the reference CCALF in VTM-5.0 results in an average YUV coding gain of approximately -1.2% compared to VTM-5.0.

[0119] The coding gain comparison results shown below are for reference only; when implementing the invention in a different encoder implementation or with a different codec specification, the reported coding gain percentages may vary.

[0120] In the following description, range notation is used with the following meanings: - ]x,y[ means the range of values between x and y, not including x or y. - [x,y[ means the range of values between x and y, including x but not including y. - ]x,y] means the range of values between x and y, including y but not including x. - [x,y] means the range of values between x and y, inclusive.

[0121] Using coefficients in the range [0.03125;0.03125[ (i.e., not using bits above the sixth bit of fixed-point precision) allows us to preserve most of the coding gain of CCALF while significantly reducing the number of bits used for multiplication. An example of this variation is shown in Figure 5-b.

[0122] Figure 5-b shows an example of a bit-wise memory representation of the CCALF filter coefficients of equation (3) when using 10-bit fixed-point precision and restricting the range of possible values of the coefficients to the range [-0.03125;0.03125[. The bit representation in 402 is then reduced to 6 bits (5 bits of precision, 1 bit of sign) in 502.

[0123] The second variant (FIG. 5-b) can be combined with the first variant (FIG. 5-a) as shown in FIG. 5-c.

[0124] Figure 5-c shows an example of a bit-wise memory representation of the CCALF filter coefficients of equation (3) when the fixed-point precision is limited to 7 bits instead of 10 bits, and the range of possible values of the coefficients is limited to the range [-0.03125;0.03125[. The bit representation in 402 is then reduced to 3 bits (2 bits of precision, 1 bit of sign) in 503.

[0125] Furthermore, by restricting the coefficients to the range ]-0.03125;0.03125[ (i.e., the filter coefficients can only take on values in the set {-0.0234375, -0.015625, -0.0078125, 0, 0078125, 0.015625, 0.0234375}), we can further simplify the problem and replace each multiplication with two additions / subtractions (due to sign operations) and a bitwise shift (as shown in Figure 5-c). For example, the multiplication in Equation (3) can be replaced by, for example,

[0126]

number

[0127] where << is an integer left bit shift operation, >> is an integer right bit shift operation, and B is the filter coefficient (f' Cb (ui,vi) ) (see Figure 5-c) (x+ui,y+vi) is an integer sample value with the same integer precision as the codec's internal bit depth (usually 10 bits, see Figure 4-c).

[0128] Multiplications can also be replaced with just one addition / subtraction (depending on the sign operation) and a bitwise shift. However, this incurs a further loss in the coding gain of CCALF. For example, it is feasible to limit the fixed-point precision to 6 bits and the coefficients to the range ]-0.03125;0.03125[ (i.e., the filter coefficients can only take on values in the set {-0.015625, 0, 0.015625}). It is also feasible to limit the fixed-point precision to 7 bits and the coefficients to the range ]-0.015625;0.015625[ (i.e., the filter coefficients can only take on values in the set {-0.0078125, 0, 0.078125}). Using the first variant, it is possible to maintain approximately 78% of the CCALF gain compared to using 10-bit fractional precision and filter coefficients in the range ]-1;1[. Using the second variant, the gain of CCALF can be reduced to about 76%.

[0129] In some other variations, the coefficient values range from [-0.0625;0.0625[,]-0.0625;0.0625[, [0.125;0.125[,]0.125;0.125[, [-0.25;0.25], or,]-0.25;0.25]. That is, one, two, or three additional most significant bits are retained for the coefficient (fractional) precision. For example, as a variation of Figure 5-c, three or four bits are retained for the filter coefficients instead of two bits. In such variations, the additional bits are one or two bits higher than those retained in Figure 5-c. The second and fourth of these four variations can replace the multiplications by three and four additions / subtractions (due to sign operations) and bitwise shifts, respectively.

[0130] A third variation to reduce the number of bits in the multiplication input of equation (3) is to reduce the precision of the sample values. This can be achieved by not considering the least significant bits of the sample values (this is equivalent to using a bitwise shift to the right using a CPU implementation). For example, equation (3) can be rewritten as:

[0131]

number

[0132] Here, DB corresponds to the number of dead bits (bits that are ignored / not considered). (x+ui,y+vi) An example of the bit representation of ≫DB) is shown in Figure 5-d.

[0133] Figure 5-d shows an example of the bitwise memory representation of the sample values used in equation (3) when the internal bit depth of VTM-5.0 is 10 bits and the precision of the sample values is limited from 10 bits to 6 bits to simplify the hardware implementation of the multiplication operation in equation (3). The crossed boxes represent unused bits. The bit representation of 403 is reduced to 6 bits in 504.

[0134] A variant could be to "round" the sample values using the value of the removed most significant bit (e.g., by adding 1 to the reduced representation if the removed most significant bit is 1, or equivalently, by adding the value corresponding to 1 in the removed most significant bit before removing / not considering the unnecessary bit). Such a variant could be interesting if reducing the sample precision significantly affects the coding efficiency of CCALF, but it could also mean additional operations or memory usage compared to just ignoring the least significant bit.

[0135] In VTM-5.0, which is improved by CCALF, we found that retaining only the most significant six bits of sample values when filtering does not significantly affect coding efficiency. For example, with an internal bit depth of 10 bits, the least significant four bits of sample values can be ignored, resulting in a small impact on CCALF efficiency. Retaining fewer than the most significant six bits of sample values is possible, but it reduces CCALF coding efficiency. For example, retaining the most significant four bits of sample values when filtering (i.e., removing the least significant six bits of 10-bit samples) can produce surprisingly interesting gains (on average, approximately 88% of the CCALF gain is maintained compared to using the full sample precision of 10-bit samples). Meanwhile, retaining the most significant five bits maintains approximately 95% of the CCALF gain compared to using the full sample precision. For the most significant six bits, approximately 98% of the gain is maintained. Retaining more than six most significant bits only slightly increases the results. For example, retaining the 7 or 8 most significant bits (i.e., removing the bottom 3 or 2 bits of a 10-bit internal bit depth) results in something on the order of 99% of the full sample-accurate CCALF gain.

[0136] In a fourth variation, the CCALF filter expressed in equation (2) is modified to be closer to the nonlinear ALF (NLALF) filter formulation of VTM-5.0.

[0137]

number

[0138] where f Cb (ui,vi) are still filter coefficients, but the filter shape sample I at the reference sample offset (ui,vi) (x+ui,y+vi) and reference sample I (x,y)The difference is expressed by multiplying the difference by a number that represents the difference between the values of the reference sample and the sample adjacent to it. (Box 401 with a gray dotted background) has a position (x,y). The shape representing the difference value may be a gradient that corresponds to the difference between the reference sample and the sample adjacent to it. Using a gradient indicates that the difference is normalized for both the sample adjacent to the reference sample and the sample not adjacent to it.

[0139] Equation (12) can be implemented in integer arithmetic, just as equation (2) was implemented in integer arithmetic in equation (3).

[0140]

number

[0141] where << is an integer left bit shift operation, >> is an integer right bit shift operation, and B is the filter coefficient f' Cb (ui,vi) The decimal precision of the fixed-point representation of is (usually 10 bits) (see Figure 4-b), and I (x+ui,y+vi) and I (x,y) is the integer sample value with the same integer precision as the internal bit depth of the codec (usually 10 bits, see Figure 4-c). When using this formulation, I (x+ui,y+vi) Compared with the bit representation of (I (x+ui,y+vi) -I (x,y) ) has an additional bit to indicate the sign. This is shown in Figure 6-a. In some alternatives, Cb (0,0) is always equal to 0 and is not coded in the bitstream. Cb (0,0) I (x,y) is not necessarily zero, and this multiplication is treated like any other multiplication that does not use the difference of the reference sample.

[0142] Figure 6-a shows the sample value difference (I (x+ui,y+vi) -I (x,y)6 shows the bitwise memory representation of the 16-bit integer 601 stored in a 16-bit integer memory space 601 (or register), where the crossed boxes represent unused bits.

[0143] From a hardware design perspective, there is no cost involved, as the difference from the reference sample is already present / calculated for the NLALF filter of the luma sample (the filter shape of the CCALF filter is included in the shape of the NLALF filter for luma).

[0144] The coding gain of CCALF when the filter of equation (3) is used is the same as when the filter of equation (13) is used.

[0145] The advantage of this representation is that, in order to reduce the number of bits in the input of the multiplication, the difference value can be clipped to stay within the range that can be represented with fewer bits.

[0146]

number

[0147] where K is, for example, a clipping function.

[0148]

number

[0149] Or equivalently, K(d,C)=max(C,min(-C,d)), where C is the clipping parameter and d is the difference number. This formula assumes that the clipping parameter C is the same at each sample position, its value is fixed, and the multiplication input K(I (x+ui,y+vi) -I (x,y) It should be noted that this differs notably from the nonlinear ALF in VTM-5.0 in that the clipping parameter C is chosen to limit the number of bits to store ( , C), whereas in the nonlinear ALF the clipping parameter C changes at each sample position and its value is transmitted (in APS) along with the filter coefficients.

[0150] A particularly advantageous fixed value of C is 2 IBitDepth-2 -1 or 2 IBitDepth-3 It turns out that C is -1, and IBitDepth is the internal bit depth of the codec (i.e., the number of bits to represent a sample value). For example, with an internal bit depth of 10 bits, good values for C are 255 or 127. For the clipping operation in this example, we can save 2-3 bits at the input of the multiplication.

[0151] Although it is possible to reduce the value of C, this tends to reduce the coding efficiency of CCALF, and conversely, increasing the value reduces the effect of simplification through clipping processing. Therefore, the clipping parameter C is generally generalized to 2 IBitDepth-n -1, and n is an integer smaller than IbitDepth. When n is small, the loss in coding efficiency is small, but the benefit of simplification is also small. When n is large, the opposite is true.

[0152] Figure 6-b shows the clipped difference K(I (x+ui,y+vi) -I (x,y) , C) in a 16-bit integer memory space 602 (or a register).

[0153] The fourth variant can be combined with the third variant to obtain the clipped difference value K(I (x+ui,y+vi) -I (x,y) , C), it is possible to make the low-precision bits of the quantized data ...

[0154] Figure 6-c shows the K(I (x+ui,y+vi) -I (x,y) , C) is stored in the input of the multiplication. (x+ui,y+vi) -I(x,y) , C) instead of 10 bits, only 4 remaining bits of precision and 1 bit for sign 603 are needed. This represents a significant saving in the number of bits to be calculated, resulting in a simpler implementation.

[0155] In some embodiments, a combination of four variations is performed. For example, variations 1, 2, 3, and optionally 4 can be combined so that the output result of the multiplication operation can be represented with 10 precision bits plus 1 sign bit, and the sum of the multiplication results can be represented with up to 15 bits plus 1 sign bit (and therefore representable / storable in a 16-bit register or memory space). Such embodiments are particularly suitable for hardware implementation. Figures 7-a and 7-b show an example of an intermediate representation (multiplication output) and a final representation (sum result) of a filter operation before final rounding and bit shifting to add the result to the chroma sample value.

[0156] Figure 7-a retains only the upper 8 bits of the sample values (e.g., as in Figure 5-d), retains the fractional part of the coefficients with 7-bit precision, and f's coefficient range is [-0.03125;0.03125[ or ]-0.03125;0.03125[ (e.g., 2 bits + sign bit as in Figure 5-c) Cb (ui,vi) I (x+ui,y+vi) This is an example of bit-wise memory representation when storing the multiplication result, which can be stored in 10 bits + 1 sign bit.

[0157] Using equation (4) when the luma bit depth IBDL and the chroma bit depth IBDC are different, and equation (11) where DB = IBDL - KB (KB = 8 in this example) is used to set the highest precision bit count KB retained for the luma sample value, equation (3) can be rewritten as follows:

[0158]

number

[0159] Also, B3 = DB+(IBDL - IBDC)=2·IBDL - IBDC - KB, B4 = BDPC - B3, where BDPC is the number of bits of the fractional part of the coefficient (for example, in this example, the precision of the fractional part of the coefficient is 7 bits, so BDPC = 7), and f’ Cb (ui,vi) is the integer representation of the coefficient (that is, in this example, the integer value of the fixed-point decimal value in the range [-0.03125; 0.03125[ or [-0.03125; 0.03125[, and in this example, the integer value takes values in the range [-4; 3] or [-3; 3]). This formula is valid when KB ≤ 2·IBDL - IBDC ≤ BDPC + KB - 1. When KB ≤ 2·IBDL - IBDC and 2·IBDL - IBDC = BDPC + KB (that is, B4 = 0), formula (16) can be replaced as follows.

[0160]

Number

[0161] When KB ≤ 2·IBDL - IBDC and 2·IBDL - IBDC > BDPC + KB, formula (16) can be replaced as follows.

[0162]

Number

[0163] And when 2·IBDL - IBDC < KB, formula (16) can be replaced as follows.

[0164]

Number

[0165] In a given example, when KB = 8 and for the 10-bit luma and chroma bit depths (that is, IBDL = IBDC = 10), formula (16) can be rewritten as follows.

[0166]

Number

[0167] As a result, each coefficient is represented by 2 bits and a sign bit, and each sample is represented by a reduced 8 bits and optionally (depending on implementation / design choice) a sign bit (equal to 0), so that the multiplication result can be represented by 8 bits and a sign.

[0168] Figure 7-a retains only the upper 6 bits of the sample values (e.g., 2 upper bits than Figure 5-d), and the precision of the fractional part of the coefficients is 7 bits, and the coefficient range of [-0.125;0.125[ or ]-0.125;0.125[ (i.e., 4 bits + sign bit to store these, 2 more bits than Figure 5-c, for example) is used to store f' Cb (ui,vi) I (x+ui,y+vi) It is useful to provide an example of a bitwise memory representation for storing the multiplication result in 10 bits + 1 sign bit.

[0169] In this example, KB = 6. If the luma and chroma bit depth is 10 bits (i.e., IBDL = IBDC = 10), then equation (16) can be written as:

[0170]

number

[0171] Figure 7-b shows f' as explained in Figure 7-a. Cb (ui,vi) I (x+ui,y+vi) When expressed and calculated as Cb (ui,vi) I (x+ui,y+vi) This provides an example of a bitwise memory representation for calculating and storing . It is important to understand that this is the case when the filter performs 17 different additions out of 18 multiplication results, as shown in Figure 4-a. The worst case of the 17 additions (considering a 10-bit integer value storing fixed-point values) is the integer value -2. 10= 18 times -1024 (i.e. -18432 which can be stored in 15 bits and a sign bit), which means that the addition result can be expressed in 15 bits and a sign bit.

[0172] In a further (fifth) variation, the filter pattern of Figure 4-a is modified so that 16 or fewer samples are used in the filter process instead of 18. This saves one bit in the sum operation. Another advantage is that the multiplication operation is eliminated, simplifying the filter calculation. For example, instead of using 29 precision bits and 1 sign bit in Figure 4-b, only 28 precision bits and 1 sign bit are used, and in Figure 7-b, instead of using 15 precision bits, only 14 precision bits are used. In some variations using the fifth variation, the bits saved in the addition operation are used to increase the precision of the multiplication result. For example, in Figure 7-a, 11 precision bits are used instead of 10. In one variation, the fractional part of the 10 precision bits is increased by one bit (from 7 to 8). This can be achieved by adding one less precise bit to the fractional part of the coefficient value. For example, in Figure 5-c, the rightmost used bit is reused. Alternatively, when the third variation is used, this can be achieved by using one more precision bit for the sample value. For example, in Figure 5-d, the rightmost bit would be reused. A second alternative is to increase the integer part of the 10 precision bits by one bit (from 3 to 4 bits). This is achieved by adding one more precision bit to the coefficient values (i.e., increasing the range of possible coefficient values). For example, in Figure 5-c, the leftmost used bit would be reused.

[0173] These fifth variants are particularly interesting when using existing hardware with fixed-size registers, where the cost of multiplication is the same whenever the result fits into a register (e.g., in the case of a software implementation acting as an instruction to a CPU that processes operations that imply a register or memory location of a given size). They are also useful in hardware designs that use existing, predefined circuit templates to implement fixed-size operations, where unused bits are not a problem (e.g., development speed / cost issues). In this case, for example, when implemented with 16-bit input and 32-bit output, the cost (execution latency and number of logic gates used for implementation) of 10-bit and 11-bit multiplication remains the same, but the coding gain of CCALF is slightly better for the second version because it uses one more bit.

[0174] Alternatively, 8 samples (or fewer) can be used in the filter, saving 2 bits (or more) in the summation operation. See, for example, Figure 7-f, which uses 8 samples (saving 2 bits), or Figure 7-g, which uses only 4 samples (saving 3 bits). In this alternative variation, the 2 bits (or more) saved in total can be redistributed between the inputs of the multiplication, for example, during filtering, to have more precision and fully utilize the precision of the 16-bit registers.

[0175] Figure 7-c provides an example of a filter pattern 706 that uses only 16 samples for the filter. Using that filter pattern, the operation of the filter is as follows:

[0176] TIFF2025119008000022.tif34127

[0177] This selection of samples can be described as a "diamond" shape, with each sample having a position (x+u, y+v) relative to the reference sample, with u∈[-2,2] and v∈[-1,2], and v∈[0,1] when |u|=2.

[0178] Fig. 7-d shows another example of a filter pattern 707 using 16 samples for the filter. This pattern is more suitable than the one in Fig. 7-c when using YUV4:2:0 video representation, as the subsample positions of the chroma samples are located halfway between the positions of samples 5, 6, 9, and 10 in 707. According to one variant, two or more filter patterns are available, and one filter pattern is selected according to the chroma subsampling in YUV4:2:0. In one variant, the filter pattern index is provided / signaled in the bitstream, for example in a parameter set (e.g., SPS or PPS). With that filter pattern, the filter operation is as follows:

[0179] TIFF2025119008000023.tif34127

[0180] This selection of samples can be expressed as each sample having a position (x+u, y+v) relative to the reference sample, with u∈[-1,2] and v∈[-1,2].

[0181] Figure 7-e shows an example of a filter pattern 708 using 12 samples for the filter. This pattern allows access to fewer samples than the pattern in Figure 7-c, and therefore fewer coefficients to be coded in the bitstream, but does not further reduce the number of bits used for the summation operation. With this filter pattern, the filter operates as follows:

[0182] TIFF2025119008000024.tif34127

[0183] This selection of samples can be expressed as each sample having a position (x+u, y+v) relative to the reference sample, with u∈[-1,1] and v∈[-1,2].

[0184] Figure 7-f provides an example of a filter pattern 709 that uses only eight samples for the filter. This pattern can further reduce the number of bits used in the addition operation by one bit compared to the pattern in Figure 7-c, and by two bits compared to the pattern in Figure 4-a. With this filter pattern, the filter operates as follows:

[0185] TIFF2025119008000025.tif34127

[0186] This selection of samples can be represented as a "diamond", where each sample has position (x+u, y+v) relative to the reference sample, with u∈[-1,1], v∈[-1,2], and v[0,1] when |u|=1.

[0187] Figure 7-g provides an example of a filter pattern 710 that uses only four samples for the filter. This pattern allows for one more reduction in the number of bits used in the sum operation compared to the pattern in Figure 7-f, and a reduction of three bits compared to the pattern in Figure 4-a. With this filter pattern, the filter operation is as follows:

[0188] TIFF2025119008000026.tif7127

[0189] This selection of samples can be expressed as each sample having a position (x+u, y+v) relative to the reference sample, with both u and v in [0,1].

[0190] In some other variants, the cost of addition is essentially zero. The multiplication inputs are restricted so that the multiplication output / result is a 16-bit (i.e., 15 precision bits and 1 sign bit) signed number. Then, if 10 bits are used for the sample values, the coefficient values are restricted to a predetermined precision and a predetermined range so that they can be represented with 5 bits + 1 sign bit. In some of these variants, the multiplication operation is performed in a 16-bit register.

[0191] In one embodiment, the fractional precision of the coefficients is signaled in the bitstream as first precision information, for example in a parameter set (eg, APS, SPS, or PPS) or in a slice header.

[0192] In one embodiment, the range of the coefficient (or its index within the set of possible ranges) is signaled in the bitstream as second precision information, for example in a parameter set (e.g., APS, SPS or PPS) or in a slice header.

[0193] In one embodiment, the number of retained bits (i.e., KB) or alternatively the number of dead bits (i.e., DB) of a sample value is signaled in the bitstream as third precision information, for example in a parameter set (e.g., APS, SPS or PPS) or in a slice header.

[0194] In one embodiment, any one combination of the first, second, and / or third precision information is signaled in the bitstream, for example in a parameter set (e.g., APS, SPS, or PPS) or in a slice header.

[0195] In one variant, the tolerance of any such combination of first, second and / or third precision information signaled in the bitstream shall guarantee a worst-case that a multiplication operation can be calculated using a fixed maximum number of bits, such as, for example, 16. As a variant, the worst-case also takes into account the maximum number of bits used during an addition operation.

[0196] In one embodiment, an index in a set of possible values of any combination of the first, second and / or third accuracy information is signaled.

[0197] In one variation, such a set is defined to guarantee the worst case that a multiplication operation can be calculated using a fixed maximum number of bits, for example 16. In a variation, the worst case also considers the maximum number of bits used during an addition operation.

[0198] A variation on the embodiment in which any combination of the first, second and / or third precision information includes first and second precision information (i.e., decimal precision and holding range) is to signal the mantissa size (number of bits) and fixed exponent of the floating-point representation.

[0199] In one variant, instead of using a fixed-point representation for the filter coefficients, a specific floating-point representation is used, which is guaranteed to be calculated using a limited number of arithmetic bits. In other words, each coefficient index is signaled with any combination of first, second, and third precision information.

[0200] In the described examples, variations and / or embodiments, it will be understood that depending on the bit depth and precision of the sample values used, the range of the coefficients may be shifted up or down (i.e., multiplied by a power of 2) while shifting the least significant bits of the representation of the coefficients by the same amount (to maintain the same precision) to provide an equivalent mathematical result. That is, the [range] of the coefficient values (or sample values) is scaled by the inverse of the scaling of the sample values (or [range] of the coefficient values).

[0201]

number

[0202] This is also true for bitwise shifts: a left bitwise shift by B is 2 B is equivalent to multiplying by 2, and a right shift is equivalent to B is roughly equivalent to integer division by (the difference is that for negative numbers, left shifts round towards minus infinity, whereas integer division rounds towards zero).

[0203] It should also be understood that although the simplification of the bit representation of the sample values and filter coefficients has been done "during the filter" by the bit operations described above, it is also possible to pre-process these values individually before filtering.

[0204] signaling The input CCALF parameters 305 and 306 in Figure 3 may contain flags to enable / disable CCALF at different levels of granularity. An example of flag signaling is shown below:

[0205] In a specific embodiment, a flag is signaled in a parameter set, e.g., a sequence parameter set (SPS) or a picture parameter set (PPS). This flag enables or disables the CCALF tool at the sequence level or picture level. When the CCALF tool is enabled, a flag is signaled for each chroma component in the slice header. This allows for enabling or disabling CCALF for each chroma component in the slice. When CCALF is enabled in a slice (i.e., when CCALF is enabled for at least one chroma component), a flag ccalf_ctb_flag_cb[x][y] (for the Cb component) or ccalf_ctb_flag_cr[x][y] (for the Cr component) is coded in the CTU data of each CTU of the slice, where x, y are the horizontal and vertical CTU coordinates / indexes in the image. This flag signals whether CCALF is enabled or disabled in the chroma CTB. If CCALF is enabled in the chroma CTB, the chroma samples are corrected / filtered in CCALF (e.g., as described above) using the luma samples in the CTB.

[0206] Flags are signaled using CABAC (Context-Adaptive Binary Arithmetic Coding), with three CABAC contexts per chroma component. For a given chroma component with index componentIdx, a context is selected by counting the number of ccalf_ctb_flag_cb (for Cb component) or ccalf_ctb_flag_cr (for Cr component) contexts (if they exist / are available) that are active in the top ([x][y-1]) and left ([x-1][y]) CTUs that belong to the same slice, and using that value as the context index.

[0207] Using the same notation and rules as those in Versatile Video Coding (Draft 5) corresponding to VTM-5.0, an example of signaling ccalf_ctb_flag_cb[x][y] and ccalf_ctb_flag_cr[x][y] is shown in Table 1.

[0208] [Table 1] coding_tree_unit() syntax

[0209] According to an embodiment, to enable slice-specific adaptation and transmission of filter coefficients, the coefficients of the CCALF filter are signaled in the slice header, as is traditionally done in the slice parameters. If CCALF is indicated as enabled for a chroma component, each coefficient of the corresponding filter is coded / decoded sequentially in / from the bitstream.

[0210] CCALF coefficients are typically coded using Golomb variable-length codes. However, the coding can be simplified by reducing the number of bits used to represent the coefficients. Specifically, the number of bits required to store coefficient values is reduced, allowing for a coding method that is easier to analyze without sacrificing coding efficiency.

[0211] In particular, Golomb coding requires signaling an "adjustable parameter" (M) to correctly decode the coefficients, which increases bitstream overhead. The present invention relates to directly signaling coefficients in the slice header without reference to other parameters. In other words, the coefficients are coded using a fixed representation (i.e., without reference to an "adjustable parameter"). This can be described as "prefix-free" or "comma-free," in that the codeword is a single element with no delimiters (inferred or explicitly coded).

[0212] In a first alternative, which is particularly suitable for combining the first and second variants described above, the used / significant bits of the coefficient values are stored as is in the bitstream without compression (i.e., a fixed number of bits to represent the sign, whatever the value). For example, if the coefficients are represented as in Figure 5-c, the coefficients can be stored in three bits in the bitstream (e.g., a sign bit and two fractional precision bits).

[0213] As a second alternative, a kind of signed unary encoding is used to reduce the cost of low-amplitude coefficients. The first bit in the bitstream indicates whether the coefficient is zero; if it is not, a bit indicates the sign of the coefficient; and then a sequence of bits equal to 1 indicates the coefficient's amplitude. The number of bits equal to 1 is equal to the absolute value of the "integer representation of the coefficient" - 1. The integer representation of the coefficient is an integer value represented by 503 integer-valued bits. The "minus 1" is used because the information equal to zero is already encoded. And, if the absolute value is lower than the maximum possible / allowed value, a bit equal to zero indicates the end of signaling for that number. This corresponds to unary encoding of the absolute value minus 1. The decoding process for this representation can be summarized by the algorithm shown in Table 2.

[0214] [Table 2] Second Alternative Coefficient Decoding Algorithm

[0215] In some variations where an asymmetric range of possible coefficient values is used, if "CCALF_MAX_COEFF_VALUE" is not equal to "-CCALF_MIN_COEFF_VALUE" in Table 2, the maximum possible / allowed value of a number can be different for negative and positive numbers. For example, using the representation in Figure 5-c, if the range of possible coefficient values is [-0.03125;0.03125] (i.e., [-0.03125;0.0234375]), the maximum possible / allowed value is 4 for negative numbers (because -0.03125 is represented as the integer -4) and 3 for positive numbers (because 0.0234375 is represented as the integer 3). In this example, a 1 in the first bit means the number is 0, and a 1 in the sign bit means the number is negative. The representation of the coefficients in the bitstream is then as follows: "01111" for -0.03125 (i.e., -4 = CCALF_MIN_COEFF_VALUE), "01110" for -0.0234375 (i.e., -3), "0110" for -0.015625 (i.e., -2), "010" for -0.0078125 (i.e., -1), "1" for 0.0, "000" for 0.0078125 (i.e., 1), "0010" for 0.015625 (i.e., 2), and "0011" for 0.0234375 (i.e., 3 = CCALF_MAX_COEFF_VALUE). In some variations, a first bit equal to 1 may mean that the number is not zero. In some variations, a sign bit equal to 1 may mean that the number is positive. In some variations, the unary encoding of a number may be a bit sequence equal to 0 followed by a bit sequence equal to 1 (if the number is lower than the maximum value).

[0216] As a third alternative, the bitstream first performs a unary encoding of the integer representation of the coefficient's absolute value, followed by a sign bit if non-zero. The algorithm in Table 3 shows an example of the decoding process for this representation. In this third alternative, the sign cannot be known before decoding the coefficient's absolute value. Furthermore, the maximum value used for the unary encoding is the same for positive and negative values (CCALF_MAX_COEFF_ABSOLUTE_VALUE in Table 3). As an example, the representation in Figure 5-c uses a range of possible coefficient values set equal to [-0.0234375;0.0234375] (i.e., CCALF_MAX_COEFF_ABSOLUTE_VALUE = 3). The representation of the coefficients in the bitstream is then: "1111" (i.e., -3) for -0.0234375, "1101" (i.e., -2) for -0.015625, "101" (i.e., -1) for -0.0078125, "0" for 0.0, "100" (i.e., 1) for 0.0078125, "1100" (i.e., 2) for 0.015625, and "1110" (i.e., 3) for 0.0234375.

[0217] [Table 3] Third Alternative Coefficient Decoding Algorithm

[0218] In some variants, the coefficients of the CCALF filter are signaled in the APS (or another parameter set). When slice_ccalf_enable_flag_cb is equal to 1 in the slice header, the slice header also provides the index / identifier of the APS (or other parameter set) where the coefficients of the CCALF filter for the Cb chroma component must be retrieved. When slice_ccalf_enable_flag_cr in the slice header is 1, the slice header also provides the index of the APS (or other parameter set) where the filter coefficients of the CCALF filter for the Cr chroma component must be obtained. When using a filter pattern such as that shown in Figure 4-a, the number of filter coefficients present in the bitstream is less than the number of samples used for the filter shape because some coefficients are shared with some sample positions.

[0219] About the embodiment of the present invention One or more of the above embodiments may be implemented in the form of an encoder or decoder that performs the method steps of the above one or more embodiments. The following embodiments illustrate such implementations.

[0220] For example, an adaptive loop filter according to any of the above embodiments may be used in the post filter 9415 performed by the encoder of FIG. 8 or the post filter 9567 performed by the decoder of FIG.

[0221] 8 is a block diagram of an encoder according to one embodiment of the present invention, the encoder being represented by connected modules, each module adapted to perform, e.g., in the form of programming instructions executed by a central processing unit (CPU) of the device, at least one corresponding step of a method for implementing at least one embodiment of encoding images of an image sequence according to one or more embodiments of the present invention.

[0222] An original sequence of digital images i0 to in9401 is received as input by the encoder 9400. Each digital image is represented by a collection of samples, sometimes called pixels (hereafter referred to as pixels). After performing the encoding process, the encoder 9400 outputs a bitstream 9410. The bitstream 9410 contains data for image portions such as multiple coding units or slices, each of which contains a slice header for transmitting coded values of coding parameters used to code the slice, and a slice body consisting of coded video data.

[0223] The input digital image i0 to in 9401 is divided by module 9402 into blocks of pixels. The blocks correspond to image portions and may be of variable size (e.g., 4x4, 8x8, 16x16, 32x32, 64x64, 128x128 pixels, and several rectangular block sizes are also considered). A coding mode is selected for each input block. There are two coding methods: spatial predictive coding (intra prediction) and temporal predictive coding (inter coding, MERGE, SKIP). Possible coding modes are tested.

[0224] The module 9403 performs an intra prediction process in which a given block to be coded is predicted by a predictor calculated from pixels in the neighborhood of the block to be coded. The difference between the representation of the selected intra predictor and the given block and its predictor is coded to provide a residual if intra coding is selected.

[0225] Temporal prediction is performed by a motion estimation module 9404 and a motion compensation module 9405. First, a reference image is selected from a set of reference images 9416, and the motion estimation module 9404 selects a portion of the reference image that is closest to a given block to be coded (closest in terms of pixel value similarity), also called a reference region or image portion. The motion compensation module 9405 then uses the selected region to predict the block to be coded. The difference between the selected reference region and a given block, also called a residual block / data, is calculated by the motion compensation module 9405. The selected reference region is indicated using motion information (e.g., a motion vector).

[0226] Thus, in both cases (spatial and temporal prediction), the residual is calculated by subtracting the predictor from the original block when not in SKIP mode.

[0227] In the intra prediction performed by module 9403, the prediction direction is coded. In the inter prediction performed by modules 9404, 9405, 9416, 9418, 9417, at least one motion vector or information (data) for identifying the motion vector is coded for temporal prediction.

[0228] If inter prediction is selected, the motion vector and information related to the residual block are coded. To further reduce the bit rate, the motion vector is coded by the difference with the motion vector predictor, assuming that the motion is homogeneous. The motion vector predictor from the set of motion information predictor candidates is obtained from the motion vector field 9418 by the motion vector prediction and coding module 9417.

[0229] The encoder 9400 further comprises a selection module 9406 for applying a coding cost criterion, such as a rate-distortion criterion, to select a coding mode. To further reduce redundancy, the residual block is subjected to a transform (e.g., DCT) by a transform module 9407, and the resulting transformed data is quantized by a quantization module 9408 and entropy coded by an entropy coding module 9409. Finally, if the selected coding mode is not SKIP mode and requires coding of the residual block, the residual block of the current block being coded is inserted into the bitstream 9410.

[0230] The encoder 9400 also performs decoding of the coded image to generate reference images (e.g., those in the reference image / picture 9416) for motion estimation of subsequent images. This allows the encoder and decoder receiving the bitstream to have the same reference frame (e.g., a reconstructed image or reconstructed image portion is used). An inverse quantization ("dequantization") module 9411 performs inverse quantization ("dequantization") of the quantized data, after which an inverse transform is performed by an inverse transform module 9412. An intra prediction module 9413 uses the prediction information to decide which predictor to use for a given block, and a motion compensation module 9414 actually adds the residual obtained by module 9412 to a reference region obtained from the set of reference images 9416. A postfilter is then applied by module 9415 to filter a reconstructed frame (image or image portion) of multiple pixels to obtain another reference image for the set of reference images 9416.

[0231] 9 is a block diagram of a decoder 9560 that may be used to receive data from an encoder according to an embodiment of the present invention. The decoder is represented by connected modules, each module adapted to perform a corresponding step of a method performed by the decoder 9560, e.g., in the form of programming instructions executed by the device's CPU.

[0232] The decoder 9560 receives a bitstream 9561 of coded units (e.g., data corresponding to image portions, blocks, or coding units), each including a header containing information about coding parameters and a body containing coded video data. With reference to Figure 8, the coded video data is entropy coded, where motion information (e.g., a motion vector predictor index) is coded with a predetermined number of bits for a given image portion (e.g., a block or CU). The received coded video data is entropy decoded by module 9562. The residual data is then dequantized by module 9563, after which an inverse transform is applied by module 9564 to obtain pixel values.

[0233] The encoder also entropy-decodes mode data indicating the coding mode, and performs intra- or inter-decoding on the block (unit / set / group) of coded image data based on the mode. In the case of an intra-mode, the intra-predictor is determined by the intra-prediction module 9565 based on the intra-prediction mode specified in the bitstream (e.g., the intra-prediction mode can be determined using data provided in the bitstream). In the case of an inter-mode, motion prediction information is extracted / obtained from the bitstream to find (identify) the reference region to be used by the encoder. The motion prediction information is, for example, composed of a reference frame index and a motion vector residual. The motion vector prediction information is added to the motion vector residual by the motion vector decoding module 9570 and used to obtain a motion vector.

[0234] The motion vector decoding module 9570 applies motion vector decoding to each image portion (e.g., current block or CU) coded by motion prediction. Once the index of the motion vector predictor for the current block is obtained, the actual value of the motion vector associated with the image portion (e.g., current block or CU) can be decoded and used by module 9566 to apply motion compensation. The reference image portion indicated by the decoded motion vector is extracted / obtained from the set of reference images 9568 so that module 9566 can perform motion compensation. The motion vector field data 9571 is updated with the decoded motion vector for use in predicting subsequent decoded motion vectors.

[0235] Finally, a decoded block is obtained, where a post filter is applied, if appropriate, by a post filter module 9567. A decoded video signal 9569 is finally obtained and provided by the decoder 9560.

[0236] 10 illustrates a data communication system in which one or more embodiments of the present invention may be implemented. The data communication system includes a transmitting device (in this case, a server 9201) operable to transmit data packets of a data stream 9204 to a receiving device (in this case, a client terminal 9202) via a data communication network 9200. The data communication network 9200 may be a wide area network (WAN) or a local area network (LAN). Such a network may be, for example, a wireless network (Wifi / 802.11a or b or g), an Ethernet network, an Internet network, or a hybrid network consisting of multiple different networks. In a particular embodiment of the present invention, the data communication system may be a digital television broadcasting system in which a server 9201 transmits the same data content to multiple clients.

[0237] The data stream 9204 provided by the server 9201 may consist of multimedia data representing video and audio data. The audio and video data streams may be captured by the server 9201 using a microphone and a camera, respectively, in some embodiments of the present invention. In some embodiments, the data streams may be stored on the server 9201, received by the server 9201 from other data providers, or generated by the server 9201. The server 9201 includes, among other things, an encoder for encoding the video and audio streams to provide a compressed bitstream for transmission that is a more compact representation of the data presented as input to the encoder. To obtain a better ratio of the quality of the transmitted data to the amount of transmitted data, the compression of the video data may be according to, for example, the High Efficiency Video Coding (HEVC) format, the H.264 / Advanced Video Coding (AVC) format, or the Versatile Video Coding (VVC) format. The client 9202 receives the transmitted bitstream, decodes the reconstructed bitstream, and reproduces the video images on a display device and the audio data on a loudspeaker.

[0238] Although the present embodiment considers a streaming scenario, it will be appreciated that in some embodiments of the present invention, data communication between the encoder and decoder may be performed using a media storage device, such as an optical disk. In one or more embodiments of the present invention, a video image may be transmitted along with data representative of a compensation offset to apply to reconstructed pixels of the image to provide filtered pixels in the final image.

[0239] 11 is a schematic diagram of a processing device 9300 configured to implement at least one embodiment of the present invention. The processing device 9300 may be a device such as a microcomputer, a workstation, a user terminal, or a lightweight handheld device. The device / apparatus 9300 may: - a central processing unit 9311 (denoted as CPU), such as a microprocessor; a read-only memory 9307, denoted ROM, for storing computer programs / instructions for the operation of the device 9300 and / or for the implementation of the invention; a random access memory 9312, called RAM, for storing the executable code of the methods of the embodiments of the invention and registers adapted to record variables and parameters necessary for implementing the methods of encoding a sequence of digital images and / or decoding a bitstream according to the embodiments of the invention; and a communication interface 9302 connected to a communication network 9303 over which digital data to be processed is transmitted or received; 9313 connected to the communication bus 9313.

[0240] Optionally, the apparatus 9300 may also include the following components: data storage means 9304, such as a hard disk, for storing computer programs for implementing the methods of one or more embodiments of the present invention and data used or generated during the implementation of one or more embodiments of the present invention; a disk drive 9305 for a disk 9306 (e.g., a storage medium), the disk drive 9305 being adapted to read data from or write data to the disk 9306, or - A screen 9309 for displaying data and / or serving as a graphical interface with the user, via a keyboard 9310, touch screen, or other pointing / input means.

[0241] The device 9300 can be connected to a variety of peripheral devices, such as a digital camera 9320 or a microphone 9308, each connected to an input / output card (not shown) to provide multimedia data to the device 9300.

[0242] The communication bus 9313 provides for communication and interoperability between the various elements included in or connected to the device 9300. The representation of the bus is not limiting, and in particular the central processing unit 9311 is operable to communicate instructions to any element of the device 9300 directly or by another element of the device 9300.

[0243] The disk 9306 may be replaced by any information medium, for example a compact disk (CD-ROM), rewritable or not, a ZIP disk or a memory card, and in general any information storage means readable by a microcomputer or processor, whether incorporated in the device or not, possibly removable, and adapted to store one or more programs, the execution of which enables the method of encoding a digital image sequence and / or the method of decoding a bitstream according to the present invention to be carried out.

[0244] The executable code may be stored either in a read-only memory 9307, on the hard disk 9304 or on a removable digital medium as previously mentioned, such as for example the disk 9306. According to a variant, the executable code of the program may be received by the communication network 9303, via the interface 9302, to be stored in one of the storage means of the device 9300 (for example the hard disk 9304) before being executed.

[0245] The central processing unit 9311 is adapted to control and direct the execution of instructions or parts of the software code of a program or program according to the invention, instructions stored in one of the aforementioned storage means. On power-up, the program or program stored in a non-volatile memory, for example on the hard disk 9304, on the disk 9306 or in the read-only memory 9307, is transferred to the random access memory 9312, which then stores the executable code of the program or program, as well as registers for storing variables and parameters necessary to implement the invention.

[0246] In this embodiment, the device is a programmable device that implements the invention using software, however, the invention may alternatively be implemented in hardware (e.g., in the form of an application specific integrated circuit (ASIC)).

[0247] It will also be appreciated that, according to other embodiments of the present invention, the decoder according to the aforementioned embodiments is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet, or any other type of device (e.g., a display device) that can provide / display content to a user. According to yet another embodiment, the encoder according to the aforementioned embodiments is provided in an imaging device, including also a camera, a video camera, or a network camera (e.g., a closed-circuit television or video surveillance camera), that captures and provides content for the encoder to encode. Two such embodiments are provided below with reference to Figures 12 and 13.

[0248] FIG. 12 is a diagram illustrating a network camera system 9450 including a network camera 9452 and a client device 9454 .

[0249] The network camera 9452 includes an imaging unit 9456, an encoding unit 9458, a communication unit 9460, and a control unit 9462. The network camera 9452 and the client device 9454 are communicatively connected to each other via the network 9200. The imaging unit 9456 includes a lens and an image sensor (e.g., a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)) to capture an image of an object and generate image data based on the image. The image may be a still image or a video. The imaging unit may also include zooming means and / or panning means adapted to zoom or pan (optically or digitally), respectively. The encoding unit 9458 encodes the image data using the encoding method described in one or more of the previous embodiments. The encoding unit 9458 uses at least one of the encoding methods described in the previous embodiments. As another example, the encoding unit 9458 may use a combination of the encoding methods described in the previous embodiments.

[0250] The communication unit 9460 of the network camera 9452 transmits the encoded image data encoded by the encoding unit 9458 to the client device 9454. Furthermore, the communication unit 9460 may receive commands from the client device 9454. The commands are configured to set parameters for encoding by the encoding unit 9458. The control unit 9462 controls other units within the network camera 9452 in accordance with the commands received by the communication unit 9460 and user input.

[0251] The client device 9454 includes a communication unit 9464, a decoding unit 9466, and a control unit 9468. The communication unit 9464 of the client device 9454 may send commands to the network camera 9452. Furthermore, the communication unit 9464 of the client device 9454 receives encoded image data from the network camera 9452. The decoding unit 9466 decodes the encoded image data using the decoding method described in one or more of the above embodiments. As another example, the decoding unit 9466 may use a combination of the decoding methods described in the above embodiments. The control unit 9468 of the client device 9454 controls other units within the client device 9454 according to a user operation or a command received by the communication unit 9464. The control unit 9468 of the client device 9454 may also control the display device 9470 to display the image decoded by the decoding unit 9466.

[0252] The control unit 9468 of the client device 9454 may also control the display device 9470 to display a GUI (Graphical User Interface) for specifying parameters of the network camera 9452, for example, parameter values for encoding by the encoding unit 9458. The control unit 9468 of the client device 9454 may also control other units within the client device 9454 in accordance with a user's operation input to the GUI displayed by the display device 9470. The control unit 9468 of the client device 9454 may also control the communication unit 9464 of the client device 9454 to send to the network camera 9452 a command specifying parameter values for the network camera 9452 in accordance with a user's operation input to the GUI displayed by the display device 9470.

[0253] 13 is a diagram illustrating a smartphone 9500. The smartphone 9500 includes a communication unit 9502, a decoding / encoding unit 9504, a control unit 9506, and a display unit 9508.

[0254] The communication unit 9502 receives encoded image data via the network 9200. The decoding / encoding unit 9504 decodes the encoded image data received by the communication unit 9502. The decoding / encoding unit 9504 decodes the encoded image data using the decoding method described in one or more of the above embodiments. The decoding / encoding unit 9504 can also use at least one of the encoding methods or decoding methods described in the above embodiments. As another example, the decoding / encoding unit 9504 can use a combination of the decoding methods or encoding methods described in the above embodiments.

[0255] The control unit 9506 controls other units in the smartphone 9500 in accordance with a user operation or a command received by the communication unit 9502. For example, the control unit 9506 controls the display unit 9508 to display an image decoded by the decoding / encoding unit 9504.

[0256] The smartphone may further include an image recording device 9510 (e.g., a digital camera and associated circuitry) for recording images or video. Such recorded images or video may be encoded by the decoding / encoding unit 9504 under the direction of the control unit 9506. The smartphone may further include a sensor 9512 adapted to sense the orientation of the mobile device. Such a sensor may include an accelerometer, gyroscope, compass, global positioning (GPS) unit, or similar position sensor. Such a sensor 9512 may determine whether the smartphone has changed orientation, and such information may be used when encoding the video stream.

[0257] While the present invention has been described with reference to embodiments, it will be understood that the present invention is not limited to the disclosed embodiments. Those skilled in the art will recognize that various changes and modifications can be made without departing from the scope of the present invention, as defined in the appended claims. All features disclosed in this specification (including the accompanying claims, abstract, and drawings), and / or all steps of any method or process so disclosed, may be combined in any combination, except where at least some of such features and / or steps are mutually exclusive. Each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each disclosed feature is only an example of a generic series of equivalent or similar features.

[0258] It will also be understood that any result of the above-mentioned comparison, determination, evaluation, selection, execution, performing, or consideration (e.g., selection made during encoding or filtering) may be indicated in or determinable / referenceable from data in the bitstream, such as a flag or information indicating the result, and the indicated or determined / referenced result may be used in processing instead of actually performing the comparison, determination, evaluation, selection, execution, performing, or consideration, e.g., during the decoding process.

[0259] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" and "an" do not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage. Reference numerals appearing in the claims are for illustrative purposes only and shall not have a limiting effect on the scope of the claims.

[0260] In the above embodiments, the described functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit.

[0261] Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communications protocol. Thus, computer-readable media generally can correspond to (1) tangible computer-readable storage media that are non-transitory, or (2) communication media such as signals or carrier waves. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

[0262] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, or microwave are also included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, and instead refer to non-transitory tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and disks reproduce data optically using lasers, and combinations of the above are also intended to be included within the scope of computer-readable media.

[0263] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein, may refer to any of the foregoing structures or other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Also, the techniques may be implemented entirely within one or more circuits or logic elements.

Claims

1. 1. A method for implementing a cross-component adaptive loop filter, comprising: obtaining a plurality of sample values ​​of the first component corresponding to a plurality of samples around a reference sample; obtaining a plurality of filter coefficients from the bitstream; generating a cross-component adaptive loop filter output for a second component different from the first component using the plurality of filter coefficients and the plurality of sample values ​​of the first component; Including, the filter coefficients are represented in the bitstream without a prefix; the values ​​of the plurality of filter coefficients are limited to a plurality of values ​​defined using a specified number of bits within a range expressed by values ​​of a power of 2 defined using the specified number of bits; the plurality of sample values ​​of the first component are sample values ​​at a plurality of positions defined relative to the reference sample, the plurality of positions being defined by a filter shape; The filter shape contains eight or fewer samples. method.

2. The specified number of bits is smaller than the number of bits defined by the bit depth. The method of claim 1.

3. The specified number of bits is a value that makes the output of the cross-component adaptive loop filter 16 bits or less. The method of claim 1.

4. The method of claim 3, further comprising: combining the output of the cross-component adaptive loop filter with the output of a filter corresponding to a different component. The method of claim 1.

5. The first component is a luma component and the second component is a chroma component. The method of claim 1.

6. The output is generated using a numerical value representing a difference between a sample value included in the plurality of sample values ​​of the first component and a value of a reference sample. The method of claim 1.

7. The output is generated by multiplying a sample value included in the plurality of sample values ​​of the first component by a filter coefficient included in the plurality of filter coefficients. The method of claim 1.

8. The filter shape includes a pattern in which a plurality of samples of the first component have positions (x+u, y+v) relative to the reference sample at position (x, y), where u is a horizontal offset in [−1, 1] and v is a vertical offset in [−1, 2]. The method of claim 1.

9. When |u| = 1, v corresponds to [0, 1] The method of claim 8.

10. The method of claim 1, further comprising determining whether a cross-component adaptive loop filter is implemented. The method of claim 1.

11. An apparatus for implementing a cross-component adaptive loop filter, comprising: a first acquiring means for acquiring a plurality of sample values ​​of a first component corresponding to a plurality of samples around a reference sample; second obtaining means for obtaining a plurality of filter coefficients from the bitstream; generating means for generating an output of a cross-component adaptive loop filter for a second component different from the first component, using the plurality of filter coefficients and the plurality of sample values ​​of the first component; and the filter coefficients are represented in the bitstream without a prefix; the values ​​of the plurality of filter coefficients are limited to a plurality of values ​​defined using a specified number of bits within a range expressed by values ​​of a power of 2 defined using the specified number of bits; the plurality of sample values ​​of the first component are sample values ​​at a plurality of positions defined relative to the reference sample, the plurality of positions being defined by a filter shape; The filter shape contains eight or fewer samples. Device.

12. The specified number of bits is smaller than the number of bits defined by the bit depth.

12. The apparatus of claim 11.

13. The first component is a luma component and the second component is a chroma component.

12. The apparatus of claim 11.

14. The output is generated using a numerical value representing a difference between a sample value included in the plurality of sample values ​​of the first component and a value of a reference sample.

12. The apparatus of claim 11.

15. A program that causes a computer to execute a method according to any one of claims 1 to 10.

Images