Method and apparatus for filtering image parts, method for encoding or decoding image, and storage medium

Abstract

The present invention relates to a method and an apparatus for filtering an image portion, a method for encoding or decoding an image, and a storage medium. The method includes receiving filter coefficients and first component sample values corresponding to samples around a reference sample, and inputting the filter coefficients and the first component sample values into a cross-component filter to produce a cross-component filter output; wherein the cross-component filter uses a constrained number of bits to represent the filter coefficients and / or the first component sample values to produce the filter output.

Description

[0001] (This application is a divisional application of an application filed on August 18, 2020, with application number 2020800584073 and invention name “CROSS-COMPONENT ADAPTIVE LOOP FILTER FOR CHROMA”.) Technical Field

[0002] The invention relates to encoding or decoding of blocks of video components. Embodiments of the invention find particular but not exclusive use in controlling filters used to filter samples of such components. In particular but not exclusively, controlling an adaptive loop filter. Background Art

[0003] Video coding includes image coding (image is equivalent to a single frame of video). In video coding, some coding tools such as quantization of transform coefficients or motion compensation (usually performed using interpolation filters) often introduce distortion deviations / effects (distortions that appear to be orderly or at least non-random in a given context). In order to compensate for these deviations / artifacts and improve (or at least maintain a good level of) coding efficiency, some specific coding tools called post filters or in-loop filters are used. Deblocking filters (DBF), sample adaptive offset (SAO) filters or adaptive loop filters (ALF) are some examples of such coding tools. In-loop filters are applied within the coding loop, so that the image quality they provide to the current frame is increased, which enables the coding efficiency of subsequent frames encoded based on the current frame to be increased. For example, quantization of DCT coefficients is efficient for video compression, but it often introduces block artifacts (deviations) on the boundaries of compressed sample blocks. Deblocking filters reduce the undesirable effects caused by such artifacts. Deblocking a decoded frame (using DBF) inside the coding loop (before the frame is used as a reference frame for motion compensation of another frame) significantly increases the coding efficiency of motion compensation compared to deblocking the frame outside the coding loop (e.g., just before displaying the frame).

[0004] The present invention particularly relates to an adaptive loop filter (ALF), which is also used as an in-loop filter to reduce undesirable compression artifacts in decoded frames / images. ALF has been studied by the Video Coding Experts Group / Moving Picture Experts Group (VCEG / MPEG) standardization group and is being considered for use in the Versatile Video Coding (VVC) standard, for example in the fifth version of the VVC Test Model software (VTM-5.0 or VVC Draft Version 5). Cross-component ALF (CCALF) was proposed in JVET-O0636 of the 15th meeting of the Joint Video Experts Group (JVET) (Gothenburg, Sweden, July 3-12, 2019), which is a complementary filter to ALF that uses the sample values ​​of the luminance channel to estimate additional error correction of artifacts performed on the sample values ​​of the chrominance channels. CCALF can be applied on each chrominance channel. Summary of the invention

[0005] The present invention is concerned with improving the coding performance of a portion of the CCALF process by simplifying the cross component filters.

[0006] Embodiments of the invention relate to using a reduced bit depth representation (compared to the internal bit depth) of either or both of the filter coefficients and the cross-component input samples. This constraint can be made by bit shifting operations and / or ignoring certain bits of the filter coefficients / input samples.

[0007] In one aspect of the present invention, a method for filtering a portion of an image is provided, the method comprising: receiving filter coefficients and first component sample values ​​corresponding to samples surrounding a reference sample, and inputting the filter coefficients and first component sample values ​​into a cross-component filter to generate a cross-component filter output; wherein the cross-component filter uses a constrained number of bits to represent the filter coefficients and / or first component sample values ​​to generate the filter output.

[0008] In this way, the filtering computations are simplified and surprisingly most of the CCALF coding improvements are maintained.

[0009] Optionally, the constrained number of bits is lower than the number of bits defined by the internal bit depth.

[0010] For efficient implementation, the constrained number of bits used to represent the filter coefficients and / or first component sample values ​​is such that the filter output may be represented with 16 bits or less.

[0011] Optionally, the method further comprises combining the cross-component filter output and an output of a filter corresponding to a different component. Optionally, the first component is a luma component and the different component is a chroma component.

[0012] For efficient implementation, the method includes performing a bit shift operation to reduce the number of bits before the filter multiplication operation. This allows for simplified filter calculations.

[0013] Constraining sample values

[0014] In one aspect of the invention, the number of bits used to represent sample values ​​is reduced.

[0015] Optionally, according to a third variant, the constraint consists in reducing the precision of the sample values ​​by considering only certain bits.

[0016] Optionally, a predefined number of least significant bits are ignored; for example, the least significant bit, the least significant 2 bits, the least significant 3 bits, or the least significant 4 bits.

[0017] Optionally, a predefined number of most significant bits are retained; for example, the highest 8, 7, 6, 5, or 4 bits.

[0018] To further simplify, the method may include rounding the sample value using the most significant removed bit.

[0019] Optionally, according to a fourth variant, inputting the sample value into the filter includes inputting a number representing the difference between the value of the first component sample value and the reference sample value.

[0020] Optionally, the method further includes clipping the difference based on a clipping function defined by a fixed clipping parameter C.

[0021] The clipping function K can be expressed as:

[0022] K(d,C) = max(-C, min(C,d))

[0023] where C is the clipping parameter and d is the number representing the difference. Thus, the input to the clipping function is the number representing the difference and the clipping parameter, and the output is the maximum of -C and the smaller value between C and d.

[0024] In an advantageous embodiment, C is 2 (IBitDepth-n) -1, where IBitDepth is the number of bits used to represent the sample value and n is an integer satisfying n < IBitDepth. Optionally, n = 2 or 3.

[0025] Constrained coefficient values

[0026] In one aspect of the present invention, the number of bits used to represent the filter coefficient values is reduced.

[0027] Optionally, according to a second variant, the constraint includes restricting the range of possible values of the filter coefficients. This allows the coefficients to be represented with fewer bits and makes subsequent calculations simpler.

[0028] Optionally, according to a first variant, the constraint includes using input coefficient values with reduced fixed-point fractional precision.

[0029] Embodiments with specific advantages include that the reduced fixed-point fractional precision is one of 8, 7, and 6 bits.

[0030] In an alternative embodiment, restricting the range includes not using any bits higher than a threshold; for example, the threshold is the 7th, 6th, 5th, 4th, or 3rd bit of the fixed-point fractional precision.

[0031] In an advantageous implementation of the invention, each multiplication is replaced by a bitwise shift and addition operation (which is easier to implement in both hardware and software). Thus, the filtering operation may include a multiplication of a sample value and a filter coefficient, and the multiplication is implemented via a bitwise shift and / or a combination with another filter coefficient to replace the multiplication operation by a bitwise shift and addition operation.

[0032] Filter shape

[0033] In another aspect of the invention, a modification of the filter shape is proposed. Such a modification consists in reducing the number of samples to be considered (and thus simplifying the result calculation) while maintaining the gain provided by the CCALF.

[0034] Optionally, each first component sample value is a sample value at a position at a defined position relative to a reference sample, the position of the sample being defined by the filter shape.

[0035] For ease of implementation, the filter shape is such that it comprises 16 samples or less.

[0036] In one embodiment, the filter shape comprises a pattern wherein said first component samples have positions (x+u, y+v) relative to said reference samples, wherein u and v belong to the integer interval [-2, 2].

[0037] In another embodiment, the filter shape comprises a pattern in which each sample has a position (x+u, y+v) relative to the reference sample, where u belongs to [-2, 2] and v belongs to [-1, 2]. In a variant, the filter is a "diamond shape"; for example, where v belongs to [0, 1] when |u|=2.

[0038] In one embodiment, the filter shape comprises a pattern in which each sample has a position (x+u, y+v) relative to the reference sample, where u belongs to [-1, 1] and v belongs to [-1, 2]. In a variant, the filter is a "diamond shape"; for example, when |u|=1, v belongs to [0, 1].

[0039] In another embodiment, the filter shape comprises a pattern where each sample has a position (x+u, y+v) relative to said reference sample, where u, v belongs to [0, 1].

[0040] signalling

[0041] Embodiments of the invention also relate to signaling CCALF filter coefficients in the bitstream.It is an object of the invention to improve the signaling to minimize the bitrate cost associated with CCALF.

[0042] Optionally, the method further comprises signaling in the bitstream the reduced number of bits of the filter coefficients and / or sample values.This allows consistent processing of different images and, if required, flexibility in processing.

[0043] According to one aspect of the invention, the method further comprises: determining whether a cross-component adaptive loop filter (CCALF) is indicated for a given component; and decoding / or encoding the filter coefficients from a bitstream.

[0044] According to another aspect of the present invention, a method for decoding an image portion encoded using CCALF is provided, the method comprising: determining whether CCALF is indicated for a given component; decoding CCALF coefficients from a bitstream; and decoding the image portion using the CCALF coefficients; wherein the CCALF coefficients are encoded into the bitstream in a prefix-free manner.

[0045] This aspect represents a simplification over the prior art, and achieves lower bit rates without significantly compromising CCALF performance (particularly when combined with reducing the number of bits used to represent the filter coefficients as described herein).

[0046] Optionally, determining whether CCALF is indicated for a given component comprises decoding a flag.

[0047] According to another aspect of the present invention, a method for encoding an image portion using a cross-component adaptive loop filter (CCALF) is provided, the method comprising: determining whether a CCALF is indicated for a given component; encoding CCALF coefficients into a bitstream; and encoding the image portion using the CCALF coefficients; wherein the CCALF coefficients are encoded into a slice header using a fixed representation.

[0048] Optionally, said determining whether CCALF is indicated for a given component comprises encoding a flag into said slice header.

[0049] In a first alternative, for ease of implementation, the CCALF coefficients may be encoded directly in the slice header.

[0050] The CCALF coefficients may be encoded in the bitstream using signed unary coding. Such a coding scheme is easy to encode and decode, and does not rely on other parameters (e.g., prefixes) to do so.

[0051] Optionally, according to a third alternative, the signed unary encoding comprises: a bit indicating the sign of the coefficient; and a series of bits equal to 1 indicating the magnitude of the coefficient.

[0052] Optionally, according to a second alternative, the signed unary encoding comprises: a first bit in the bit stream indicating whether the coefficient is zero; a bit indicating the sign of the coefficient (if not the coefficient is non-zero), and a sequence of bits equal to 1 (indicating the magnitude of the coefficient).

[0053] Optionally, the first bit equal to 1 indicates that the coefficient is zero.

[0054] Optionally, a sign bit equal to 1 indicates that the coefficient is negative.

[0055] Optionally, for ease of parsing, the signed unary encoding also includes a bit equal to zero at the end of the sequence, indicating the signaled end of the coefficient.

[0056] Yet another aspect of the present invention relates to an apparatus for filtering an image.

[0057] Yet another aspect of the invention relates to an encoder and a decoder.

[0058] Further aspects of the invention relate to a program. The program may be provided separately, or may be carried on, by or in a carrier medium. The carrier medium may be non-transitory, such as a storage medium, in particular a computer-readable storage medium. The carrier medium may also be transient, such as a signal or other transmission medium. The signal may be transmitted via any suitable network, including the Internet.

[0059] Further characteristics of the invention are characterized by the other independent and dependent claims.

[0060] Any features in one aspect of the invention may be applied to other aspects of the invention in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa.

[0061] Furthermore, features implemented in hardware may be implemented in software and vice versa.Any references herein to software and hardware features should be interpreted accordingly.

[0062] Any apparatus features as described herein may also be provided as method features, and vice versa.As used herein, means-plus-function features may alternatively be represented in terms of their corresponding structure, such as a suitably programmed processor and associated memory or the like.

[0063] It should also be understood that specific combinations of the various features described and defined in any aspect of the present invention may be independently implemented and / or provided and / or used. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Embodiments of the present invention will now be described, by way of example only, and with reference to the following drawings, in which:

[0065] Figure 1 Shows where ALF occurs in a typical decoding loop of VTM-5.0;

[0066] Figure 2 is a flow chart providing an overview of ALF filtering in VTM-5.0;

[0067] Figure 3 In addition to providing an overview of VTM-5.0's ALF, a flowchart also provides an overview of CCALF filtering;

[0068] Figure 4-a Providing a filter shape and coefficient arrangement for a CCALF filter according to an embodiment of the present invention;

[0069] Figure 4-b shows a typical bit-by-bit memory representation of CCALF filter coefficients;

[0070] Figure 4-c shows a typical bit-by-bit memory representation of sample values ​​used during CCALF filtering with an internal bit depth of 10 bits;

[0071] Figure 4-d shows a typical bit-by-bit memory representation of the output of multiplying the CCALF filter coefficients by the sample values ​​when performing CCALF filtering;

[0072] Figure 4-e shows a typical bit-by-bit memory representation of the output of an addition operation performed to sum the results of multiplying the CCALF filter coefficients by the sample values ​​when performing CCALF filtering;

[0073] Figure 5-a A bit-by-bit memory representation showing a reduction in the number of bits of CCALF filter coefficients by reducing the precision of the coefficient values ​​according to an example of a first variant of the present invention;

[0074] Figure 5-b A bit-by-bit memory representation showing a reduction in the number of bits of CCALF filter coefficients by limiting the range of possible coefficient values ​​according to an example of a second variant of the present invention;

[0075] Figure 5-c shows a bit-by-bit memory representation of CCALF filter coefficients by reducing the precision of the coefficient values ​​and by limiting the range of possible coefficient values ​​according to an example combining the first and second variants of the present invention;

[0076] Figure 5-d A bit-by-bit memory representation showing the reduction of the number of bits of sample values ​​used inside the filtering process by ignoring the least significant bit according to an example of a third variant of the present invention;

[0077] Figure 6-ashows a bit-by-bit memory representation of the differences of sample values ​​used inside the filtering process according to an example of a fourth variant of the invention;

[0078] Figure 6-b shows a bit-by-bit memory representation of the difference in sample values ​​when a clipping operation is used to reduce the number of bits inside a filtering process according to an example of a fourth variant of the present invention;

[0079] Figure 6-c shows a bit-by-bit memory representation of the difference in sample values ​​when a clipping operation is used and the least significant bits are ignored to reduce the number of bits inside a filtering process according to an example combining the third and fourth variants of the present invention;

[0080] Figure 7-a shows a typical bit-by-bit memory representation of the output of multiplying CCALF filter coefficients by sample values ​​when performing CCALF filtering according to a variant of the present invention;

[0081] Figure 7-b A typical bit-by-bit memory representation of the output of an addition operation performed to sum the multiplication results of the CCALF filter coefficients and the sample values ​​when performing CCALF filtering according to a variant of the present invention is shown;

[0082] Figure 7-c Provides an example filter shape and coefficient arrangement for a CCALF filter using 16 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variation of the present invention;

[0083] Figure 7-d Provides an example filter shape and coefficient arrangement of a CCALF filter using 16 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variation of the present invention;

[0084] Figure 7-e Provides an example filter shape and coefficient arrangement of a CCALF filter using 12 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variation of the present invention;

[0085] Figure 7-f Provides an example filter shape and coefficient arrangement of a CCALF filter using 8 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variation of the present invention;

[0086] Figure 7-g Provides an example filter shape and coefficient arrangement of a CCALF filter using 4 samples to reduce the number of bits and operations used in the CCALF filtering process according to a fifth variation of the present invention;

[0087] Figure 8 is a flowchart showing the steps of an encoding method according to an embodiment of the present invention;

[0088] Fig. 9 is a flow chart showing the steps of a decoding method according to an embodiment of the present invention;

[0089] Fig.10 is a block diagram schematically illustrating a data communication system in which one or more embodiments of the present invention may be implemented;

[0090] Fig.11 is a block diagram illustrating components of a processing device that may implement one or more embodiments of the present invention;

[0091] Fig.12 is a diagram illustrating a network camera system in which one or more embodiments of the present invention may be implemented; and

[0092] Fig.13 is a diagram illustrating a smart phone in which one or more embodiments of the present invention may be implemented. DETAILED DESCRIPTION

[0093] Embodiments of the invention described below are directed to improving the encoding and decoding of images.

[0094] In this specification, "signaling" may refer to inserting (providing / including / encoding in) information related to one or more parameters for controlling a filter (e.g., using, not using, enabling or disabling a mode / scheme or other filtering control-related information) into a bitstream or extracting / obtaining (decoding) the information from a bitstream.

[0095] In this specification, the term "slice" is used as an example of an image part (other examples of such an image part would be a tile or a tile group (which is a group / set of tiles)). It should be understood that embodiments of the present invention may also be implemented based on image parts (e.g. tiles or tile groups) instead of being slice-based, and appropriately modified parameters / values / syntax (such as headers of image parts / tiles / tile groups (instead of slice headers), types of image parts / tiles / tile groups (instead of slice types), and statistical information of image parts / tiles / tile groups, etc. (instead of slice statistics)). It should also be understood that instead of a slice header or a sequence parameter set (SPS), an adaptive parameter set (APS) or a tile (group) header may also be used to signal ALF or CCALF parameters (or information for using (CC)ALF filtering). When an APS is used to signal (CC)ALF parameters (or information for using (CC)ALF filtering), a slice header or a tile group header may be used to indicate which APS must be used to obtain the (CC)ALF parameters (or information for using (CC)ALF filtering), for example by indicating an adaptation set identifier (aps_id). It should also be understood that any of a slice, a tile group, a tile, a coding tree unit (CTU) / largest coding unit (LCU), a coding tree block (CTB), a coding unit (CU), a prediction unit (PU), a transform unit (TU), or a pixel / sample block may be referred to as an image portion.

[0096] It should also be understood that when a filter or tool is described as "active", the filter / tool ​​is "enabled" or "available for use" or "used"; when described as "inactive", the filter / tool ​​is "disabled" or "unusable" or "not used"; and "class" refers to a group, grouping, category or classification of one or more elements. In addition, it should also be understood that when a flag is described as "active", it means that the flag indicates that the relevant filter / tool ​​is "active".

[0097] Adaptive Loop Filter (ALF)

[0098] Figure 1The location where the ALF is present in a typical decoding loop of VTM-5.0 is shown. In 101, image portions (e.g., slices) are decoded in units of coding tree units (CTUs: the largest coding unit in VVC, typically 128×128 samples / pixels in size). The CTUs are split into rectangular blocks or coding units (CUs), which are encoded using a specific prediction scheme / mode and are typically lossy encodings of residual blocks. Due to the use of block-based coding, block artifacts may be visible at the boundaries between coded blocks. In 102, the decoded image portion is then processed by a DBF to reduce / remove these artifacts. Typically, in order to encode the residual (block) used for block prediction, the residual values ​​are transformed using a DCT-like transform (to compress the residual energy in several coefficients), and the transformed coefficients are quantized to reduce the coding cost. This quantization typically introduces some ringing artifacts in the reconstructed blocks (i.e., the blocks in the reference frame stored in the frame buffer 106). In 103, the output image portion of the DBF is then processed by the SAO filter, which helps to reduce some of these artifacts at a lower computational cost. In 104, the output image portion of the SAO filter is then processed by the ALF. The ALF can further reduce artifacts such as, for example, "ringing". The ALF has higher-order error modeling capabilities, but the computational cost is higher. The output image portion of the ALF is then sent to the output (e.g., a display or a communication interface for communicating with a display) 105. The output image portion of the ALF can also be placed in a frame buffer 106 (as part of a reference frame stored therein) so that it can be used for temporal prediction (when a temporal prediction tool is used). This is why the DBF, SAO filter and ALF are called "in-loop" filters. The encoder can disable some in-loop filters so that they can be bypassed at decoding (i.e., no filtering is performed, and the output of the step corresponding to the disabled tool is the same as its input). In addition, in some cases, the processed image portion is not limited to slices, but can be a full frame containing one or more slices, with the possibility of applying filters across slice boundaries (if there are more than one) to reduce artifacts on these boundaries. For multi-component images (eg, images in YCrCb format), the DBF, SAO filter, or ALF process is applied to each component separately and may be different (eg, using different filtering parameters than other components).

[0099] Figure 2An overview of ALF filtering in VTM-5.0 is provided. For each channel (luminance channel: Y and chrominance channels: Cb and Cr), input image portions 201, 202, 203 are provided to adaptive loop filtering (ALF) processes 207, 208, 209 along with filtering parameters 204, 205, 206. The ALF processes generate processed output image portions for each channel 210, 211, 212. The ALF chrominance processing in 208, 209 is the same for both chrominance channels, but can be configured to work differently for each channel due to specific parameters provided in 205 and 206. The ALF luminance processing performed in 207 is typically more complex than the ALF chrominance processing performed in 208. There are two reasons for this. The first reason is that the processing / encoding of the luminance channel content is typically more complex than the chrominance channels because the luminance channel contains more structure (contours, textures, etc.) than the chrominance channels, which are typically smoother. The second reason is that the human eye is more sensitive to errors on the luma channel than on the chroma channels. Therefore, the luma channel prefers more advanced and complex (i.e., more power-hungry and / or slower) processing to provide better coding gain and better visual quality.

[0100] The input parameter information provided in 204, 205 and 206 is sent in an Adaptation Parameter Set (APS) and / or a slice header. 204, 205 and 206 may also share some common information.

[0101] ALF processing 207, 208, 209 can be performed in parallel on each channel because the task results of each are independent of the task results of the others. This opportunity for parallel processing can reduce the overall time spent on processing and can be efficiently implemented in hardware or software.

[0102] The input parameter information provided in 204, 205 and 206 may also include internal bit depth information. The internal bit depth is the maximum number of bits that can be used to represent a sample value (i.e., it represents the precision of the sample value). In VTM-5.0, it is permitted to use an internal bit depth for the luma channel that is different from the internal bit depth of the chroma channels (the bit depth information is signaled in the sequence parameter set (SPS) for both channel types (luminance and chroma)). In the following description, "IBDL" will refer to the internal bit depth of luma and "IBDC" will refer to the internal bit depth of chroma. When "IBD" is used for the internal bit depth, it means that IBDL is considered to be equivalent to IBDC. When not mentioned or specified, the internal bit depth does not affect the processing discussed, or has been omitted because the internal bit depth is handled in an orthodox manner.

[0103] Figure 3 supply Figure 2An extension of the ALF filtering in to use cross-component ALF (CCALF) processing.

[0104] For each chroma channel 202, 203, the input image portion 201 of the luma channel is provided to a cross-component adaptive loop filtering (CCALF) process 308, 309 along with filtering parameters 305, 306. The output of the CCALF filtering process is added to the output of the ALF chroma filtering process 208, 209 in 311, 312 to generate a processed output image portion 313, 314 for each chroma channel. The CCALF process in 308, 309 is the same for both chroma channels, but the CCALF process in 308, 309 may be configured to work differently on each channel due to specific parameters provided in 305 and 306. For example, specific filter coefficients may be used to process each chroma channel.

[0105] In some variations, the chroma channels are subsampled compared to the luma channel, such as for video using a YUV 4:2:0 color representation. In such variations, the CCALF filtering process 308, 309 is not applied to each luma sample position, but only to the integer luma sample positions obtained when upsampling the chroma sample positions. Thus, for a YUV 4:2:0 representation, filtering is applied to only one quarter of the luma sample positions.

[0106] The ALF filtering processes 207, 208 and 209, and the CCALF filtering processes 308 and 309 are independent, and therefore, they can be performed in parallel. Figure 2 In contrast, by performing addition operations at 311 and 312 to combine the results of the two chrominance filters for each channel, Figure 3 However, since CCALF and chroma ALF are usually less complex than luma ALF, it should be feasible to implement the overall processing of 3 channels without additional latency in VTM-5.0, i.e., the rate determination step may be luma ALF, and thus the additional latency in the other two channels does not affect the latency of the overall processing.

[0107] The input parameter information provided in 305 and 306 is sent in an Adaptation Parameter Set (APS) and / or a slice header. 305 and 306 may also share some common information.

[0108] In one variation, 305 and 306 contain filter coefficients for a linear convolution filter (see, for example, Figures 4-a to 4-e). In an alternative, 305 and 306 contain filter coefficients and clipping parameters for a non-linear filter, similar to the filter coefficients and clipping parameters used for non-linear ALF filtering in VTM-5.0, for example. Both linear and non-linear filters have been found to provide gains in coding performance, however linear filters are less complex and therefore may be preferred in certain implementations.

[0109] In some variations, the input parameter information provided in 305 and 306 may include IBDL and IBDC to infer some left or right bitwise shift operation parameters (or multiplication / division operation parameters) to "normalize" data (e.g., sample values) between channels (i.e., using comparable precision values).

[0110] It should be understood that in some variations, CCALF may be applied even if ALF filtering has not been performed. It should also be understood that a luma channel or other types of channels may be used to correct other types of channels in addition to chroma channels. For example, a red channel may be used to correct a green channel and / or a blue channel in a video represented using RGB color.

[0111] It should also be understood that in some variations, more than one channel may be used to reduce the error of another channel. For example, input image portions of both chroma channels 202 and 203 may be filtered with a CCALF filter, and the filtered results of the respective image portions are added to the output of the ALF luminance filter 207 to correct the remaining error in the luminance channel. Such variations may be particularly applicable when the chroma channels are subsampled compared to the luminance channel. For example, in a YUV 4:2:0 color representation, four different filters may be used for the chroma samples, one filter for each of the four luminance sample positions that are downsampled / associated to one chroma sample position. In other words, for a 2×2 block of luminance samples, a first filter is applied at the corresponding chroma position to correct the luminance sample at position (0,0) in the 2×2 block, and a second, third, and fourth filter are applied to the same chroma sample position to correct the luminance samples at positions (0,1), (1,0), and (1,1), respectively. In a variation, only one CCALF filter is used for the four luma sample positions, but the filtered chroma sample values ​​are obtained by upsampling the chroma samples to obtain chroma samples aligned on the same grid as the luma samples.

[0112] Figure 4-aAn example of a filter shape 401 that can be used when performing CCALF with a linear convolution filter is shown. This filter shape is well suited for filtering the luma channel to correct errors in the chroma channels of video using a YUV 4:2:0 color representation, but can also be used with other kinds of color representations. The individual boxes correspond to samples located around the location of the reference sample (the box with the gray dashed background). The individual numbers in the individual boxes correspond to the indices of the filter coefficients applied to the samples at the location of the box.

[0113] Position(x c ,y c ) is filtered / corrected by adding the following amount to its value:

[0114]

[0115]

[0116] where f i Cb is the filter coefficient with index i of the filter for the Cb chrominance component.

[0117] Similar filtering is applied to the Cr component, but using f i Cr Filter coefficients.

[0118] The same filtering can be obtained by simplifying the notation using the following formula notation:

[0119]

[0120] in is the offset (u) from the reference sample (box with grey dashed background in 401) with position (x,y) in the filter shape. i ,v i ) The filter coefficients to be applied together.

[0121] The implementation of equation (2) is usually done using integer arithmetic to approximate real number arithmetic with fixed-point calculations. Equation (2) can then be rewritten as:

[0122]

[0123] where << is an integer left bitwise shift operation, >> is an integer right bitwise shift operation, and B is the filter coefficient The decimal precision of the fixed-point representation (usually 10 digits) (see Figure 4-b );and is an integer sample value with integer precision equal to the codec's IBD (typically 10 bits, see Figure 4-c ).

[0124] In the YUV 4:2:0 color representation, the luma sample position (x,y) corresponds to the nearest integer upsampled chroma position: (x,y) = (2.x c ,2.y c ).

[0125] In some variations, when IBDL is greater than IBDC, equation (3) can be rewritten as:

[0126]

[0127] or

[0128]

[0129] or

[0130]

[0131] And when IBDL is lower than IBDC, equation (3) can be rewritten as:

[0132]

[0133] Or, if B+IBDL is strictly higher than IBDC, then this is equivalent to:

[0134]

[0135] Among them, B1=B+IBDL-IBDC.

[0136] The additional bitwise shift operations in equations (4), (5), (6) and (7) or the modified bitwise shift parameters in equation (8) allow the results of the filtering to be normalized to maintain the same accuracy as the chrominance sample values. To achieve this, in equation (4), the luma sample values ​​are normalized; in equation (5), the result of multiplying the luma sample values ​​by the coefficient values ​​is normalized, or in equations (6), (7) and (8), the result of the sum of the multiplication outputs (i.e., the filtering result) is normalized.

[0137] Equation (6) can also be rewritten as in equation (8) with the same result.

[0138] In equation (7), brackets are intentionally not placed around the bitwise shift because when IBDL is lower than IBDC, the result of the calculation will be the same regardless of where the left shift is performed. This is a matter of hardware or software design choice. Alternatively, the shift value in equation (4) can also be rounded by adding 1 << (IBDC - IBDL - 1) before applying the right shift (IBDL - IBDC).

[0139] In terms of hardware / software design, when IBDL is greater than IBDC, an implementation following the formula of equation (8) may be preferred because it requires fewer operations. In the specific case where IBDL is always higher than IBDC, equation (4) may preferably limit the number of bits of the multiplication input: the number of bits obtained in a system implementation using equation (4) will then depend on the maximum value of IBDC that can be used in the system versus the maximum value of IBDL that can be used (because this is the worst case that must be dealt with). When IBDC is greater than IBDL, it may be preferred to use equation (8) to not increase the input bit depth of the multiplication, especially when the maximum value of IBDC that can be used is always greater than the maximum value of IBDL that can be used. When the maximum value of IBDC is equal to the maximum value of IBDL, equation (8) is the best compromise.

[0140] For the case where B+IBDL is not strictly higher than IBDC, equation (8) can be rewritten as:

[0141]

[0142] Where B2 = IBDC-IBDL-B.

[0143] Figure 4-b A typical bit-by-bit memory representation of the CCALF filter coefficients in equation (3) is shown. The bit-by-bit representation 402 of the CCALF filter coefficients typically uses a 16-bit signed integer representation (in tow's complement) to store a fixed-point approximation of a real number. 10 bits of precision are used for the fractional part, and the remaining bits are used to store the integer part and the sign.

[0144] Figure 4-c A typical bit-by-bit memory representation of the sample values ​​in equation (3) is shown for an internal bit depth of 10 bits in VTM-5.0 when stored in a 16-bit integer memory space 403 (or register). The crosses represent unused bits.

[0145] Figure 4-d The CCALF filter coefficients shown correspond to the Figure 4-b is used to store the multiplication of the 10-bit internal bit depth sample value with the CCALF filter coefficient (e.g., for a given i, ) is a typical bit-by-bit memory representation of a 32-bit signed integer of the result of 403. As can be seen, 24 bits of precision are required (10 bits of precision of 403 plus 14 bits of precision of 402), plus one bit for the sign (i.e., a 25-bit signed integer representation).

[0146] Figure 4-e The CCALF filter coefficients shown correspond to the Figure 4-b In the case of filter coefficients used in Figure 4-a In the filter pattern case, it is used to store the accumulation (sum) of the results of multiplying the 10-bit internal bit depth sample values ​​with the CCALF filter coefficients (for example, ) is: . This corresponds to the bit-by-bit memory required to store the sum of 18 25-bit signed integers, such as Figure 4-b As can be seen, 29 bits of precision are required (10 bits of precision for 403 plus 14 bits of precision for 402 plus 5 bits for storing the worst case result of 18 additions), plus one bit for the sign (ie, a 30-bit signed integer representation).

[0147] According to an embodiment of the present invention, using Figure 4-b The filtering operation in equation (3) for CCALF filtering is highly simplified when compared to the typical coefficient representation of , especially for hardware-specific designs (where the number of logic gates has a cost and where the number of gates used to perform the multiplication depends on the maximum number of bits of the input). It has been surprisingly found that significant simplifications can be achieved with an impact on coding efficiency that is acceptable when accompanied by significant design simplifications. In general, the simplifications disclosed below involve using the filter coefficients Or input samples across components A reduced bit depth representation of either or both of (compared to the internal bit depth). A constrained bit depth representation is constrained compared to the internal bit depth of the first component samples, or constrained compared to a typical filter coefficient bit depth (both are common examples of 10 bits).

[0148] It will be appreciated that certain combinations of bit depth reduction are particularly advantageous (e.g. when leading to calculations that can be done on a 16-bit architecture). In any case, calculations using the constrained number of bits are faster, which means that the additional cost (in terms of increased time or computational power) of CCALF (compared to simple ALF) is lower.

[0149] A first variation to reduce the number of bits in the multiplication input in equation (3) is to reduce the fixed-point precision. This can be done by removing Figure 4-bFor example, it has been found that using 7 bits instead of 10 bits performs a good compromise (e.g., when using a 10-bit sample intra-bit depth) so as not to reduce the coding gain of CCALF too much. In some variants, using 6 bits instead of 10 bits reduces the coding gain of CCALF a little, but this is still an interesting hardware design trade-off.

[0150] Figure 5-a An example of a bit-by-bit memory representation of the CCALF filter coefficients in equation (3) is shown when the fixed-point fractional precision is limited to 7 bits instead of 10. The bit-by-bit representation of 402 is then reduced in 501 to 13 bits (12 bits for precision and 1 bit for sign).

[0151] A second alternative to reduce the number of bits of the multiplication input in equation (3) is to restrict the range of possible values ​​of the filter coefficients. This can be done by removing Figure 4-b . It has been surprisingly found that CCALF can still work efficiently even when the range of possible values ​​of the coefficients is significantly limited. This is particularly surprising because the effect is specific to CCALF and the effect is not replicated in the case of ALF (for example, doing so will introduce a higher loss to the coding gain). The surprisingly low impact of such simplification on coding efficiency can be explained because the inventors have found that cross-component ALF is most useful when the gradient of the luminance sample is strong (for example, close to the edge) (because the chrominance error at this position is usually stronger and related to the luminance sample gradient). In addition, the CCALF output in these areas is usually a small / limited value (and usually zero elsewhere). Therefore, the mathematically optimal filter coefficients (in the real domain) usually have a small amplitude and can be well approximated by using a small number of bits (a fixed-point decimal number) integer representation. Therefore, most of the increase in coding performance can be achieved from a subset of coefficients. Embodiments of the present invention utilize this feature to simplify calculations without significantly affecting coding performance. This also explains why the third and fourth variants presented below work: the precision of the samples / coefficients is reduced, which generates a kind of "rounding error noise" (which does not significantly affect the estimation of small values ​​of the CCALF output).

[0152] In summary, it has been found that:

[0153] i. Clamping the coefficients to the range between -0.03125 and 0.03125 (e.g., not using any bits above the 6th decimal point precision) has an acceptable coding impact

[0154] ii. Clamping the coefficients to the range between -0.0625 and 0.0625 (e.g., not using any bits above the 5th decimal point precision) has no significant coding impact

[0155] iii. Clamping the coefficients to the range between -0.125 and 0.125 (e.g. not using any bits above the 4th decimal point precision) has little coding impact

[0156] iv. Clamping the coefficients to the range between -0.25 and 0.25 (e.g., not using any bits above the 3rd place of decimal precision) has negligible coding impact

[0157] v. Clamping the coefficients to the range between -0.015625 and 0.015625 (e.g., not using any bits above the 7th place of decimal precision) has coding implications that may be tolerable in some implementations.

[0158] The limits of the range may or may not be included in the available coefficient values. Examples using these three ranges are described in detail below, but alternative ranges may also be used.

[0159] In the following description, the results are provided. These results are obtained using the "JVET common test conditions and software reference configurations for SDR video" defined in the JVET-N01010 document; but the test conditions are constrained to be applied to the first 17 frames of the video sequence. Therefore, the variant of the present invention has been implemented on the VTM-5.0 software (or during the development process for providing the VTM-6.0 software, the subsequent intermediate development version), which has been used as a benchmark. Compared with the benchmark (e.g., VTM-5.0), the average BD rate coding gain results for the "full intra" (AI), "random access" (RA), "low delay B" (LB) and "low delay P" (LP) configurations are obtained for the luma channel and the two chroma channels. These average channel coding gains are combined by the following operation: the luma coding gain is summed 14 times with the coding gain of each chroma channel, and the sum is divided by 16 to obtain the average YUV coding gain. This is a conventional way to combine the luma and chroma channel results, because the chroma channel is easier to encode than the luma channel. The average YUV coding gains obtained for the AI, RA, LB and LP configurations are averaged together, and these final average coding gain values ​​are compared together to provide a coding gain comparison. The coding gain comparison is expressed as a percentage of the benchmark as (when not explicitly specified as different) the benchmark software (i.e., VTM-5.0 or a later intermediate development version) (which has CCALF implemented on it), as specified in the JVET-00636 contribution, but excluding the present invention used to simplify the benchmark. The compared variants are implemented by simplifying the benchmark (i.e., using the same VTM-5.0 or a later intermediate development version, and CCALF implemented on it but simplified with the present invention). The percentage is obtained by calculating the ratio between the coding gain of the variant (which is the average BD rate gain expressed as a percentage of the rate reduction for the same quality) and the coding gain of the benchmark (also the average BD rate gain). For information, using the benchmark CCALF on top of VTM-5.0 provides an average YUV coding gain of about -1.2% compared to VTM-5.0.

[0160] The coding gain comparison results provided below are intended to be informative, and when the present invention is implemented on top of another encoder implementation or on top of another codec specification, the reported percentages of coding gain may be different.

[0161] In the following description, range notations are used with the following meanings:

[0162] ]x,y[, meaning the range of values ​​between x and y, but not including x and y

[0163] [x,y[, means the range of values ​​between x and y, including x but excluding y

[0164] ]x,y], meaning the range of values ​​between x and y, including y but excluding x

[0165] [x,y], meaning the range of values ​​between x and y, inclusive

[0166] Using coefficients in the range [-0.03125; 0.03125[ (not using any bits above the 6th bit of fixed-point fractional precision) retains most of the coding gain of CCALF while significantly reducing the number of bits used for multiplication. Figure 5-b An example of this modification is shown.

[0167] Figure 5-b An example of a bit-by-bit memory representation of the CCALF filter coefficients in equation (3) is shown when 10 bits of fixed-point fractional precision is used and the range of possible values ​​of the coefficients is restricted to the range [-0.03125; 0.03125[. The bit-by-bit representation of 402 is then reduced to 6 bits (5 bits for precision and 1 bit for sign) in 502.

[0168] Second Modification Example ( Figure 5-b ) can be combined with the first variant ( Figure 5-a ) combination, such as Figure 5-c shown.

[0169] Figure 5-c An example of a bit-by-bit memory representation of the CCALF filter coefficients in equation (3) is shown when the fixed-point fractional precision is limited to 7 bits instead of 10 bits and the range of possible values ​​of the coefficients is limited to the range [-0.03125; 0.03125[. The bit-by-bit representation of 402 is then reduced to 3 bits (2 bits for precision and 1 bit for sign) in 503.

[0170] By restricting the coefficients to the range [-0.03125; 0.03125] (i.e., the filter coefficients can only take values ​​from the set {-0.0234375, -0.015625, -0.0078125, 0, 0.0078125, 0.015625, 0.0234375}), it is possible to use two additions / subtractions (depending on the sign handling) and a bitwise shift (in Figure 5-c In the case shown in the figure), the multiplications can be slightly simplified. For example, the multiplications in equation (3) can be replaced by, for example:

[0171]

[0172] where << is an integer left bitwise shift operation, >> is an integer right bitwise shift operation, and B is the filter coefficient The decimal precision (7 digits) of the fixed-point representation (see Figure 5-c );and is an integer sample value with integer precision equal to the internal bit depth of the codec (usually 10 bits, see Figure 4-c ).

[0173] The multiplication can even be replaced by just one addition / subtraction (depending on the sign handling) and a bitwise shift. However, this introduces more loss in the coding gain of the CCALF. This can be done, for example, when the precision of the fixed-point fractional precision is limited to 6 bits and the coefficients are limited to the range of ]-0.03125; 0.03125[ (i.e., the filter coefficients can only take values ​​in a set of values ​​{-0.015625, 0, 0.015625}). This can also be done, for example, when the precision of the fixed-point fractional precision is limited to 7 bits and the coefficients are limited to the range of ]-0.015625; 0.015625[ (i.e., the filter coefficients can only take values ​​in a set of values ​​{-0.0078125, 0, 0.078125}). Using the first alternative allows maintaining approximately 78% of the CCALF gain compared to using 10 bits of fractional precision and using filter coefficients in the range ]-1; 1[. Using the second alternative allows maintaining a CCALF gain of approximately 76%.

[0174] In some other alternatives, the values ​​of the coefficients range from [-0.0625; 0.0625[,]-0.0625; 0.0625[, [-0.125; 0.125[,]-0.125; 0.125[, [-0.25; 0.25] or]-0.25; 0.25], i.e., one, two or three additional most significant bits are kept for the (fractional) precision of the coefficients. For example, as Figure 5-c An alternative is to reserve 3 or 4 bits instead of 2 for the filter coefficients. In such an alternative, the additional bits are Figure 5-c The second and fourth of these four alternatives allow replacing the multiplications with three and four additions / subtractions (depending on the sign handling) and bitwise shifts, respectively.

[0175] A third variant to reduce the number of bits of the multiplication input in equation (3) is to reduce the precision of the sample values. This can be done by not considering the least significant bit of the sample value (which would be equivalent to using a right-shift bitwise shift using a CPU implementation). For example, equation (3) can be rewritten as:

[0176]

[0177] where DB corresponds to the number of dead bits (i.e., bits that are ignored / not considered). Figure 5-d The bit-by-bit representation of the sample values ​​used in the multiplication for this variant is shown in .

[0178] Figure 5-d An example of a bit-by-bit memory representation of sample values ​​used in equation (3) for a 10-bit internal bit depth in VTM-5.0 is shown when the precision of the sample values ​​is limited to 6 bits instead of 10 bits to simplify the hardware implementation of the multiplication operation in equation (3). The cross boxes represent unused bits. The bit-by-bit representation of 403 is reduced to 6 bits in 504.

[0179] An alternative is to "round" the sample value using, for example, the highest removed bit value (e.g., by adding 1 to the representation with the reduced number of bits if the highest removed bit is 1, or equivalently adding a value corresponding to one of the highest removed bits before removing / disregarding the unwanted bits). This alternative is interesting when reducing sample precision affects the coding efficiency of CCALF to a greater extent, but it also means using additional operations and / or memory compared to just ignoring the lowest bit.

[0180] It has been found that in the context of VTM-5.0 improved by using CCALF, keeping only the 6 most significant bits of the sample value when performing a filtering operation will not significantly affect the coding efficiency. For example, in the case of an internal bit depth of 10 bits, the 4 least significant bits of the sample value can be ignored, and the impact on the CCALF efficiency is small. It is possible to keep less than 6 most significant bits of the sample value, but the coding efficiency of CCALF is reduced. For example, keeping the 4 most significant bits of the sample value when performing a filtering operation (i.e., removing the 6 least significant bits on a 10-bit sample) still provides a surprisingly interesting gain (on average, about 88% of the CCALF gain is maintained compared to using the full sample accuracy on a 10-bit sample). And keeping the 5 most significant bits allows maintaining about 95% of the CCALF gain compared to using the full sample accuracy. In the case of 6 most significant bits, the result is that about 98% of the gain is retained. Keeping more than 6 most significant bits only slightly increases the result. For example, using full sample precision, keeping the 7 or 8 most significant bits (ie, removing the least significant 3 or 2 bits of a 10-bit internal bit depth) provides a result of approximately 99% CCALF gain.

[0181] In the fourth variation, the CCALF filtering as expressed in equation (2) is modified to be closer to the filtering formula of the nonlinear ALF (NLALF) in VTM-5.0:

[0182]

[0183] in, Still filter coefficients, but with an offset (u i ,v i ) With reference sample I with position (x,y) (x,y) The number representing the difference between the values ​​of (boxes with a gray dashed background in 401) can be multiplied. The number representing the difference can be a gradient, which is equivalent to the difference of samples adjacent to the reference sample. The gradient is used to normalize the difference between samples that are both adjacent and non-adjacent to the reference sample.

[0184] Equation (12) can be implemented using integer arithmetic in the same way that equation (2) is implemented using integer arithmetic in equation (3):

[0185]

[0186] where << is an integer left bitwise shift operation, >> is an integer right bitwise shift operation, and B is the filter coefficient The decimal precision of the fixed-point representation (usually 10 digits) (see Figure 4-b );and and I (x,y) is an integer with precision equal to the internal bit depth of the codec (usually 10 bits, see Figure 4-c ). When this formula is used, Compared to the bit-by-bit representation of This is in the bit-by-bit representation of Figure 6-a In some alternatives, is always equal to zero and is not encoded in the bitstream. In other alternatives, is not necessarily equal to zero, and multiplication The multiplication in other variants where the difference to the reference sample is handled is not used.

[0187] Figure 6-a The difference of the sample values ​​in equation (13) when stored in the 16-bit integer memory space 601 (or register) for the 10-bit internal bit depth in VTM-5.0 is shown. A bit-by-bit memory representation of . The cross-boxes represent unused bits.

[0188] From a hardware design point of view, the difference to the reference sample is cost-free since it is already present / computed for the NLALF filtering of luma samples (the filter shape of the CCALF filter is included in the shape of the filter for luma in NLALF).

[0189] The coding gain of CCALF when using the filtering of equation (3) is equivalent to that when using the filtering of equation (13).

[0190] One advantage of using this representation is that in order to reduce The number of bits in the input to the multiplication can be used to clip the difference to keep it within the range that requires fewer bits to represent:

[0191]

[0192] Where k is the cropping function, for example:

[0193] K(d,C)=max(-C,min(C,d)) (15)

[0194] Or equivalently:

[0195] K(d,C)=min(C,max(-C,d)); and where C is the clipping parameter and d is a number representing the difference. It should be noted that the specific difference between this expression and the nonlinear ALF in VTM-5.0 is that the clipping parameter C is the same for each sample position and its value is fixed and selected to constrain the number of cells used to store the input to the multiplication. The number of bits used in the nonlinear ALF is , while in the nonlinear ALF the clipping parameter C varies for each sample position and its value is transmitted together with the filter coefficients (in the APS).

[0196] A particularly advantageous fixed value for C has been found to be 2 IBitDepth-2 -1 or 2 IBitDepth-3 -1, where IBitDepth is the internal bit depth of the codec (i.e., the number of bits used to represent a sample value). For example, with a 10-bit internal bit depth, a good value for C is 255 or 127. This exemplary clipping operation then allows saving two or three bits in the input to the multiplication.

[0197] Smaller values ​​of C can be used, but will tend to reduce the coding efficiency of CCALF, while higher values ​​reduce the simplification efficiency provided by the clipping process. Therefore, a generalization of the clipping parameter C is 2 IBitDepth-n -1, where n is an integer less than IBitDepth. For lower values ​​of n, the reduction in coding efficiency is lower, but the advantages associated with simplification are also lower; for higher values ​​of n, the opposite is true.

[0198] Figure 6-b Given the clipped difference in equation (14) for 10-bit internal bit depth and C=255 when stored in 16-bit integer memory space 602 (or registers), Example of a bit-by-bit memory representation of .

[0199] The fourth variation can be combined with the third variation to ignore the clipping difference. A particularly advantageous combination is to keep the same number of lower precision bits as in the second variant so that the CCALF coding gain is not significantly affected. This example consists of Figure 6-c Shown.

[0200] Figure 6-c Provided for storage with an internal bit depth of 10 bits, C = 255 and the 4 least significant bits ignored Then, at the input of the multiplication, only the four remaining precision bits of 603 plus one bit for the sign are needed to represent rather than 10. This represents a significant saving in the number of bits computed, and results in a simpler implementation.

[0201] In some embodiments, a combination of four variants is performed. For example, variants one, two, three and optionally four are combined so that the output result of the multiplication operation can be represented on 10 precision bits plus one sign bit, so that the sum of the multiplication results can be represented on a maximum of 15 bits plus one sign bit (thus representable / storable in a 16-bit register or memory space). Such an embodiment is particularly suitable for hardware implementation. Figure 7-a and Figure 7-b An example of an intermediate representation (multiplication output) and a final representation (sum result) of a filtering operation is shown before final rounding and bit shifting to add the result to the chroma sample values.

[0202] Figure 7-a Provided for storage in the following situations An example of a bit-by-bit memory representation of : only the 8 most significant bits of the sample value are kept (e.g., as Figure 5-d ), maintaining 7 digits of precision for the fractional part of the coefficient, and the range of the coefficient is [-0.03125; 0.03125 [or] -0.03125; 0.03125 [(e.g., as Figure 5-c 2 bits plus a sign bit are used to store them as shown). The multiplication result can then be stored in 10 bits plus a sign bit.

[0203] Using equation (4) for handling different luma bit depths IBDL and chroma bit depths IBDC, and equation (11) where DB = IBDL - KB, where KB equals the number of highest precision bits held for luma sample values ​​(e.g., KB = 8 in this example), equation (3) can be rewritten as:

[0204]

[0205] where B3 = DB+(IBDL - IBDC)=2·IBDL - IBDC - KB, and B4 = BDPC - B3, where BDPC is the number of bits of the fractional part of the coefficient (e.g., in this example, for 7 - bit precision of the fractional part of the coefficient, BDPC = 7), and is the integer representation of the coefficient (i.e., in this example, the integer value representing the fixed - point value in the range [-0.03125; 0.03125[ or [-0.03125; 0.03125[, and in this example, the integer value takes a value in the integer range [-4; 3] or [-3; 3]). This equation is valid for KB ≤ 2·IBDL - IBDC ≤ BDPC + KB - 1. When KB ≤ 2·IBDL - IBDC and 2·IBDL - IBDC = BDPC + KB (i.e., B4 = 0), equation (16) can be replaced by:

[0206]

[0207] When KB ≤ 2·IBDL - IBDC and 2·IBDL - IBDC > BDPC + KB, equation (16) can be replaced by:

[0208]

[0209] And when 2·IBDL - IBDC < KB, equation (16) can be replaced by:

[0210]

[0211] In the given example, where KB = 8, for 10 - bit luminance and chrominance bit - depths (i.e., IBDL = IBDC = 10), equation (16) can be written as:

[0212]

[0213] This allows each coefficient to be represented by 2 bits and a sign bit, and each sample to be represented by a reduced number of 8 bits, and optionally (depending on the implementation / design choice) the sign bit (equal to zero), and thus the multiplication result can be represented by 8 bits and a sign.

[0214] Figure 7-a For the example of providing a bit - by - bit memory representation for storage is also valid: only keeping the 6 most significant bits of the sample value (e.g., having two more higher bits than in Figure 5-d ), keeping 7 - bit precision for the fractional part of the coefficient, and the coefficient range being [-0.125; 0.125[ or [-0.125; 0.125[ (i.e., 4 bits plus a sign bit to store them (when e.g., having more bits than in Figure 5-cThe multiplication result can then be stored in 10 bits plus a sign bit.

[0215] In this given example, KB = 6. For 10-bit luma and chroma bit depths (ie, IBDL = IBDC = 10), equation (16) can be written as:

[0216]

[0217] Figure 7-b Provides the following conditions for calculating and storing sum operations An example of a bit-by-bit memory representation of Figure 7-a Represent and calculate as explained It should be understood that this is for filters based on Figure 4-a The worst case scenario for 17 different additions with 18 multiplication results (when considering 10-bit integer values ​​for storing fixed-point values) is an integer value of -2. 10 = 18 times -1024, that is -18432, which can be stored in 15 bits and a sign bit, meaning that the result of the addition can be represented using 15 bits and a sign bit.

[0218] In another (fifth) modification, Figure 4-a The filter mode of , so that 16 samples or less are used in the filtering process instead of 18 samples. Therefore, in the summation operation, one bit can be saved. In addition, since some multiplication operations are removed, it has the advantage of simplifying the filtering calculation. For example, in Figure 4-b In , only 28 precision bits and a sign bit are used instead of 29 precision bits and a sign bit, and in Figure 7-b In some alternatives using this fifth variant, the bits saved in the addition operation are used to increase the precision of the multiplication result. For example, in Figure 7-a In , 11 precision bits are used instead of 10 precision bits. In an alternative, the fractional part of the 10 precision bits is increased by one bit (from 7 to 8). This can be achieved by adding a lower precision bit to the fractional part of the coefficient value. For example, in Figure 5-c , the bit to the right of the rightmost used bit will be used again. Alternatively, if the third variant is used, this can be achieved by using one more precision bit for the sample value. For example, in Figure 5-d In the second alternative, the bit to the right of the rightmost used bit is reused. In the second alternative, the integer part of the 10 bits of precision is increased by one bit (from 3 bits to 4 bits). This is accomplished by adding a higher precision bit to the coefficient value (i.e., increasing the range of possible coefficient values). For example, in Figure 5-cThe bits to the left of the leftmost used bit will be used again.

[0219] These fifth variant alternatives are of particular interest when using existing hardware with fixed-size registers, and where the cost of the multiplication is the same whenever the result fits in a register (e.g. for a software implementation that runs as an instruction for a CPU to process an operation that implies a register or memory location of a predefined size). It is also of interest whether some bits are not used when the hardware design uses existing predefined circuit templates to implement fixed-size operations (e.g. for development speed / cost issues). Then, in both cases, the cost of multiplication on, for example, 10 or 11 bits does not change (in terms of execution latency and / or in terms of the number of logic gates used for the implementation) if implemented, for example, with 16-bit inputs and 32-bit outputs, but the coding gain of CCALF is slightly better in the case of the second version because it uses one more bit of precision.

[0220] In an alternative, 8 samples (or less) are used in the filtering so that 2 bits (or more) can be saved in the summation operation, see e.g. Figure 7-f (which uses 8 samples (saving 2 bits)) or Figure 7-g (where only 4 samples are used (saving 3 bits)). In a variation of this alternative, the 2 bits (or more) saved in the summation are redistributed between the inputs to the multiplication, for example to have higher precision during filtering processes and to fully use the precision of the 16-bit registers.

[0221] Figure 7-c An example of a filter mode 706 using only 16 samples for filtering is provided. With this filter mode, the filtering operation becomes:

[0222]

[0223] This sample selection can be described as a "diamond" shape, where each sample has a position (x+u, y+v) relative to the reference sample, where u∈[-2, 2] and v∈[-1, 2], and when |u|=2, v∈[0, 1].

[0224] Figure 7-d Another example of a filter mode 707 using 16 samples for filtering is provided. This mode is more efficient than the filter mode 707 when using YUV 4:2:0 video representation. Figure 7-cThe mode in 707 is more suitable, where the chroma sample subsampling position is located in the middle of the positions of sample 5, sample 6, sample 9 and sample 10 in 707. According to a variant, two or more filter modes are available, and a filter mode is selected according to the chroma subsampling in YUV 4:2:0. In an alternative, the filter mode index is provided / signaled in the bitstream (e.g., in a parameter set (e.g., SPS or PPS)). With this filter mode, the filtering operation becomes:

[0225]

[0226] This sample selection can be described as each sample having a position (x+u, y+v) relative to a reference sample, where u∈[-1, 2] and v∈[-1, 2].

[0227] Figure 7-e An example of a filter mode 708 using 12 samples for filtering is provided. Figure 7-c This mode allows access to fewer samples and encoding fewer coefficients in the bitstream than the mode in , but does not allow further reduction in the number of bits used in the summation operation. With this filter mode, the filtering operation becomes:

[0228]

[0229] This sample selection can be described as each sample having a position (x+u, y+v) relative to a reference sample, where u∈[-1,1] and v∈[-1,2].

[0230] Figure 7-f An example of filter mode 709 using only 8 samples for filtering is provided. Figure 7-c This mode enables the number of bits used in the summation operation to be further reduced by one compared to the image in Figure 4-a Compared with the mode in , it allows to reduce two bits. Using this filter mode, the filtering operation becomes:

[0231]

[0232] This sample selection can be described as a "diamond shape" where each sample has a position (x+u, y+v) relative to the reference sample, where u∈[-1,1] and v∈[-1,2], and when |u|=1, v∈[0,1].

[0233] Figure 7-g An example of a filter mode 710 using only 4 samples for filtering is provided. Figure 7-f This mode enables the number of bits used in the summation operation to be further reduced by one compared to the mode in Figure 4-aCompared with the mode in , this allows three fewer bits. Using this filter mode, the filtering operation becomes:

[0234]

[0235] This sample selection can be described as each sample having a position (x+u, y+v) relative to a reference sample, where both u and v belong to [0, 1].

[0236] In some other variants, the cost of the addition is essentially zero. The multiplication input is constrained so that the multiplication output / result is a 16-bit signed number (i.e., 15 bits of precision plus 1 sign bit). Then, if 10 bits are used for the sample values, the coefficient values ​​are constrained to be within a given range with a given precision so that the coefficient values ​​can be represented on 5 bits plus a sign bit. In some of these variants, the multiplication operation is performed on a 16-bit register.

[0237] In one embodiment, the fractional precision of the coefficients is signaled in the bitstream (eg, in a parameter set (eg, APS, SPS, or PPS), or in a slice header) as first precision information.

[0238] In one embodiment, the range of coefficients (or an index into a set of possible ranges) is signaled in the bitstream (e.g., in a parameter set (e.g., APS, SPS, or PPS), or in a slice header) as second precision information.

[0239] In one embodiment, the number of reserved bits (i.e., KB) or alternatively the number of dead bits (i.e., DB) of a sample value is signaled in the bitstream (e.g., in a parameter set (e.g., APS, SPS, or PPS), or in a slice header) as third precision information.

[0240] In one embodiment, a combination of any one of the first precision information, the second precision information, and / or the third precision information is signaled in a bitstream (eg, in a parameter set (eg, APS, SPS, or PPS), or in a slice header).

[0241] In one variant, the allowed values ​​of such combination of any of the first precision information, the second precision information and / or the third precision information signaled in the bitstream will ensure that the worst case of the multiplication operation can be calculated using a fixed maximum number of bits, such as 16. In an alternative, the worst case also considers the maximum number of bits used during the addition operation.

[0242] In one embodiment, an index into a set of possible values ​​for a combination of any of the first precision information, the second precision information and / or the third precision information is signaled.

[0243] In one variant, such a set is defined to ensure that the worst case for a multiplication operation can be calculated using a fixed maximum number of bits, for example 16. In an alternative, the worst case also takes into account the maximum number of bits used during an addition operation.

[0244] In an alternative to an embodiment where a combination of any of the first precision information, the second precision information, and / or the third precision information includes the first precision information and the second precision information (i.e., the fractional precision and the range maintained), the alternative is to signal the mantissa size (in bits) and the fixed exponent of the floating point representation.

[0245] In one alternative, instead of using fixed point representation for the filter coefficients, a specific floating point representation is used, which guarantees computation using finite number of bit operations. In other words, a combination of any of the first precision information, the second precision information and / or the third precision information is signaled for each coefficient index.

[0246] It should be understood that in the described examples, variations and / or embodiments, depending on the bit depth and precision of the sample values ​​used, the coefficient range may be shifted up and / or down (i.e., multiplied by a power of 2) while shifting the least significant bit of the representation of the coefficient by the same amount (to maintain the same precision) to provide an equivalent mathematical result. That is, the [range] of coefficient values ​​(or sample values) is scaled by the inverse amount of any sample value (or [range] of coefficient values) scaling:

[0247]

[0248] This is the same as a bitwise shift, since a left bitwise shift of B is equivalent to multiplying by 2 B , and a right shift is roughly equivalent to integer division by 2 B (The difference is that for negative numbers, for left shift, rounding is towards negative infinity, while for integer division, rounding is towards 0).

[0249] It should also be understood that, although the simplification of the bit-wise representation of the sample values ​​and / or filter coefficients has been performed "during filtering" via the above-described bit-wise operations, these values ​​may also be pre-processed separately prior to filtering.

[0250] Signal

[0251] Figure 3 The input CCALF parameters 305 and 306 in may include flags for enabling / disabling CCALF at different levels of granularity. An example of signaling of a flag is provided below.

[0252] In some embodiments, a flag is signaled in a parameter set: for example, a sequence parameter set (SPS) or a picture parameter set (PPS). The flag enables the CCALF tool to be activated or deactivated at the sequence level or the picture level. When the CCALF tool is active in a picture, a flag is signaled per chroma component in the slice header. This enables CCALF to be enabled or disabled for each chroma component in the slice. When CCALF is enabled in a slice (i.e., CCLAF is enabled on at least one chroma component), for each chroma component for which CCALF is enabled, a ccalf_ctb_flag_cb[x][y] (for Cb components) or ccalf_ctb_flag_cr[x][y] (for Cr components) flag is encoded in the CTU data of each CTU of the slice, where x, y are the horizontal and vertical CTU coordinates / indexes in the picture. If CCALF is enabled or disabled in the chroma CTB, the flag enable is signaled. When CCALF is enabled in a chroma CTB, the CTB luma samples are corrected / filtered via CCALF chroma samples (eg, as discussed above).

[0253] The flag is signaled using CABAC (Context Adaptive Binary Arithmetic Coding) with 3 CABAC contexts per chroma component. For a given chroma component with index componentldx, the context is selected by counting the number of ccalf_ctb_flag_cb (for Cb components) or ccalf_ctb_flag_cr (for Cr components) active in the top ([x][y-1]) and left ([x-1][y]) CTUs belonging to the same slice (when they exist / are available) and using this value as the context index.

[0254] An example of signaling ccalf_ctb_flag_cb[x][y] and ccalf_ctb_flag_cr[x][y] is given in Table 1, using the same notations and conventions as in Generic Video Coding (Draft 5) corresponding to VTM-5.0.

[0255]

[0256] Table 1. Syntax of coding_tree_unit()

[0257] According to an embodiment, in order to be able to adapt and transmit the filter coefficients specifically for a slice, and as is usually done with slice parameters, the coefficients of the CCALF filter are signaled in the slice header. If CCALF is indicated as active for a given chroma component, the individual coefficients of the corresponding filter are encoded / decoded sequentially in / from the bitstream.

[0258] The coefficients of CCALF are usually encoded using Golomb variable length codes. However, when the number of bits representing the above coefficients is reduced, the encoding can be simplified. Specifically, since the number of bits used to store the coefficient values ​​is reduced, an encoding method that is easier to parse without losing encoding efficiency can be used.

[0259] Specifically, Golomb coding requires signaling of "tunable parameters" (M) to correctly decode the coefficients; this increases the bitstream overhead. The present invention involves signaling the coefficients directly in the slice header without reference to any other parameters. In other words, the coefficients are encoded using a fixed representation (i.e., without reference to "tunable parameters"). This can be described as "prefix-free" or "comma-free" because the codeword is a single element with no separators (inferred or explicitly encoded).

[0260] In a first alternative, which is particularly suitable when combining the first and second variants discussed above, the used / significant bits of the coefficient values ​​are stored as such in the bitstream without any compression (i.e., a fixed number of bits to represent the code, no matter what the value is). As an example, if the coefficients are as Figure 5-c As shown in , the coefficients can be stored on three bits in the bitstream: for example, a sign bit followed by two bits of fractional precision.

[0261] In a second alternative, a signed unary encoding is used to reduce the cost of low magnitude coefficients. The first bit in the bitstream indicates whether the coefficient is zero, then if the coefficient is not zero, the bit indicates the sign of the coefficient, and then a sequence of bits equal to 1 indicates the magnitude of the coefficient. The number of bits equal to 1 is equal to the absolute value of the "integer representation of the coefficient" minus 1. The integer representation of the coefficient is the integer value represented by the bits of the integer in 503. "Minus 1" is used because the information equal to zero is already encoded. Then, if the absolute value is below the maximum possible / authorized value, the bit equal to zero indicates the signaled end of the quantity. This corresponds to a unary encoding of the absolute value minus 1. The decoding process of this representation can be summarized by the algorithm described in Table 2.

[0262]

[0263] Table 2 Coefficient decoding algorithm for the second alternative

[0264] In some alternatives using an asymmetric range of possible coefficient values, as can be seen in the case of Table 2, when "CCALF_MAX_COEFF_VALUE" is not equal to "-CCALF_MIN_COEFF_VALUE", the maximum possible / authorized value of the quantity is different for negative and positive numbers. For example, if using Figure 5-c, and if the range of possible coefficient values ​​is [-0.03125; 0.03125] (i.e., [-0.03125; 0.0234375]), the maximum possible / authorized value for negative numbers is 4 (because -0.03125 is represented by the integer -4), and the maximum possible / authorized value for positive numbers is 3 (because 0.0234375 is represented by the integer 3). In this example, the first bit equal to 1 means that the number is zero, and the sign bit equal to 1 means that the number is negative. The coefficients in the bitstream are then represented as: "01111" (i.e., -4 = CCALF_MIN_COEFF_VALUE) for -0.03125, "01110" (i.e., -3) for -0.0234 375, "0110" (i.e., -2) for -0.015625, "010" (i.e., -1) for -0.0078125, "1" for 0.0, "000" (i.e., 1) for 0.0078125, "0010" (i.e., 2) for 0.015625, and "0011" (i.e., 3=CCALF_MAX_COEFF_VALUE) for 0.0234375. In some alternatives, the first bit equal to 1 can mean that the number is non-zero. In some alternatives, the sign bit equal to 1 can mean that the number is positive. In some alternatives, the unary encoding of the number can be a sequence of bits equal to 0 followed by bits equal to 1 (if the number is below the maximum value).

[0265] In a third alternative, the bitstream first provides a unary encoding of the integer representation of the absolute value of the coefficient. If non-zero, this is followed by a sign bit. The algorithm in Table 3 describes an example of a decoding process for this representation. In this third alternative, the sign is unknown before the absolute value of the coefficient is decoded. The maximum value used for the unary encoding is then the same for positive and negative values ​​(CCALF_MAX_COEFF_ABSOLUTE_VALUE in Table 3). As an example, Figure 5-c The representation is used with a range of possible coefficient values ​​set equal to [-0.0234375; 0.0234375] (i.e., CCALF_MAX_COEFF_ABSOLUTE_VALUE=3). The representation of the coefficients in the bitstream is then: "1111" (i.e., -3) for -0.0234375, "1101" (i.e., -2) for -0.015625, "101" (i.e., -1) for -0.0078125, "0" for 0.0, "100" (i.e., 1) for 0.0078125, "1100" (i.e., 2) for 0.015625, and "1110" (i.e., 3) for 0.0234375.

[0266]

[0267] Table 3 Coefficient decoding algorithm for the third alternative

[0268] In some variants, the coefficients of the CCALF filter are signaled in the APS (or another parameter set). When slice_ccalf_enable_flag_cb is equal to 1 in the slice header, the slice header also provides the APS (or other parameter set) index / identifier of the APS (or other parameter set) where the filter coefficients of the CCALF filter must be retrieved for the Cb chroma component. When slice_ccalf_enable_flag_cr is equal to 1 in the slice header, the slice header also provides the APS (or other parameter set) index of the APS (or other parameter set) where the filter coefficients of the CCALF filter must be retrieved for the Cr chroma component. When using e.g. Figure 4-a When the filter mode shown in , the number of filter coefficients present in the bitstream is smaller than the number of samples used in the filter shape, since some coefficients are shared by several sample positions.

[0269] Implementation of the embodiment of the present invention

[0270] One or more of the aforementioned embodiments may be implemented in the form of an encoder or decoder that performs the method steps of one or more of the aforementioned embodiments. The following embodiments illustrate such an implementation.

[0271] For example, the adaptive loop filter according to any of the foregoing embodiments may be used to generate Figure 8 The post-filtering 9415 performed by the encoder in Fig. 9 Post-filtering performed by the decoder in 9567.

[0272] Figure 8 A block diagram of an encoder according to an embodiment of the invention is shown. The encoder is represented by connected modules, each module being suitable for implementing at least one corresponding step of a method for implementing at least one embodiment of encoding an image of a sequence of images according to one or more embodiments of the invention, e.g. in the form of programming instructions executed by a central processing unit (CPU) of the device.

[0273] An original sequence of digital images i0 to in 9401 is received as input by the encoder 9400. Each digital image is represented by a set of samples, sometimes also referred to as pixels (hereinafter, they are referred to as pixels). After the encoding process is implemented, a bit stream 9410 is output by the encoder 9400. The bit stream 9410 includes data of a plurality of coding units or image portions such as slices, each slice including a slice header for transmitting an encoding value of an encoding parameter for encoding the slice and a slice body including encoded video data.

[0274] The input digital images i0 to in 9401 are divided into pixel blocks by module 9402. Blocks correspond to image parts and can be of variable size (e.g., 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 pixels and several rectangular block sizes can also be considered). A coding mode is selected for each input block. Two categories of coding modes are provided: coding modes based on spatial prediction coding (intra-frame prediction) and coding modes based on temporal prediction (inter-frame coding, MERGE, SKIP). Possible coding modes are tested.

[0275] Module 9403 implements an intra prediction process in which a given block to be encoded is predicted by a predictor calculated from its neighboring pixels. If intra coding is selected, an indication of the selected intra predictor and the difference between the given block and its predictor are encoded to provide a residual.

[0276] Temporal prediction is implemented by a motion estimation module 9404 and a motion compensation module 9405. First, a reference image is selected from a set of reference images 9416, and a portion of the reference image (also referred to as a reference region or image portion, which is the region closest to a given block to be encoded (closest in terms of pixel value similarity)) is selected by the motion estimation module 9404. The motion compensation module 9405 then uses the selected region to predict the block to be encoded. The difference (also referred to as residual block / data) between the selected reference region and the given block is calculated by the motion compensation module 9405. The selected reference region is indicated using motion information (e.g., a motion vector).

[0277] Therefore, in both cases (spatial and temporal prediction), when not in SKIP mode, the residual is calculated by subtracting the predictor from the original block.

[0278] In the intra prediction implemented by module 9403, the prediction direction is encoded. In the inter prediction implemented by modules 9404, 9405, 9416, 9418, 9417, at least one motion vector or information (data) for identifying such a motion vector is encoded for temporal prediction.

[0279] If inter prediction is selected, information related to the motion vector and the residual block is encoded. To further reduce the bit rate, the motion vector is encoded by the difference relative to the motion vector predictor, assuming that the motion is uniform. The motion vector predictor from the motion information predictor candidate set is obtained from the motion vector field 9418 by the motion vector prediction and encoding module 9417.

[0280] The encoder 9400 also includes a selection module 9406 for selecting a coding mode by applying a coding cost criterion such as a rate-distortion criterion. To further reduce redundancy, a transform module 9407 applies a transform (such as DCT) to the residual block, and the obtained transform data is then quantized by a quantization module 9408 and entropy encoded by an entropy encoding module 9409. Finally, when not in SKIP mode and the selected coding mode requires encoding of the residual block, the encoded residual block of the current block being encoded is inserted into the bitstream 9410.

[0281] The encoder 9400 also performs decoding of the encoded image to generate a reference image for motion estimation of subsequent images (e.g., a reference image in a reference image / picture 9416). This enables the encoder and decoder receiving the bitstream to have the same reference frame (e.g., using a reconstructed image or a reconstructed image portion). The inverse quantization ("dequantization") module 9411 performs inverse quantization ("dequantization") of the quantized data, which is then inversely transformed by the inverse transform module 9412. The intra-frame prediction module 9413 uses the prediction information to determine which predictor to use for a given block, and the motion compensation module 9414 actually adds the residual obtained by the module 9412 to a reference area obtained from a set of reference images 9416. The reconstructed frame (image or image portion) of pixels is then filtered by module 9415 with post filtering to obtain another reference image of the set of reference images 9416.

[0282] Fig. 9 A block diagram of a decoder 9560 that can be used to receive data from an encoder according to an embodiment of the present invention is shown. The decoder is represented by connected modules, each module being suitable for implementing the corresponding steps of the method implemented by the decoder 9560, for example in the form of programming instructions executed by the CPU of the device.

[0283] The decoder 9560 receives a bitstream 9561, which includes coding units (e.g., data corresponding to image parts, blocks, or coding units), each coding unit consisting of a header containing information about coding parameters and a body containing coded video data. Figure 8 As illustrated, the coded video data is entropy encoded at a predetermined number of bits for a given image portion (e.g., a block or CU), and motion information (e.g., an index to a motion vector predictor) is encoded. The received coded video data is entropy decoded by module 9562. The residual data is then dequantized by module 9563, and then an inverse transform is applied by module 9564 to obtain pixel values.

[0284] The mode data indicating the coding mode is also entropy decoded, and based on the mode, the coding block (unit / set / group) of the image data is intra-type decoded or inter-type decoded. In the case of intra-frame mode, the intra-frame prediction module 9565 determines the intra-frame predictor based on the intra-frame prediction mode specified in the bitstream (for example, the intra-frame prediction mode can be determined using the data provided in the bitstream). If the mode is an inter-frame mode, motion prediction information is extracted / obtained from the bitstream to find (identify) the reference area used by the encoder. For example, motion prediction information includes a reference frame index and a motion vector residual. The motion vector predictor is added to the motion vector residual to obtain a motion vector by the motion vector decoding module 9570.

[0285] The motion vector decoding module 9570 applies motion vector decoding to each image portion (e.g., current block or CU) encoded by motion prediction. Once the index of the motion vector predictor of the current block is obtained, the actual value of the motion vector associated with the image portion (e.g., current block or CU) can be decoded and used to apply motion compensation by module 9566. The reference image portion indicated by the decoded motion vector is extracted / obtained from the set of reference images 9568 so that module 9566 can perform motion compensation. The motion vector field data 9571 is updated with the decoded motion vector for use in predicting subsequently decoded motion vectors.

[0286] Finally, a decoded block is obtained. Where appropriate, post-filtering is applied by post-filtering module 9567. Finally, a decoded video signal 9569 is obtained and provided by decoder 9560.

[0287] Fig.10 The data communication system in which one or more embodiments of the present invention can be implemented is illustrated. The data communication system includes a transmission device (in this case, a server 9201) that is operable to transmit data packets of a data stream to a receiving device (in this case, a client terminal 9202) via a data communication network 9200. The data communication network 9200 can be a wide area network (WAN) or a local area network (LAN). Such a network can be, for example, a wireless network (Wifi / 802.11a or 802.11b or 802.11g), an Ethernet network, an Internet network, or a hybrid network consisting of several different networks. In a specific embodiment of the present invention, the data communication system can be a digital television broadcasting system in which the server 9201 sends the same data content to multiple clients.

[0288] The data stream 9204 provided by the server 9201 can be composed of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams can be captured by the server 9201 using a microphone and a camera, respectively. In some embodiments, the data stream can be stored on the server 9201 or received by the server 9201 from other data providers, or generated at the server 9201. The server 9201 is provided with an encoder for encoding video and audio streams, in particular to provide a compressed bit stream for transmission, which is a more compact representation of the data presented as the input of the encoder. In order to obtain a better ratio of the quality of the transmitted data to the amount of the transmitted data, the video data can be compressed, for example, according to the high-efficiency video coding (HEVC) format or the H.264 / advanced video coding (AVC) format or the versatile video coding (VVC) format. The client 9202 receives the transmitted bit stream and decodes the reconstructed bit stream to reproduce the video image on the display device and reproduce the audio data using a speaker.

[0289] Although a streaming scenario is considered in this embodiment, it will be appreciated that in some embodiments of the invention, data communication between the encoder and decoder may be performed using, for example, a media storage device such as an optical disk, etc. In one or more embodiments of the invention, a video image may be transmitted along with data representing a compensating offset to be applied to reconstructed pixels of the image to provide filtered pixels in the final image.

[0290] Fig.11 A processing device 9300 configured to implement at least one embodiment of the present invention is schematically illustrated. The processing device 9300 may be a device such as a microcomputer, a workstation, a user terminal, or a light portable device. The device / apparatus 9300 includes a communication bus 9313, which is connected to:

[0291] - a central processing unit 9311 denoted as CPU, such as a microprocessor, etc.;

[0292] - a read-only memory 9307, denoted ROM, for storing computer programs / instructions for operating the device 9300 and / or implementing the invention;

[0293] - a random access memory 9312 represented as a RAM, which is used to store executable codes of the method of the embodiment of the present invention, and registers suitable for recording variables and parameters required for implementing the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the embodiment of the present invention; and

[0294] - A communication interface 9302 connected to a communication network 9303, through which digital data to be processed are transmitted or received.

[0295] Optionally, the device 9300 may further include the following components:

[0296] - a data storage component 9304, such as a hard disk, for storing a computer program for implementing the method of one or more embodiments of the present invention and data used or generated during the implementation of one or more embodiments of the present invention;

[0297] a disk drive 9305 for a disk 9306 (e.g. a storage medium), the disk drive 9305 being adapted to read data from the disk 9306 or to write data to the disk 9306; or,

[0298] - A screen 9309 for displaying data and / or serving as a graphical interface for interaction with a user by means of a keyboard 9310, a touch screen or any other indication / input means.

[0299] The device 9300 may be connected to various peripheral devices such as a digital camera 9320 or a microphone 9308 , each of which is connected to an input / output card (not shown) to provide multimedia data to the device 9300 .

[0300] The communication bus 9313 provides communication and interoperability between the various elements included in or connected to the device 9300. The representation of the bus is not limiting, and in particular, the central processing unit 9311 is operable to communicate instructions to any element of the device 9300 directly or with the aid of other elements of the device 9300.

[0301] Disk 9306 can be replaced by any information medium, such as a rewritable or non-rewritable compact disk (CD-ROM), a ZIP disk or a memory card, and in general, by an information storage component that can be read by a microcomputer or a processor, the disk 9306 being integrated into the device or not, possibly removable and suitable for storing one or more programs, the execution of which enables the implementation of the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the present invention.

[0302] The executable code may be stored in a read-only memory 9307, on a hard disk 9304 or on a removable digital medium such as, for example, a disk 9306 as previously described, etc. According to a variant, the executable code of the program may be received via the interface 9302 by means of a communication network 9303 to be stored in one of the storage means of the device 9300 (for example, the hard disk 9304) before execution.

[0303] The central processing unit 9311 is suitable for controlling and directing the execution of instructions or parts of software code of one or more programs according to the present invention, the execution of instructions stored in one of the above-mentioned storage means. At power-on, one or more programs stored in non-volatile memory (for example, on hard disk 9304, disk 9306 or in read-only memory 9307) are transferred to random access memory 9312 (which then contains the executable code of one or more programs) and registers for storing variables and parameters necessary for implementing the present invention.

[0304] In this embodiment, the device is a programmable device that implements the invention using software. Alternatively, however, the invention may be implemented in hardware, for example in the form of an application specific integrated circuit (ASIC).

[0305] It should also be understood that according to other embodiments of the present invention, a decoder according to the above embodiment is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet or any other type of device (e.g., a display device) capable of providing / displaying content to a user. According to yet another embodiment, an encoder according to the above embodiment is provided in a camera device, which also includes a camera, a video camera or a network camera (e.g., a closed-circuit television or video surveillance camera) for capturing and providing content for encoding by the encoder. See below Fig.12 and 13 Two such embodiments are provided.

[0306] Fig.12 is a diagram illustrating a network camera system 9450 including a network camera 9452 and a client device 9454 .

[0307] The network camera 9452 includes a camera unit 9456, an encoding unit 9458, a communication unit 9460, and a control unit 9462. The network camera 9452 and the client device 9454 are connected to each other via the network 9200 so as to be able to communicate with each other. The camera unit 9456 includes a lens and an image sensor (e.g., a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS)), and captures an image of an object and generates image data based on the image. The image may be a still image or a video image. The camera unit may also include a zoom component and / or a pan component that are respectively suitable for zooming or panning (optically or digitally). The encoding unit 9458 encodes the image data by using the encoding method described in the aforementioned embodiment. The encoding unit 9458 uses at least one of the encoding methods described in the aforementioned embodiment. For another example, the encoding unit 9458 may use a combination of the encoding methods described in the aforementioned embodiment.

[0308] The communication unit 9460 of the network camera 9452 transmits the encoded image data encoded by the encoding unit 9458 to the client device 9454. In addition, the communication unit 9460 can also receive commands from the client device 9454. The commands include commands for setting parameters for encoding by the encoding unit 9458. The control unit 9462 controls other units in the network camera 9452 according to the commands received by the communication unit 9460 or user input.

[0309] The client device 9454 includes a communication unit 9464, a decoding unit 9466, and a control unit 9468. The communication unit 9464 of the client device 9454 can transmit a command to the network camera 9452. In addition, the communication unit 9464 of the client device 9454 receives the encoded image data from the network camera 9452. The decoding unit 9466 decodes the encoded image data by using the decoding method described in one or more of the aforementioned embodiments. For another example, the decoding unit 9466 can use a combination of the decoding methods described in the aforementioned embodiments. The control unit 9468 of the client device 9454 controls other units in the client device 9454 according to the user operation or command received by the communication unit 9464. The control unit 9468 of the client device 9454 can also control the display device 9470 to display the image decoded by the decoding unit 9466.

[0310] The control unit 9468 of the client device 9454 also controls the display device 9470 to display a GUI (Graphical User Interface) for specifying values ​​of parameters of the network camera 9452 (e.g., parameters for encoding by the encoding unit 9458). The control unit 9468 of the client device 9454 may also control other units in the client device 9454 according to user operation input to the GUI displayed by the display device 9470. The control unit 9468 of the client device 9454 may also control the communication unit 9464 of the client device 9454 according to user operation input to the GUI displayed by the display device 9470 to transmit a command for specifying values ​​of parameters of the network camera 9452 to the network camera 9452.

[0311] Fig.13 9500 is a diagram illustrating a smartphone 9500. The smartphone 9500 includes a communication unit 9502, a decoding / encoding unit 9504, a control unit 9506, and a display unit 9508.

[0312] The communication unit 9502 receives the encoded image data via the network. The decoding / encoding unit 9504 decodes the encoded image data received by the communication unit 9502. The decoding unit 9504 decodes the encoded image data by using the decoding method described in one or more of the aforementioned embodiments. The decoding / encoding unit 9504 may also use at least one of the encoding or decoding methods described in the aforementioned embodiments. For another example, the decoding / encoding unit 9504 may use a combination of the decoding or encoding methods described in the aforementioned embodiments.

[0313] The control unit 9506 controls other units in the smartphone 9500 according to a user operation or a command received by the communication unit 9502. For example, the control unit 9506 controls the display unit 9508 to display an image decoded by the decoding / encoding unit 9504.

[0314] The smartphone may also include an image recording device 9510 (e.g., a digital camera and associated circuitry) for recording images or videos. Such recorded images or videos may be encoded by the decoding / encoding unit 9504 under the instruction of the control unit 9506. The smartphone may also include a sensor 9512 suitable for sensing the posture of the mobile device. Such a sensor may include an accelerometer, a gyroscope, a compass, a global positioning (GPS) unit, or a similar position sensor. Such a sensor 2212 may determine whether the smartphone changes posture, and such information may be used when encoding the video stream.

[0315] Although the present invention has been described with reference to the embodiments, it should be understood that the present invention is not limited to the disclosed embodiments. It will be appreciated by those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention as defined in the appended claims. All features disclosed in this specification (including any appended claims, abstracts and drawings) and / or all steps of any disclosed method or process may be combined in any combination, except for at least some mutually exclusive combinations of such features and / or steps. Unless otherwise expressly stated, each feature disclosed in this specification (including any appended claims, abstracts and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless otherwise expressly stated, each feature disclosed is only an example of a general series of equivalent or similar features.

[0316] It should also be understood that any results of the above-described comparison, determination, evaluation, selection, execution, proceeding, or consideration (e.g., selections made during an encoding or filtering process) may be indicated in data in the bitstream (e.g., a flag or information indicating the result) or may be determined / inferred from data in the bitstream so that the indicated or determined / inferred result may be used in the processing rather than actually comparing, determining, evaluating, selecting, executing, proceeding, or considering, for example, during a decoding process.

[0317] In the claims, the word "comprising" does not exclude other elements or steps and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage. Reference signs appearing in the claims are by way of illustration only and shall not have a limiting effect on the scope of the claims.

[0318] In the foregoing embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or sent through a computer-readable medium as one or more instructions or codes, and may be executed by a hardware-based processing unit.

[0319] Computer-readable media may include computer-readable storage media, which corresponds to tangible media such as data storage media or communication media including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for implementing the techniques described in the present invention. A computer program product may include a computer-readable medium.

[0320] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store the desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave, etc.) is used to send instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave, etc.) is included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connection, carrier wave, signal or other transient media, but are directed to non-transient tangible storage media. The disk and disc used here include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blue-ray disc, wherein the disc usually copies data magnetically, and the disc reproduces data optically by laser. Combinations of the above should also be included within the scope of computer-readable media.

[0321] Instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

Claims

1. A method for filtering an image portion using a cross-component adaptive loop filter (CCALF), the method comprising: Obtaining filter coefficients and first component sample values ​​corresponding to samples surrounding the reference sample; as well as using the filter coefficients for the cross-component adaptive loop filter and the first component sample values ​​to produce an output of the cross-component adaptive loop filter for a second component, The method comprises: constraining the possible values ​​of each filter coefficient, wherein possible values ​​of the filter coefficients are constrained to a range represented by values ​​of powers of 2 defined by a predetermined number of bits and are limited to a plurality of values, wherein the plurality of values ​​includes values ​​represented by powers of 2 and zero, and The method also includes combining an output of the cross-component adaptive loop filter with an output of a filter corresponding to a different component.

2. The method according to claim 1, wherein: The constrained number of bits used to represent each filter coefficient and / or first component sample value enables the filter output to be represented on 16 bits or less.

3. The method according to claim 2, wherein: The first component is luminance and the different components are chrominance components.

4. The method according to claim 3, wherein: The constraints include using reduced fixed-point fractional precision for the values ​​of the filter coefficients.

5. The method according to claim 4, wherein: The reduced fixed-point fractional precision is 7 digits.

6. The method according to claim 1, wherein: The filtering operation includes multiplication of sample values ​​and filter coefficients, and the multiplication is implemented via bitwise shifting and / or combination with another filter coefficient to replace the multiplication operation with bitwise shifting and addition operations.

7. The method according to claim 1, further comprising: determining whether a cross-component adaptive loop filter is indicated for a given component; as well as The filter coefficients are decoded from the bitstream.

8. The method according to claim 1, further comprising: determining whether a cross-component adaptive loop filter is indicated for a given component; as well as The filter coefficients are encoded into a bitstream.

9. The method according to claim 7, wherein: Determining whether CCALF is indicated for a given component includes decoding a flag.

10. The method according to claim 8, wherein: Determining whether CCALF is indicated for a given component includes encoding a flag into a slice header.

11. The method according to claim 8, wherein: The coefficients of the CCALF are directly encoded in the parameter set.

12. The method according to claim 11, wherein: The parameter set is an adaptive parameter set, namely, APS.

13. A method of encoding an image using the method according to any one of claims 1 to 6, 8 and 10 to 12.

14. A method of decoding an image using the method according to any one of claims 1 to 7 and 9.

15. An apparatus for filtering an image portion using a cross-component adaptive loop filter (CCALF), comprising: means for obtaining filter coefficients and first component sample values ​​corresponding to samples surrounding a reference sample; as well as means for using the filter coefficients for the cross-component adaptive loop filter and the first component sample values ​​to produce an output of the cross-component adaptive loop filter for a second component, wherein the apparatus is configured to constrain possible values ​​of each filter coefficient, wherein possible values ​​of the filter coefficients are constrained to a range represented by values ​​of powers of 2 defined by a predetermined number of bits and are limited to a plurality of values, wherein the plurality of values ​​includes values ​​represented by powers of 2 and zero, and The apparatus is further configured to combine an output of the cross-component adaptive loop filter with an output of a filter corresponding to a different component. 16 . A non-transitory computer-readable storage medium storing instructions of a program which, when executed by a computer or a processor, causes the computer or the processor to perform the method according to any one of claims 1 to 14.

Images