Method and apparatus for deblocking image

Abstract

To provide an apparatus, a method, and a program for deblocking an image.SOLUTION: A method for deblocking an image includes: determining at least one boundary between a first block of a sample and a second block of the sample; determining boundary strength (BS) according to at least one of a prediction mode for the first block and a prediction mode for the second block; and filtering, according to the boundary strength, the first block and the second block of the sample adjacent to the at least one boundary on the basis of a deblocking filter (DBF). When the prediction mode for the first block is a weighted prediction mode, the boundary strength depends on the relative weight of the sample used in predicting the first block of the sample according to the weighted prediction mode for the first block. The same applies to the second block.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] Technical Field At least one of the present embodiments relates generally to a method or apparatus for, for example, video encoding or decoding, and more particularly to a method or apparatus for deblocking an image. [Background technology]

[0002] background The technical field of one or more implementations relates generally to video compression. At least some embodiments relate to improving compression efficiency over existing video compression systems such as HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2, described in "ITU-T H.265 Telecommunications Standardization Sector of the ITU (10 / 2014), Series H: Audiovisual and Multimedia Systems, Infrastructure for Audiovisual Services - Coding of Moving Images, High Efficiency Video Coding, Recommendation ITU-T H.265"), or over developing video compression systems such as VVC (Versatile Video Coding, an emerging standard being developed by JVET, the Joint Video Experts Team).

[0003] To achieve high compression efficiency, image and video coding schemes typically employ image segmentation, prediction, including motion vector prediction, and transformation to exploit spatial and temporal redundancy in video content. In general, intra- or inter-prediction is used to exploit intra-frame or inter-frame correlation, and then the difference between the original and predicted image, often denoted as prediction error or prediction residual, is transformed into frequency domain coefficients, which are quantized and entropy coded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy decoding, inverse quantization, inverse transformation, and prediction.

[0004] In codecs such as HEVC or VVC, a deblocking filter DBF is applied after the picture is reconstructed, aiming to reduce blocking artifacts by smoothing sample values ​​near block edges. The deblocking filter is defined using a traditional square or rectangular block division shape with a size equal to at least four samples of the luma samples. Then, the selection of the motion vector, reference index, and reference deblocking sample to be used for the determination of the deblocking filter strength is performed. For example, Norkin et al. describe the principle of deblocking in "HEVC Deblocking Filter" (IEEE Transactions on Circuits and Systems for Video Technology, vol.22 No12, Dec. 2012).

[0005] Modern video codec approaches consider new coding (prediction) modes in which prediction samples are constructed using combinations of non-square or non-rectangular blocks (e.g., triangles) and / or unequal (e.g., Generalized bi-prediction (GBi), also called Bi-prediction with CU level Weights (BCW)) and / or spatially variable weightings (e.g., Multiple Hypothesis or Combined Inter merge Intra Prediction (CIIP)). Therefore, it is desirable to adapt the deblocking filter process to such coding modes in order to still efficiently reduce blocking artifacts. Summary of the Invention

[0006] overview SUMMARY OF THE PRESENT EMBODIMENT It is an object of the present invention to overcome at least one of the disadvantages of the prior art. [Means for solving the problem]

[0007] To this end, according to a general aspect of at least one embodiment, a method for deblocking a part of an image is presented. The method includes: determining at least one boundary between a first block of samples and a second block of samples, the first block and the second block belonging to a part of the image to be filtered; determining a boundary strength according to at least one of a prediction mode of the first block and a prediction mode of the second block; and filtering the at least one boundary according to the boundary strength, i.e. filtering the samples of the first and second blocks adjacent to the at least one boundary. If the prediction mode of the first block is a weighted prediction mode, the boundary strength further depends on the relative weight of the samples used in predicting the first block of samples according to the weighted prediction mode of the first block. Naturally, the same applies to the second block without distinction: if the prediction mode of the second block is a weighted prediction mode, the boundary strength further depends on the relative weight of the samples used in predicting the second block of samples according to the weighted prediction mode of the second block. A weighted prediction mode is a coding mode in which the prediction is obtained from a weighted combination of a first predictor determined using a first prediction mode and a second predictor determined using a second prediction mode.

[0008] According to another general aspect of at least one embodiment, a method for encoding a block in image coding is presented, the method comprising: reconstructing an image portion and filtering the reconstructed image portion according to any of the embodiments of the deblocking method.

[0009] According to another general aspect of at least one embodiment, a method is presented for decoding a block of an image, the method including decoding a portion of the image and filtering the decoded image portion according to any of the embodiments of the deblocking method.

[0010] According to another general aspect of at least one embodiment, an apparatus for video encoding is presented, the apparatus comprising means for performing any of the embodiments of the encoding method.

[0011] According to another general aspect of at least one embodiment, an apparatus for video decoding is presented, the apparatus comprising means for performing any of the embodiments of the decoding method.

[0012] According to another general aspect of at least one embodiment, there is provided an apparatus for video encoding, the apparatus comprising one or more processors and at least one memory, the one or more processors configured to perform any of the embodiments of the encoding method.

[0013] According to another general aspect of at least one embodiment, there is provided an apparatus for video decoding, the apparatus comprising one or more processors and at least one memory, the one or more processors configured to perform any of the embodiments of the decoding method.

[0014] According to another general aspect of at least one embodiment, the weighted prediction mode is one of generalized bidirectional prediction, bidirectional prediction with coding unit level weights, multiple hypothesis prediction combining intra- and inter-predicted samples or combining inter- and inter-predicted samples, combined inter-merge-intra prediction combining inter-merge and intra-predicted samples, geometric prediction combining inter-predicted samples according to a geometric partitioning of the block, and triangular prediction combining inter-predicted samples along the diagonal edges of the block.

[0015] According to another general aspect of at least one embodiment, predicting the first block of samples according to a weighted prediction mode of the first block includes predicting the samples of the first block as a weighted combination of a first predictor determined using the first prediction mode and a second predictor determined using the second prediction mode, wherein the samples of the first block are considered as using the first prediction mode for determining boundary strength if a weight of the first predictor is above a level.

[0016] According to another general aspect of at least one embodiment, the first prediction mode is a one intra prediction mode, and samples of the first block are considered to use the intra prediction mode to determine the boundary strength. In one variant, the first prediction mode is an intra planar prediction mode, and samples of the first block are considered to use the intra prediction mode to determine the boundary strength. In one variant, in the case of the intra prediction mode, the boundary strength is set to strong.

[0017] According to another general aspect of at least one embodiment, the first prediction mode is an inter prediction mode, and the samples of the first block are considered to use the inter prediction mode to determine the boundary strength. In one variant, the first prediction mode is an inter bi-directional prediction mode, and the samples of the first block are considered to use the inter bi-directional prediction mode to determine the boundary strength. In another variant, the first prediction mode is an inter uni-directional prediction mode, and the samples of the first block are considered to use the inter uni-directional prediction mode to determine the boundary strength.

[0018] According to another general aspect of at least one embodiment, the level in the BS's decision is set to zero.

[0019] According to another general aspect of at least one embodiment, a boundary strength is determined for a group of samples of a block that share the same relative weight, the group of samples including at least one sample, 4x4 samples, up to all samples of the block.

[0020] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is presented that includes data content generated according to the method or apparatus of any of the preceding descriptions.

[0021] According to another general aspect of at least one embodiment there is provided a signal or bitstream including video data generated in accordance with the method or apparatus of any of the preceding descriptions.

[0022] One or more of the embodiments also provide a computer readable storage medium having stored thereon instructions for deblocking, encoding, or decoding video data according to any of the methods described above. The embodiments also provide a computer readable storage medium having stored thereon a bitstream generated according to the methods described above. The embodiments also provide methods and apparatus for transmitting a bitstream generated according to the methods described above. The embodiments also provide a computer program product including instructions for performing any of the methods described above. [Brief description of the drawings]

[0023] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1a] FIG. 1a shows an exemplary flow chart of an in-loop post-filtering stage in a decoder architecture. [Figure 1b] FIG. 1b shows an example of a decoded image before (left) and after (right) a deblocking filter. [Diagram 2] FIG. 2 shows an example of a block boundary sample (left) with blocking artifacts between two blocks of samples (right). [Diagram 3] FIG. 3 shows an exemplary flow chart of the determination of a Boundary Strength (BS) parameter in a method of a deblocking filter according to a particular embodiment. [Figure 4] FIG. 4 shows an exemplary flow chart of determining the filter strength (FS) in a method of a deblocking filter according to a particular embodiment. [Diagram 5] FIG. 5 shows an example of multiple hypothesis prediction for a combination of inter and intra modes. [Figure 6] FIG. 6 shows an example of a non-rectangular partitioning (top) and an example of diagonal weighting of overlapped block motion compensation associated with a triangular partitioning (bottom). [Figure 7] FIG. 7 illustrates an example of a triangle logical partition (left) and a blending map (right) according to certain embodiments. [Figure 8] FIG. 8 illustrates an example of a method for deblocking portions of an image in accordance with a general aspect of at least one embodiment. [Figure 9] FIG. 9 shows a flow chart of the boundary strength determination in the method of the deblocking filter according to a particular embodiment. [Figure 10] FIG. 10 shows two flow charts of the storage of CU parameters used in the determination of the boundary strength (BS) parameter in a method of a deblocking filter according to a particular embodiment. [Figure 11] FIG. 11 shows a block diagram of an embodiment of a video encoder in which various aspects of the embodiments may be implemented. [Figure 12] FIG. 12 shows a block diagram of an embodiment of a video decoder in which various aspects of the embodiments may be implemented. [Figure 13] FIG. 13 illustrates a block diagram of an exemplary device in which various aspects of the embodiments may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Detailed Description It should be understood that the figures and descriptions are simplified to illustrate elements relevant for a clear understanding of the present principles, while excluding many other elements found in a typical encoding and / or decoding device for purposes of clarity. It will be understood that although the terms first and second may be used herein to describe various elements, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another.

[0025] Various embodiments are described with respect to encoding / decoding an image, which may be applied to encode / decode portions of an image, such as slices or tiles, groups of tiles, or entire sequences of images.

[0026] Various methods are described above, each of which includes one or more steps or actions for achieving the method described above. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be varied or combined.

[0027] At least some embodiments relate to adapting the deblocking filter DBF when a block is coded using one of the following three prediction modes: Generalized Bidirectional Prediction (GBi) or Bidirectional Prediction with CU-level Weights (BCW), A simplified version called Multi-Hypothesis (MH), or Combined Inter-Merge-Intra Prediction (CIIP), Geometric or triangular mode.

[0028] Advantageously, the present principles increase the efficiency of prior art methods by preserving the intent and efficiency of DBF in block-based video codecs where non-square or non-rectangular blocks are supported and / or where combinations of blocks with variable weightings are supported.

[0029] In the following, a general embodiment for the deblocking method is disclosed, followed by at least three prediction modes for which the deblocking filter is adapted. Then, several embodiments of modified deblocking filters are disclosed.

[0030] General embodiment for the deblocking method and exemplary prediction modes to which the deblocking filter is adapted Traditional video coding schemes include an in-loop filtering process to improve the quality of the reconstructed image. In-loop filtering can include several filtering processes, such as deblocking filtering (DBF), sample adaptive offset (SAO) filtering in HEVC, and / or adaptive loop filtering (ALF), such as the Wiener filter. These filters can be applied consecutively in this order or in a different order. FIG. 1a shows an example flow chart of the in-loop post-filtering stage in a decoder architecture. Typically, the deblocking filter (DBF) is one of the coding artifact reduction post-filters (20) and is applied after the block samples are reconstructed (10). The post-filtered picture can be displayed and possibly stored in a decoded picture buffer (30) to be used to build a motion compensated prediction. Advantageously, the same in-loop post-filtering is also used in the encoder for the reconstructed image used in the temporal prediction.

[0031] Figure 1b shows an example of a decoded image before (left) and after (right) a deblocking filter. The deblocking filter is applied after the picture is reconstructed (10). Its purpose is to reduce blocking artifacts by smoothing sample values ​​near block edges, as shown on Figure 1b.

[0032] Figure 2 shows an example of a block boundary sample (left) with blocking artifacts between two blocks of samples P and Q (right). Samples P={p0,p1,p2,p3} and Q={q0,q1,q2,q3} belong to two adjacent blocks P and Q. The samples belong to square 4x4 blocks P and Q as shown on the right side of Figure 2, where sample P={p0 i ,p1 i,p2 i ,p3 i The second index i in} is the row index in the 4X4 block. In general, the visibility of the artifacts is proportional to the relative difference between the sample values ​​P and Q as shown on the left side of Figure 2. That is why DBF filtering performs a sample smoothing S across the block edges. The parameters of the smoothing / filtering function S are: - Boundary Strength (BS) = {0-weak, 1-normal, or 2-strong} - Sample values ​​of blocks P and Q

[0033] The determination of DBF parameters for a square block is done by dividing the set of row (or column) samples on each side of the vertical (or horizontal) boundary by p3 i ,p2 i ,p1 i ,p0 i ,q0 i ,q1 i ,q2 i ,q3 i} every}.

[0034] FIG. 3 illustrates an exemplary flow chart 300 of determining a boundary strength (BS) parameter in a deblocking filter method according to a particular embodiment. X ,ref-i X Let {MV value and reference index for list "i" for block X (X=P or Q)} denote the motion vector MV value and reference index for list "i" for block X (X=P or Q). By convention, if block X is unidirectionally predicted using list 0, then {MV0 X is set to 0, ref-0 X is set to "-1"}. Separately, if block X is unidirectionally predicted using list 1, then {MV1 X is set to 0, ref-1 X is set to "-1"}.

[0035] The determination 300 of the boundary strength BS depends on several block parameters and is a result of a series of checks. For example, the checks are: (305): P or Q is intra (otherwise considered as inter). If the check results in no, then both P and Q blocks are inter predicted using the associated motion vector and reference index picture, and further checks are processed to determine BS. If P or Q is intra, the check results in yes and BS is set to 2-strong. - (310): P or Q has non-zero coefficients and the boundary is a transform boundary. - (320): P and Q have different reference indices. Therefore, according to a non-limiting example, P is bi-predicted ({MV0 P ,ref-0 P},{MV1 P ,ref-1 P}), Q is unidirectionally predicted {MV0 Q ,ref-0 Q}If so, then check whether: ref-0 P ≠ ref-0 Q And ref-1 P ≠ ref-0 Q (Check 1) (330): P and Q have different numbers of references (unidirectional or bi-predictive). In one variant, (330) does not exist and in unidirectional the MV value of the missing reference is inferred to be 0. (340): The motion vectors of P and Q with the same reference have different values. Therefore, according to a non-limiting example, P is bi-predicted ({MV0 P ,ref-0 P},{MV1 P ,ref-1 P}), Q is unidirectionally predicted {MV0 Q ,ref-0 Q}If so, then check whether: ref-0 P =ref-0 Q And |MV0 P -MV0 Q |>Is it a threshold? or ref-1 P =ref-0 Q And |MV1 P -MV0 Q |>Is it a threshold? (check 2) Those skilled in the art will unambiguously derive the check for the case of two bidirectionally predicted blocks or two unidirectionally predicted blocks from the above formula of (check 2).

[0036] In the case of blocks of luma samples, only block boundaries with BS equal to 1 or 2 are filtered. In the case of blocks of chroma samples, only block boundaries with BS equal to 2 are filtered.

[0037] 4 shows an exemplary flow chart of the filter strength determination in the method of deblocking filter according to a particular embodiment. Thus, independent of how BS is determined (300), an additional test (415) decides whether filtering is applied or not based on a set of conditions including a combination of the absolute difference of the weighted sample values ​​compared to a predefined threshold β: |p20-2p10+p00|+|p23-2p13+p03|+|q20-2q10+q00|+|q23-2q13+q03|>β (Equation 1)

[0038] If a filter is applied, a filtering process (430) modifies one or more of the P, Q samples on each side of the boundary in Figure 2 depending on the sample values ​​(410, accessing the sample values). The strength of the filtering process (normal or strong) is determined for each row (and column) of samples on each side of the boundary (420) based on a set of conditions that include a combination of the absolute differences of the weighted sample values ​​compared to predefined thresholds. Some thresholds are a function of a quantization parameter (QP) (440).

[0039] The number of samples modified by (430) on each side of the block boundary depends on the filter strength: the stronger the filter, the more pixels it affects on each side of the block boundary.

[0040] At least three new prediction modes are now disclosed, including weighted prediction with unequal or spatially varying weights within a block, in which the deblocking filter is adapted.

[0041] The first prediction mode to which the deblocking filter is adapted: Generalized Bidirectional Prediction (GBI) In the case of bi-prediction, two prediction blocks are calculated and combined with a weighted sum to obtain the final block prediction as follows: P GBI =(((1<<gs)-gw1).P0+gw1.P1)> >gs (Formula 2)

[0042] In the VVC reference software (VTM), the weight to use is coded for each CU using a "gbi index" whose values ​​are shown in Table 1.

[0043] [Table 1]

[0044] In the following, the weight pair {1 / 2;1 / 2} will be called the default weights.

[0045] Second prediction mode in which the deblocking filter is adapted: multiple hypotheses The general idea of ​​Multi Hypothesis MH is to combine an inter prediction P0 (which can be unidirectional or bidirectional) performed in a merge mode (where a list of merge candidates {reference index, motion value} is built and a merge index identifying one candidate is signaled to obtain motion information for motion compensated inter prediction) with another prediction P1, which is either an intra prediction (MH-inter-intra) or another inter prediction (MH-inter-inter: e.g. unidirectional prediction AMVP, skip and merge). The final prediction is a weighted average of the prediction generated by the merge indexed prediction and the other prediction mode (intra or inter), where different weights are applied depending on the combination.

[0046] FIG. 5 shows an example of multiple hypothesis prediction for the case of inter and intra mode combination MH-inter-intra. The intra prediction mode is signaled (it can be a subset (e.g. 4) of the classical prediction modes). The weight w_intra(y) gradually decreases the further the region is from the intra reference sample. According to a non-limiting example, the current block is divided into four equal area regions sharing the same weighting: (w0=w_intra i ,w1=w_inter iEach set of weights, denoted as w0=w_intra=1 / 2 and w1=w_inter=1 / 2, where i is 1 to 4, is applied to the corresponding region, as shown in the example of FIG. 5 for intra vertical prediction. When DC or planar mode is selected or the width or height of the CU is smaller than 4, equal weights are applied. In other words, when the intra prediction mode of prediction P1 is DC or planar mode, the relative weights used in the prediction of any sample of the current block are w0=w_intra=1 / 2 and w1=w_inter=1 / 2. Thus, there is only one region of equal weights (i=1). The coding mode defined in VVC, Combined Inter-Merge-Intra Prediction CIIP, is a specific instance of MH-Inter-Intra that combines the inter prediction P0 (which can be unidirectional or bidirectionally predicted) performed in merge mode with another prediction P1, which is a planar intra prediction with equal weight 1 / 2.

[0047] According to another example of multiple hypothesis prediction in case of inter and inter mode combination MH-inter-inter, the weights are the default ones (1 / 2;1 / 2). According to another variant of MH-inter-inter mode, one or two of the inter modes are bi-predictive {2 MV, 2 reference}, so that the construction of the sample prediction requires up to four motion compensation calculations.

[0048] The third prediction mode in which the deblocking filter is adapted: geometric partitioning Geometric (e.g. triangular) modes allow more flexibility to divide a picture into blocks before encoding. Figure 6 shows an example of a non-rectangular division (top) and an example of diagonal weighting of overlapped block motion compensation (bottom) associated with triangular division. The top two left examples of Figure 6 represent triangular division, while the top four right examples of Figure 6 represent a more general scheme of geometric division. Additional coding performance is obtained when blending overlap boundaries, for example along diagonal edges shown at the bottom of Figure 6. Thus, luma samples with a weighting factor of 4 located on the diagonal of a block use equal weights of 1 / 2 (W1=W2=4 / 8=1 / 2), while luma samples adjacent to the diagonal of a block with a weighting factor of 2 use weights (W1=2 / 8=1 / 4 and W2=6 / 8=3 / 4). In some variants, this weighting process is cascaded with other weightings, such as GBI or Local Illumination Compensation (LIC), increasing the implementation complexity.

[0049] In the case of triangulation, the samples can be divided into two groups: - Samples predicted using unidirectional motion compensation (P0 or P1) - Bi-directional predictive motion compensation, samples predicted using a mixture of unidirectional prediction P0 and unidirectional prediction P1.

[0050] As previously explained, at least theses three coding modes that use weighted prediction create problems in assessing boundary or filter strengths in the deblocking filter.

[0051] Some embodiments of a modified deblocking filter adapted to new prediction modes At least one embodiment of the present principles relates to modifying the deblocking filter DBF when blocks are coded using combinations of non-square or non-rectangular blocks (e.g., triangles) and / or combinations of unequal (e.g., GBi) and / or spatially variable weightings (e.g., MH). Advantageously, non-square or non-rectangular blocks, as shown for triangular decomposition, or more generally for geometric decomposition, are considered as a combination of two predictions with spatially variable weightings as shown on FIG. 7.

[0052] Advantageously, in at least one embodiment, the DBF process (filter strength derivation and filtering) is modified to preserve the intent and efficiency of the original DBF, thus increasing its efficiency compared to prior art methods.

[0053] FIG. 8 shows an example of a method for deblocking a portion of an image according to a general aspect of at least one embodiment. According to a preliminary step S110, information about two adjacent blocks is accessed. The two adjacent blocks will subsequently be called a first block P and a second block Q, which are interchangeable without limitation. The terms first and second may be used in this document to describe the two blocks, but these two blocks should not be limited by these terms. These terms are merely used to distinguish one element from another. The first block P and the second block Q belong to a portion of the image to be filtered. The blocks of samples are, according to a non-limiting example, one of a coding unit or a transform unit. In a first step S120, at least one boundary between the first block of samples P and the second block of samples Q is determined. In a particular embodiment, a flag EdgeFlag is set to 1 for samples at the boundary to be filtered using DBF, and the flag EdgeFlag is set to 0 for samples that are not to be filtered using DBF. The samples that should not be filtered using DBF include samples at the border of an image, or at the border of a slice, or samples (not boundary samples) inside a block, e.g., a coding unit or a transform unit. The border is either a horizontal border (horizontal edge) or a vertical border (vertical edge). In step S130, at least one parameter of the DBF is determined. As previously indicated, the at least one parameter of the DBF includes the border strength BS and the filter strength FS of the filter. The border strength BS is selected between the values ​​of weak (BS=0), normal (BS=1) or strong (BS=2). The filter strength FS refers to the result of a decision process affecting more or less pixels on each side of the block border as to whether to apply a filter. The decision process includes at least one condition on the values ​​of the samples of the first block P and the second block Q. In step S140, the border strength BS is determined according to at least one of the prediction mode of the first block and the prediction mode of the second block.Advantageously, the boundary strength BS is adapted for a prediction mode that is a weighted prediction mode combining a first prediction with a second prediction. According to a particular feature, if the prediction mode of the first block is a weighted prediction mode, the boundary strength BS further depends on the relative weights of the samples used in predicting the first block of samples according to the weighted prediction mode of the first block. According to a particular feature, the weighted prediction mode is one of the previously described coding modes, including Generalized Bidirectional Prediction GBi, also called Bidirectional Prediction with Coding Unit Level Weights BCW, a multiple hypothesis prediction combining intra and inter predicted samples (MH-Inter-Intra) or combining inter and inter predicted samples (MH-Inter-Inter), a combined inter merge-intra prediction combining inter merge and intra predicted samples, a geometric prediction combining inter predicted samples according to a geometric partitioning of the block, a triangular prediction combining inter predicted samples along the diagonal edges of the block. In the following, various embodiments for determining the BS are described. According to a particular embodiment, predicting the first block of samples according to a weighted prediction mode of the first block comprises predicting the samples of the first block as a weighted combination (w0, w1=1-w0) of a first predictor P0 determined using the first prediction mode and a second predictor P1 determined using the second prediction mode, the samples of the first block being considered as using the first prediction mode for determining the boundary strength if the weight w0 of the first predictor exceeds a level. According to another particular embodiment, the number of samples in the block with the weight w0 of the first predictor exceeds a level and determines the mode to be used for the first block at the BS. According to a particular variant, the level is set to 0. According to another particular variant, the boundary strength is determined for a group of samples of the block sharing the same relative weight. According to a non-limiting example, the group of samples includes one sample of the block, or 4x4 samples, and up to all samples.Next, in step S160, a DBF filtering process is applied to the first and second blocks along the boundary according to the determined boundary strength BS.

[0054] In previous approaches, the selection of DBF parameters for one rectangular or square block was done by selecting a set of row (or column) samples on each side of the vertical (or horizontal) boundary as shown in FIG. 3i ,p 2i ,p 1i ,p 0i ,q 0i ,q 1i ,q 2i ,q 3i}. However, for ease of implementation, it is generally preferred to manage / set DBF parameters per group of samples, e.g., per 4x4 sub-block.

[0055] Therefore, in the following, an embodiment is described in which the DBF parameters for the current block are set for each group of samples of a sub-block of size 4x4. However, the present principle is not limited to determining DBF parameters for 4x4 sub-blocks. Those skilled in the art will directly infer parameters for other sizes, or even for each single row (or column) from the following example. Then, the block is divided into a set of sub-blocks, and the same DBF parameters (e.g., BS) are derived for all samples of the sub-blocks, except for the DBF parameters (e.g., filtering strength) related to the sample values ​​in one row (or another, column).

[0056] FIG. 7 shows an example of a triangular logical division (left) and a blending map (right) according to a particular embodiment. For example, in the case of a non-rectangular division such as a triangle, it may happen that some samples of one subblock are predicted using unidirectional prediction, while others are predicted using bidirectional prediction (a blend of two unidirectional predictions). On the left side of FIG. 7, subblock sb2 indicates a subblock predicted using unidirectional prediction, while subblock sb3 indicates a subblock predicted using bidirectional prediction. In this case, the BS may be different for each sample, since steps 320, 330, 340 may give different results if the selection of the BS is performed individually for each sample.

[0057] In the following, we will associate two mixing maps with each sub-block, which contain the mixing weights w0(x) and w1(x) associated with the first predictor block P0 and the second predictor block P1, respectively, for each sample (x). The purpose of this map is to facilitate the description of the embodiment, and it may be virtual, i.e., it may not be calculated or stored in the memory system. In the case of traditional bi-prediction, the two maps are uniform and equal to 1 / 2. In the case of GBi, the two maps are uniform and, for example, equal to 3 / 8 and 5 / 8, respectively. In the case of MH, for example, w0(x)=w_intra(x) and w1(x)=w_inter(x).

[0058] Several embodiments are described for determining a boundary strength BS used in a general method for deblocking a portion of an image. The embodiments may be arranged together according to any combination of several embodiments.

[0059] First embodiment of the deblocking filter A first embodiment comprises a deblocking filter in which the boundary strength is adapted to the number of samples and the relative weights of the samples used in bi-prediction or weighted bi-prediction. This embodiment is advantageously well adapted when one sub-block on one side of the boundary uses a geometric partition or GBi.

[0060] For a given sub-block, process 320 of FIG. 3 is modified as follows: a) If at least one sample in a sub-block is bi-predicted, then the sub-block is considered as bi-predicted for the purposes of BS decision (e.g., sb2 and sb3 for the mixing map on the right side of FIG. 7). b) In one variant, if the relative number of samples using bidirectional prediction within a subblock is above a threshold, then the subblock is considered as bidirectionally predicted for the BS decision (e.g. sb3 with 14 bidirectionally predicted samples according to the mixing map on the right of FIG. 7 is considered as bidirectionally predicted, while sb2 with 3 bidirectionally predicted samples according to the mixing map on the right of FIG. 7 is considered as unidirectionally predicted). c) In one variant, the calculation of the number of bidirectionally predicted samples in a subblock excludes samples for which one mixing weight (w0(x) or w1(x)) is below a threshold or level (e.g., th=1 / 4). These samples are considered as unidirectionally predicted. d) In one variant, let n0 represent the relative number of samples that use reference 0 for prediction and n1 the relative number of samples that use reference 1 for prediction, and if n0 / (n0+n1) and n1 / (n0+n1) are above a threshold, then the subblock is considered as bidirectionally predicted for the BS decision.

[0061] By way of non-limiting value, in b), c) and d), the threshold or level is 0.

[0062] Advantageously, the thresholds are hard-coded, for example, inside a look-up table indexed by block size and partition type.

[0063] Second embodiment of the deblocking filter A second embodiment comprises a deblocking filter in which the boundary strength is adapted locally to the relative weights of the samples used in combining inter and intra predicted samples, or in combining inter and inter predicted samples, which is advantageously well adapted when one sub-block on one side of the boundary uses MH.

[0064] Considering a sub-block using MH-Inter-Intra: a) Areas of a block where w_intra(x) is above a threshold / level are considered as intra, for example in the case of the MH-Inter-Intra example of Figure 5, the first three upper areas are considered as intra. b) The area of ​​the block where w_inter(x) is above a threshold / level is considered as inter. If the inter prediction is bi-predictive (or uni-directional), the area is considered as bi-predictive (alternatively, uni-directional). For example, in the case of the MH-inter-intra example of FIG. 5, the last lower area (w_inter(x)=6 / 8) is considered as inter.

[0065] As before, note that in one variation, the threshold or level is zero.

[0066] Additionally, in one variation, the thresholds are hard-coded inside a look-up table indexed by the size of the block and the intra direction used.

[0067] Therefore, according to a), if the first prediction mode is 1 intra prediction mode, and if the weight of the first predictor (w_intra(x)) is above level, the sample of the first block is considered to use intra prediction mode to determine the boundary strength. In particular, this applies to any of the intra prediction modes, including directional intra prediction (e.g., vertical intra prediction as shown on FIG. 5), and non-directional modes such as DC or planar. In addition, in the case of planar or DC mode, equal weights are applied for the whole area of ​​the block, resulting in the same intra prediction mode for the whole area of ​​the block. According to the original DBF intent, if the first block P or the second block Q is intra, BS is set to strong (BS=2), and therefore, if the first prediction mode is 1 intra prediction mode, and if the weight of the first predictor (w_intra(x)) is above level, the boundary strength is set to strong. In a particular variant corresponding to CIIP (MH-Inter-Intra, where Intra is Planar), BS is set to strong for the block (BS=2).

[0068] In addition, according to b), if the first prediction mode is 1 inter prediction mode, and if the weight (w_inter(x)) of the first predictor is above level, the sample of the first block is considered to use the inter prediction mode to determine the boundary strength. In particular, this applies to any of the inter prediction modes, including unidirectional and bidirectional inter prediction. Thus, if the first prediction mode is an inter bidirectional prediction mode, then the sample of the first block is considered to use the inter bidirectional prediction mode to determine the boundary strength, and separately, if the first prediction mode is an inter unidirectional prediction mode, then the sample of the first block is considered to use the inter unidirectional prediction mode to determine the boundary strength. Then, different checks on the difference of the reference index and the motion vector are processed according to the bidirectional or unidirectional inter prediction mode.

[0069] Consider a sub-block with MH-Inter-Inter, where P0 is bi-predicted and P1 is uni-predicted: c) Areas of the block where w0(x) is above a threshold / level are considered as bi-directionally predicted, while areas of the block where w1(x) is above a threshold are considered as uni-directionally predicted. d) In one variant, if at least P0 or P1 are bi-predicted and {w0(x);w1(x)} is above a threshold, then check 1 and check 2 are modified as follows: For all pairs of values ​​(i;j): ref-i P ≠ref-j Q Whether it is (Check 1) ref-i P =ref-j Q For one pair of values ​​(i;j) where: |MVi P -MVj Q |>Is it a threshold? (check 2) (i;j) is ref-i P >=0 and ref-j Q >=0, etc.

[0070] As before, note that in one variation, the threshold or level is zero.

[0071] Advantageously, the thresholds can be hard-coded inside a look-up table indexed by block size and partition type.

[0072] Third embodiment of the deblocking filter A third embodiment comprises a deblocking filter in which the boundary strength is adapted to the weight of the samples used in the prediction. This embodiment is advantageously well adapted when one sub-block on one side of the boundary uses generalized bidirectional prediction GBi or BCW.

[0073] Considering the sub-blocks with non-default GBi weights (w0(x);w1(w)) associated with reference 0 and reference 1: a) If w0(x) or w1(x) is below a threshold, then the block is considered as unidirectionally predicted {MV1, ref-1} (or alternatively, MV0, ref-0) in steps (320, 330) and only {MV1, ref-1} (or alternatively, MV0, ref-0) is used in (340).

[0074] Fourth embodiment of the deblocking filter A fourth embodiment comprises a deblocking filter, in which motion vectors are normalized with the POC difference to determine boundary strength. If the first block P and the second block Q have different reference indices, the test (320) is inferred as "false" and for (340), the MV values ​​of Q (or P) are rescaled to the same POC as P (or alternatively, Q).

[0075] Fifth embodiment of the deblocking filter A fifth embodiment comprises a deblocking filter, wherein determining the boundary strength includes a test (an additional condition) as to whether the coding mode of one of the blocks at the boundary is at least one of MH, GBi, bidirectional prediction.

[0076] FIG. 9 shows a flow chart of the boundary strength determination in the method of the deblocking filter according to the fifth embodiment. The BS determination (300) is modified to include an additional condition (315) on the coding mode (XX) of the first block P or the second block Q. If the BS determination follows the scheme described in FIG. 3, this additional condition can be placed before / after / between the conditions (310, 320, 330, 340) as shown in one non-limiting example in FIG. 9. According to a non-limiting example, XX is MH, and / or GBi, and / or bi-prediction. According to another non-limiting example, determining the coding mode (XX) of the first block P or the second block Q conforms to any of the variants described for the first, second, or third embodiment. For example, the coding mode of the block is determined based on the number of samples and the relative weight of the samples used in bi-prediction, and the coding mode of the block is determined based on the relative weight of the samples used in MH. If the coding mode is determined to be intra, BS is set to 2.

[0077] Sixth embodiment of the deblocking filter A sixth embodiment comprises an encoder / decoder, in which the coding parameters are modified and stored during decoding / reconstruction for later use in the deblocking filter (in determining the boundary strength parameter). Figure 10 shows two flow charts of the storage of CU parameters used in determining the boundary strength (BS) parameter in the method of the deblocking filter according to a particular embodiment. The method 500 for deblocking the parts of the image on the left and right of Figure 10 is implemented in the in-loop filtering of the encoder or decoder. The method includes a step 330 of determining the BS and a step 400 of applying the filter DBF.

[0078] In one embodiment represented on the left side of Fig. 10, the determination of the BS parameters depends on a subset of CU (or partial CU) parameters (e.g. P, Q mode, reference index, MV value) and is modified according to the previously shown embodiment. In another embodiment represented on the right side of Fig. 10, the determination of the BS parameters 300 is not modified, but some of this subset of CU (or partial CU) parameters are modified (530) after their use to decode (510) and reconstruct (520) the current CU (separately, partial CU) and are stored to be available by other processes (e.g. DBF process). The modification of this subset of CU parameters is a function of other decoded CU parameters 515. This function therefore allows the value of the BS (output of 300) to depend on parameters other than the decoded subset of CU (or partial CU) parameters.

[0079] For example, if the coding mode is MH-inter-intra, and if w_intra(x) is above a threshold, then the coding mode is changed as intra before storage, so that BS is set to strong (BS=2) corresponding to intra prediction mode in 300. In this way, the same behavior is obtained as in variant a) of the second embodiment described for MH-inter-intra or CIIP.

[0080] In another example, if the relative number of samples using bidirectional prediction within a subblock exceeds a threshold, then the subblock coding mode is changed to bidirectionally predicted before storage, so that in step 300 the current CU is set as bidirectionally predicted, resulting in the same behavior as described in variant b) of the first embodiment.

[0081] In one variant, these modified CU (or partial CU) parameters stored for later use are used by other CUs (or partial CUs) for processes other than DBF, for example, the modified MV values ​​can be used later for temporal motion prediction (e.g., ATMVP).

[0082] Additional Embodiments and Information This application describes various aspects, including tools, features, embodiments, models, approaches, and the like. Many of these aspects are described specifically and in what may often be considered limiting manners, at least to illustrate the individual features. However, this is for purposes of clarity of description and does not limit the applicability or scope of the aspects. In fact, all of the different aspects may be combined and interchanged to produce further aspects. Additionally, aspects may also be combined and interchanged with aspects described in previous applications.

[0083] The aspects described and contemplated in this application may be implemented in many different forms. Figures 11, 12, and 13 below provide some embodiments, but other embodiments are contemplated and the descriptions in Figures 11, 12, and 13 do not limit the breadth of implementations. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects may be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the methods described above, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the methods described above.

[0084] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture", and "frame" may be used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0085] Various methods are described herein, each of which includes one or more steps or actions for achieving the method described above. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be varied or combined.

[0086] Various methods and other aspects described in the present application may be used to modify modules, such as the in-loop filters (165, 265), of the video encoder 100 and decoder 200 as shown in Figures 11 and 12. Moreover, the aspects are not limited to VVC or HEVC, but may be applied to other standards and recommendations, such as extensions of any such standards and recommendations (including VVC and HEVC), whether existing or developed in the future. Unless otherwise indicated or technically excluded, the aspects described in the present application may be used individually or in combination.

[0087] In this application, various numerical values ​​are used, for example, the size of the sub-block for which the DBF parameters are estimated is 4x4, and the specific values ​​are for illustrative purposes and the described aspects are not limited to these specific values.

[0088] 11 illustrates an encoder 100. Variations of this encoder 100 are contemplated, but in the following, for purposes of clarity, encoder 100 is described without describing all possible variations.

[0089] Before being encoded, a video sequence may undergo pre-encoding processing (101), such as applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata can be associated with the pre-processing and attached to the bitstream.

[0090] In the encoder 100, a picture is coded by the encoder elements as described below. The picture to be coded is divided (102) and processed, for example, in units of CUs. Each unit is coded, for example, using either intra mode or inter mode. When a unit is coded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) whether to use intra mode or inter mode to code the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (110) the prediction block from the original image block.

[0091] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is coded directly without applying a transform or quantization process.

[0092] The encoder decodes the coded blocks to provide references for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct an image block. To reduce coding artifacts, an in-loop filter (165) is applied to the reconstructed picture, for example to perform deblocking / SAO (sample adaptive offset) filtering. The filtered image is stored in a reference picture buffer (180).

[0093] Figure 12 shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass as described in Figure 11. The encoder 100 also generally performs video decoding as part of encoding the video data.

[0094] Specifically, the decoder input includes a video bitstream, which may be generated by video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. Picture partition information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partition information (235). The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct an image block. The prediction block may be obtained from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0095] The decoded picture may further undergo post-decoding processing (285), such as an inverse color conversion (e.g., from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping that performs the inverse of the remapping process performed in the pre-encoding processing (101). The post-decoding processing may use metadata derived in the pre-encoding processing and signaled in the bitstream.

[0096] FIG. 13 illustrates a block diagram of an example of a system in which various aspects and embodiments may be implemented. System 1000 may be embodied as a device including various components described below and configured to perform one or more of the aspects described in this document. Examples of such devices include various electronic devices, such as, but not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 1000 may be embodied, singly or in combination, in a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or individual components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more of the aspects described in this document.

[0097] The system 1000 includes at least one processor 1010, e.g., configured to execute instructions loaded therein for implementing various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuitry as known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash, magnetic disk drives, and / or optical disk drives. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device.

[0098] The system 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that may be included within a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding and decoding module. Additionally, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0099] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and then loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during performance of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing of equations, formulas, operations, and operational logic.

[0100] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and to provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory can be the memory 1020 and / or the storage device 1040, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the television's operating system. In at least one embodiment, a fast external dynamic volatile memory such as a RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, an emerging standard being developed by JVET, the Joint Video Experts Team).

[0101] Inputs to the elements of system 1000 may be provided through a variety of input devices, as indicated in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives RF signals transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples, not shown in FIG. 13, include composite video.

[0102] In various embodiments, the input devices of block 1130 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select a signal frequency band that may be referred to as a channel (for example) in a particular embodiment, (iv) demodulating the down-converted, band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band-limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner to perform various of these functions, including down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements perform frequency selection by receiving, filtering, downconverting, and filtering again to a desired frequency band an RF signal transmitted over a wired (e.g., cable) medium. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0103] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting the system 1000 to other electronic devices through USB and / or HDMI connections. It should be appreciated that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within the processor 1010, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, for example, in a separate interface IC or within the processor 1010, as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including the processor 1010 and an encoder / decoder 1030, which operate in combination with memory and storage elements to process the data stream as desired for presentation on an output device.

[0104] The various elements of system 1000 may be provided within a unitary housing in which the various elements may be interconnected and transmit data therebetween using suitable connection mechanisms, e.g., internal buses as are known in the art, including Inter-IC (I2C) buses, wiring, and printed circuit boards.

[0105] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented in the form of a wired and / or wireless medium, for example.

[0106] Data is streamed or otherwise provided to the system 1000 in various embodiments using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received through a communication channel 1060 and communication interface 1050 adapted for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 1000 using a set-top box that delivers data through an HDMI connection in the input block 1130. Still other embodiments provide streamed data to the system 1000 using an RF connection in the input block 1130. As indicated above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0107] The system 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 in various embodiments includes, for example, one or more of a touch screen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, a tablet, a laptop, a cell phone, or other device. The display 1100 can also be integrated with other components (e.g., as in the case of a smartphone) or be separate (e.g., an external monitor for a laptop). The other peripheral devices 1120, in various example embodiments, include one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 to provide functionality based on the output of the system 1000. For example, a disc player performs the function of playing the output of the system 1000 .

[0108] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using a signaling scheme such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow device-to-device control with or without user intervention. The output devices may be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices may be connected to the system 1000 using a communication channel 1060 via the communication interface 1050. The display 1100 and speaker 1110 may be integrated into a single unit with other components of the system 1000 within an electronic device such as, for example, a television. In various embodiments, the display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0109] For example, if the RF portion of input 1130 is part of a separate set-top box, display 1100 and speakers 1110 can alternatively be separate from one or more of the other components. In various embodiments in which display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0110] The embodiments may be implemented by computer software executed by the processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technology environment, and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 1010 may be of any type suitable for the technology environment, and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0111] Various implementations include decoding. "Decoding," as used herein, can encompass all or part of the processes performed, for example, on a received encoded sequence to generate a final output suitable for display. In various embodiments, such processes include processes typically performed by a decoder, such as one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of various implementations described herein, such as reconstructing a picture, determining parameters for a deblocking filter, and then filtering the reconstructed picture with the determined deblocking filter parameters.

[0112] As a further example, in one embodiment "decoding" refers only to entropy decoding, in another embodiment "decoding" refers only to differential decoding, and in another embodiment "decoding" refers to a combination of entropy decoding and differential decoding. Whether the expression "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0113] Various implementations include encoding. In a manner similar to the above description of "decoding," "encoding," as used herein, can encompass all or part of the processes performed, for example, on an input video sequence to generate an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, such as partitioning, differential encoding, transforming, quantizing, and entropy encoding. In various embodiments, such processes also or alternatively include the processes performed by an encoder in various implementations described herein, such as reconstructing an encoded picture, determining parameters for a deblocking filter, and then filtering the reconstructed picture with the determined deblocking filter parameters.

[0114] As a further example, in one embodiment, "encoding" refers to entropy encoding only, in another embodiment, "encoding" refers to differential encoding only, and in another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the expression "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0115] It should be noted that syntax elements as used herein are, for example, descriptive terms, and therefore, they do not preclude the use of other syntax element names.

[0116] It should be understood that when a diagram is presented as a flow diagram, it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0117] The implementations and aspects described herein may be embodied in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if only described in the context of a single implementation (e.g., only described as a method), the implementation of the described features may also be embodied in other forms (e.g., an apparatus or a program). An apparatus may be embodied in, for example, appropriate hardware, software, and firmware. A method may be embodied in, for example, a processor, which generally refers to a processing device, including, by way of example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0118] References to "one embodiment," or "an embodiment," or "one implementation," or "an implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment," or "in an embodiment," or "in one embodiment," or "in an implementation," as well as any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment.

[0119] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0120] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, transferring information, copying information, calculating information, determining information, predicting information, or estimating information.

[0121] In addition, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing the information or retrieving the information (e.g., from a memory). Furthermore, "receiving" is typically included in some way during an operation such as, for example, storing information, processing information, transmitting information, transferring information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0122] For example, in the cases of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). This can be extended for as many items as listed, as would be apparent to one of ordinary skill in this and related arts.

[0123] Also, as used herein, the word "signal" refers to, among other things, instructing something to a corresponding decoder. For example, in a particular embodiment, an encoder signals a particular one of a plurality of parameters for a deblocking filter in de-artifact filtering. Thus, in one embodiment, the same parameters are used at both the encoder and decoder sides. Thus, for example, the encoder can transmit a particular parameter to the decoder (explicit signaling), which allows the decoder to use the same particular parameter. Conversely, if the decoder already has the particular parameter and others, then signaling can be used without transmission, simply to allow the decoder to know and select the particular parameter (implicit signaling). By avoiding transmission of any actual function, bit savings are realized in various embodiments. It should be understood that signaling can be accomplished in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. Although the above relates to the verb form of the word "signal," the word "signal" can also be used as a noun in this specification.

[0124] As would be apparent to one skilled in the art, implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the implementations described above. For example, a signal can be formatted to carry a bit stream of the above-described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is well known. The signal can be stored on a processor-readable medium.

[0125] We have described a number of embodiments. The features of these embodiments may be provided alone or in any combination across various claim categories and types. In addition, the embodiments may include one or more of the following features, devices, or aspects, alone or in any combination across various claim categories and types: Modifying the deblocking filter process applied in the decoder and / or encoder, Allows some advanced deblocking filters in the decoder and / or encoder, Inserting syntax elements in the signaling that allow the decoder to identify the deblocking filter process to be used; Selecting a deblocking filter to apply at the decoder based on these syntax elements; Adapting deblocking filter parameters at the decoder and / or encoder; Adapting the boundary strength of the deblocking filter if the blocks at the boundary are a combination of non-square or non-rectangular blocks and / or combinations of unequal and / or spatially variable weightings; Adapting the filter strength of the deblocking filter if the blocks at the boundary are a combination of non-square or non-rectangular blocks and / or a combination of unequal and / or spatially variable weightings; Determine a mixture map for each sample in the block that represents the weighted combination used in predicting each sample; Adapting the boundary strength to the number of samples and / or to the weights of the samples used in bi-prediction; Adapting the boundary strength to the relative weights of the samples used when combining inter and intra predicted samples or when combining inter and inter predicted samples; Adapting the boundary strength to the weights of the samples used in the prediction; Normalize the motion vectors using the POC difference of the reference frame to determine boundary strength; determining the boundary strength comprises a test as to whether the prediction mode of one of the blocks at the boundary is at least one of MH, GBi or bi-prediction; Modifying coding parameters and storing the modified parameters for later use in the deblocking filter (e.g., in determining boundary strength parameters); A bitstream or signal comprising one or more of the syntax elements described above or variations thereof, A bitstream or signal including syntax conveying information generated according to any of the above described embodiments; Inserting syntax elements into the signaling that allow the decoder to adapt the deblocking filter in a manner corresponding to the one used by the encoder; Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the syntax elements described above or variations thereof; Creating and / or transmitting and / or receiving and / or decoding according to any of the embodiments described above; A method, process, apparatus, instruction storage medium, data storage medium, or signal according to any of the above-mentioned embodiments; A TV, set-top box, cell phone, tablet, or other electronic device performing deblocking filter parameter adaptation according to any of the embodiments described above; a TV, set-top box, cell phone, tablet, or other electronic device that performs the deblocking filter parameter adaptation according to any of the embodiments described above and displays the resulting image (e.g., using a monitor, screen, or other type of display); a TV, set-top box, cell phone, tablet, or other electronic device that selects (e.g., using a tuner) a channel for receiving a signal containing encoded images and performs adaptation of deblocking filter parameters according to any of the embodiments described above; A TV, set-top box, cell phone, tablet, or other electronic device that receives (e.g., using an antenna) a wireless signal containing an encoded image and performs deblocking filter parameter adaptation according to any of the embodiments described above.

Claims

1. 1. A method for deblocking a portion of an image, the method comprising: - determining (S120) at least one boundary between a first block of samples and a second block of samples, said first block and said second block belonging to said portion of the image; determining a boundary strength according to at least one of a prediction mode of the first block and a prediction mode of the second block (S140); filtering (S160) samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength; Including, If the prediction mode of the first block is a weighted prediction mode, the boundary strength further depends on relative weights of samples used in predicting the first block of samples according to the weighted prediction mode of the first block.

2. 16. An apparatus for deblocking a portion of an image, the apparatus comprising at least one processor, the at least one processor comprising: determining at least one boundary between a first block of samples and a second block of samples, said first block and said second block belonging to said portion of an image; determining a boundary strength according to at least one of a prediction mode of the first block and a prediction mode of the second block; filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength; The device is configured to: If the prediction mode of the first block is a weighted prediction mode, the boundary strength is further dependent on relative weights of samples used in predicting the first block of samples according to the weighted prediction mode of the first block.

3. 3. The method of claim 1 or the apparatus of claim 2, wherein the weighted prediction mode is one of generalized bidirectional prediction, bidirectional prediction with coding unit level weights, multiple hypothesis prediction combining intra and inter predicted samples or combining inter and inter predicted samples, combined inter merge intra prediction combining inter merge and intra predicted samples, geometric prediction combining inter predicted samples according to a geometric partitioning of the block, triangular prediction combining inter predicted samples along diagonal edges of the block.

4. 4. The method of claim 1 or 3 or the apparatus of claim 2 or 3, wherein predicting the first block of samples according to the weighted prediction mode of the first block comprises predicting the samples of the first block as a weighted combination of a first predictor determined using a first prediction mode and a second predictor determined using a second prediction mode, and the samples of the first block are considered as using the first prediction mode for determining the boundary strength if the weight of a first predictor is above a level.

5. The method of claim 4 or the device of claim 4, wherein the first prediction mode is an intra prediction mode and the samples of the first block are considered as using the intra prediction mode to determine the boundary strength.

6. The method of claim 5 or the apparatus of claim 5, wherein the first prediction mode is an intra-plane prediction mode and the samples of the first block are considered as using the intra-plane prediction mode to determine the boundary strength.

7. The method according to claim 5 or 6 or the apparatus according to claim 5 or 6, wherein the boundary strength is set to strong.

8. The method of claim 4 or the apparatus of claim 4, wherein the first prediction mode is an inter prediction mode and the samples of the first block are considered as using the inter prediction mode to determine the boundary strength.

9. 9. The method of claim 8, or the device of claim 8, wherein the first prediction mode is an inter bidirectional prediction mode and the samples of the first block are considered to use the inter bidirectional prediction mode to determine the boundary strength, or the first prediction mode is an inter unidirectional prediction mode and the samples of the first block are considered to use the inter unidirectional prediction mode to determine the boundary strength.

10. A method according to any one of claims 4 to 9 or an apparatus according to any one of claims 4 to 9, wherein said level is set to zero.

11. A method according to any one of claims 1, 3 to 10 or an apparatus according to any one of claims 2 to 10, wherein the boundary strength is determined for a group of samples of the block sharing the same relative weight, the group of samples including at least one sample and up to all samples of the block.

12. 1. A method for decoding an image, comprising: - decoding a portion of the image portion; filtering the decoded image portion according to a deblocking method according to claim 1 or any one of claims 3 to 11; A method comprising:

13. 1. A method for encoding an image, comprising the steps of: - reconstructing image portions; filtering said reconstructed image portion according to a deblocking method according to claim 1 or any one of claims 3 to 11; A method comprising:

14. 1. An apparatus for decoding an image, the apparatus comprising one or more processors, the one or more processors comprising: - decoding a portion of the image portion; filtering the decoded image portion according to a deblocking method according to claim 1 or any one of claims 3 to 11; 4. An apparatus configured to:

15. 1. An apparatus for encoding an image, the apparatus comprising one or more processors, the one or more processors comprising: - reconstructing image portions; filtering said reconstructed image portion according to a deblocking method according to claim 1 or any one of claims 3 to 11; 4. An apparatus configured to:

16. A computer program comprising software code instructions for performing the method according to any one of claims 1, 2 or 5 to 11, when said computer program is executed by one or several processors.

Images