Method and apparatus for deblocking image

Abstract

Various implementations are described, particularly implementations for video encoding and decoding, including methods and apparatus for deblocking images. According to an implementation, in a method for deblocking an image, at least one boundary between a first block of a sample and a second block of the sample is determined; determining a boundary strength according to at least one of a prediction mode of the first block and a prediction mode of the second block; and filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength. Advantageously, if the prediction mode of the first block is a weighted prediction mode, the boundary strength further depends on the relative weights of samples used in the first block that predicts samples according to the weighted prediction mode of the first block, and vice versa for the second block.

Description

[0001] This application is a divisional application of the Chinese invention patent application with the application date of December 10, 2019, application number 201980081023.0, and invention name “Method and device for deblocking an image”. Technical Field

[0002] At least one embodiment of the present invention generally relates to a method or apparatus for, for example, video encoding or decoding, and more particularly, to a method or apparatus for deblocking an image. Background Art

[0003] One or more implemented technical fields generally relate to video compression. At least some embodiments relate to improving compression efficiency compared to existing video compression systems such as HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265, and MPEG-H Part 2 described in "ITU-T H.265 Telecommunication Standardization Sector of ITU (10 / 2014), Series H: Audiovisual and Multimedia Systems, Infrastructure for Audiovisual Services - Coding for Mobile Video, High Efficiency Video Coding, Recommendation ITU-T H.265") or compared to video compression systems under development such as VVC (Versatile Video Coding, a new standard developed by JVET (Joint Video Experts Group)).

[0004] To achieve high compression efficiency, image and video coding schemes typically employ segmentation of the image, prediction including motion vector prediction, and transformations that exploit spatial and temporal redundancy in the video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlations, and then the difference between the original image and the predicted image (usually expressed as a prediction error or prediction residual) is transformed into frequency domain coefficients, which are quantized and entropy decoded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy decoding, inverse quantization, inverse transformation, and prediction.

[0005] In codecs such as HEVC or VVC, the deblocking filter DBF is applied after the picture is reconstructed and aims to reduce block artifacts by smoothing sample values ​​near block edges. The deblocking filter is defined using a traditional square or rectangular block partition shape, whose size is at least equal to 4 samples of luminance (Luma) samples. Then, a selection of motion vectors, reference indices, and reference deblocking filter samples that will be used to determine the strength of the deblocking filter is performed. For example, Norkin et al. describe the deblocking principle in "HEVC Deblocking Filter" (IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, December 2012).

[0006] In recent video codec approaches, new coding (prediction) modes are considered, where prediction samples are constructed using combinations of non-square or non-rectangular blocks (e.g., triangles) and / or combinations of unequal (e.g., generalized bidirectional prediction GBi, also known as bidirectional prediction with CU-level weights BCW) and / or spatially variable weighting (e.g., multiple hypotheses or combined inter-merging intra-frame prediction CIIP). Therefore, it is desirable to adapt the deblocking filtering process to such coding modes in order to still effectively reduce blocking artifacts. Summary of the invention

[0007] It is an object of the present invention to overcome at least one of the disadvantages of the prior art.

[0008] To this end, according to a general aspect of at least one embodiment, a method for deblocking a portion of an image is proposed. The method comprises: determining at least one boundary between a first block of samples and a second block of samples, the first block and the second block belonging to the portion of the image to be filtered; determining the boundary strength according to at least one of the prediction mode of the first block and the prediction mode of the second block; and filtering at least one boundary according to the boundary strength, i.e. filtering the samples of the first block and the second block adjacent to the at least one boundary. In the case where the prediction mode of the first block is a weighted prediction mode, the boundary strength further depends on the relative weights of the samples used in predicting the first block of samples according to the weighted prediction mode of the first block. Naturally, the same applies to the second block without distinction: in the case where the prediction mode of the second block is a weighted prediction mode, the boundary strength further depends on the relative weights of the samples used in predicting the second block of samples according to the weighted prediction mode of the second block. The weighted prediction mode is a decoding mode in which the prediction is obtained from a weighted combination of a first prediction value determined using the first prediction mode and a second prediction value determined using the second prediction mode.

[0009] According to another general aspect of at least one embodiment, a method for encoding a block in image coding is provided. The method comprises: reconstructing an image portion; and filtering the reconstructed image portion according to any one of the embodiments of the deblocking method.

[0010] According to another general aspect of at least one embodiment, a method for decoding a block of an image is proposed, comprising: decoding a portion of an image portion according to any one of the embodiments of the deblocking method, and filtering the decoded image portion.

[0011] According to another general aspect of at least one embodiment, a device for video encoding is provided, comprising means for implementing any one of the embodiments of the encoding method.

[0012] According to another general aspect of at least one embodiment, an apparatus for video decoding is proposed, comprising means for implementing any of the embodiments of the decoding method.

[0013] According to another general aspect of at least one embodiment, there is provided a device for video encoding, comprising one or more processors and at least one memory, wherein the one or more processors are configured to implement any one of the embodiments of the encoding method.

[0014] According to another general aspect of at least one embodiment, a device for video decoding is provided, comprising one or more processors and at least one memory. The one or more processors are configured to implement any one of the embodiments of the decoding method.

[0015] According to another general aspect of at least one embodiment, the weighted prediction mode is one of: generalized bidirectional prediction, bidirectional prediction using coding unit level weights, multiple hypothesis prediction combining intra and inter prediction samples or combining inter and inter prediction samples, combined inter merging intra prediction combining inter merging and intra prediction samples, geometric prediction combining inter prediction samples according to geometric partitioning of blocks, triangular prediction combining inter prediction samples along diagonal edges of blocks.

[0016] According to another general aspect of at least one embodiment, predicting a first block of samples according to a weighted prediction mode of the first block includes predicting the samples of the first block as a weighted combination of a first prediction value determined using the first prediction mode and a second prediction value determined using the second prediction mode, and in a case where the weight of the first prediction value is above a level, the samples of the first block are considered to be determined using the first prediction mode to determine boundary strength.

[0017] According to another general aspect of at least one embodiment, the first prediction mode is an intra prediction mode, and the samples of the first block are considered to use the intra prediction mode to determine the boundary strength. In a variant, the first prediction mode is an intra planar prediction mode, and the samples of the first block are considered to use the intra prediction mode to determine the boundary strength. In a variant, in the case of the intra prediction mode, the boundary strength is set to strong.

[0018] According to another general aspect of at least one embodiment, the first prediction mode is an inter prediction mode, and the samples of the first block are considered to use the inter prediction mode to determine the boundary strength. In one variation, the first prediction mode is an inter bi-directional prediction mode, and the samples of the first block are considered to use the inter bi-directional prediction mode to determine the boundary strength. In another variation, the first prediction mode is an inter uni-directional prediction mode, and the samples of the first block are considered to use the inter uni-directional prediction mode to determine the boundary strength.

[0019] According to another general aspect of at least one embodiment, the level in the determination of the BS is set to zero.

[0020] According to another general aspect of at least one embodiment, a boundary strength is determined for a set of samples of a block that share a same relative weight, the set of samples comprising at least one sample, 4x4 samples, up to all samples of the block.

[0021] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided, which contains data content generated according to any of the methods or apparatuses of the preceding descriptions.

[0022] According to another general aspect of at least one embodiment, there is provided a signal or bitstream comprising video data generated according to the method or apparatus of any one of the preceding descriptions.

[0023] One or more embodiments of the present invention also provide a computer-readable storage medium on which instructions for deblocking, encoding or decoding video data according to any of the above methods are stored. This embodiment also provides a computer-readable storage medium on which a bit stream generated according to the above method is stored. This embodiment also provides a method and device for transmitting a bit stream generated according to the above method. The embodiment of the present invention also provides a computer program product, which includes instructions for executing any of the described methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1a An exemplary flow chart of an in-loop post-filtering stage in a decoder architecture is shown.

[0025] Figure 1b Examples of decoded images are shown before (left) and after (right) a deblocking filter.

[0026] Figure 2 An example of a block boundary sample with blocking artifacts (left) between 2 blocks of samples (right) is shown.

[0027] Figure 3 An exemplary flow chart of determination of a boundary strength (BS) parameter in a deblocking filtering method according to certain embodiments is shown.

[0028] Figure 4 An exemplary flow chart of determination of filter strength (FS) in a deblocking filtering method according to certain embodiments is shown.

[0029] Figure 5 An example of multi-hypothesis prediction with inter and intra modes combined is shown.

[0030] Figure 6 An example of a non-rectangular partitioning (top) and an example of overlapping block motion compensated diagonal weighting associated with a triangular partitioning (bottom) are shown.

[0031] Figure 7 Examples of triangle logical partitioning (left) and blending mapping (right) are shown in accordance with certain embodiments.

[0032] Figure 8 An example of a method for deblocking a portion of an image according to a general aspect of at least one embodiment is shown.

[0033] Fig. 9 A flow chart for determining boundary strength in a deblocking filter method according to certain embodiments is shown.

[0034] Fig.10 Two flow charts are shown for storing CU parameters used in determining boundary strength (BS) parameters in a deblocking filter method according to certain embodiments.

[0035] Fig.11 A block diagram of an embodiment of a video encoder is shown in which various aspects of the embodiments may be implemented.

[0036] Fig.12 A block diagram of an embodiment of a video decoder is shown in which various aspects of the embodiments may be implemented.

[0037] Fig.13 A block diagram is shown of an example device in which various aspects of the embodiments may be implemented. DETAILED DESCRIPTION

[0038] It should be understood that the drawings and descriptions have been simplified to illustrate the elements relevant to a clear understanding of the present principles, while many other elements found in typical encoding and / or decoding devices have been removed for clarity. It should be understood that although the terms first and second may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0039] Various embodiments are described with respect to encoding / decoding of images. They may be applied to encoding / decoding a portion of an image, such as a slice or tile, a group of tiles, or an entire image sequence.

[0040] Various methods are described above, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0041] At least some embodiments relate to adapting a deblocking filter DBF when a block has been encoded using one of the following three prediction modes:

[0042] Generalized Bidirectional Prediction (GBi) or Bidirectional Prediction with CU-Level Weights (BCW),

[0043] Multiple Hypothesis (MH) or a simplified version called Combined Inter-Intra Prediction (CIIP),

[0044] Geometric or triangular patterns.

[0045] Advantageously, the present principles increase the efficiency of prior art approaches by maintaining the DBF intent and efficiency in block-based video codecs that support non-square or non-rectangular blocks and / or support combinations of blocks with variable weighting.

[0046] In the following, a general embodiment of a deblocking method is disclosed, and at least 3 prediction modes to which the deblocking filter is adapted are disclosed. Next, several embodiments of a modified deblocking filter are disclosed.

[0047] General embodiment of a deblocking method and exemplary prediction modes for adapting a deblocking filter

[0048] Conventional video decoding schemes include in-loop filtering processes for improving the quality of reconstructed images. In-loop filtering can include several types of filtering processes, such as deblocking filtering (DBF), sample adaptive offset (SAO) filtering in HEVC, and / or adaptive loop filtering ALF such as Wiener filter. These filters can be applied consecutively in this order or in a different order. Figure 1a An exemplary flow chart of an in-loop post-filtering stage in a decoder architecture is shown. Typically, a deblocking filter (DBF) is one of the decoding artifact reduction post-filters (20) and is applied after the block samples have been reconstructed (10). The post-filtered picture can be displayed and possibly stored in a decoded picture buffer (30) for use in establishing motion compensated predictions. Advantageously, the same in-loop post-filtering is also used in the encoder for reconstructed images used in temporal prediction.

[0049] Figure 1b An example of a decoded image before (left) and after (right) a deblocking filter is shown. The deblocking filter (10) is applied after the picture has been reconstructed. Its purpose is to reduce blocking artifacts by smoothing sample values ​​near block edges, such as Figure 1b shown.

[0050] Figure 2An example of a block boundary sample with blocking artifacts (left) between 2 blocks of samples P and Q (right) is shown. Samples P = {p0, p1, p2, p3} and Q = {q0, q1, q2, q3} belong to two adjacent blocks P and Q, the samples belonging to square 4X4 blocks P and Q, as shown in Figure 2 As shown on the right side of i ,p1 i ,p2 i ,p3 i The second index i in} is the row index in the 4X4 block. In general, the artifact visibility is proportional to the relative difference between the sample values ​​P and Q, such as Figure 2 This is shown on the left. This is why the DBF filter performs sample smoothing S on the block edges. The smoothing / filtering function S parameters are:

[0051] -Border Strength (BS) = {0-weak, 1-normal or 2-strong}

[0052] - Sample values ​​of blocks P, Q

[0053] For the vertical (respectively horizontal) border {p3 i ,p2 i ,p1 i ,p0 i ,q0 i ,q1 i ,q2 i ,q3 i For each set of row (or column) samples on each side of}, determination of the DBF parameters for the square block is performed.

[0054] Figure 3 An exemplary flow chart of determining 300 a boundary strength (BS) parameter in a deblocking filtering method according to a particular embodiment is shown. X ,ref-i X} is represented by the motion vector MV value of block X and the reference index of list "i" (X=P or Q). By convention, if block X is unidirectionally predicted using list-0, then {MV0 X is set to zero, and ref-0 X is set to "-1"}. Respectively, if block X is unidirectionally predicted using list-1, then {MV1 X is set to zero, and ref-1 X is set to "-1"}.

[0055] The determination 300 of the boundary strength BS depends on several block parameters and occurs in a continuous check. For example, the check is:

[0056] -(305): P or Q is intra (otherwise considered intra). In the case of a negative check, both the P block and the Q block are inter predicted using the associated motion vector and reference index picture, and further checked to determine BS. In the case of P or Q is intra,

[0057] If the check result is yes, set BS to 2-Strong.

[0058] -(310): P or Q has non-zero coefficients and the boundary is a transform boundary.

[0059] -(320): P and Q have different reference indices. Thus, according to a non-limiting example, if P is bi-directionally predicted ({MV0 P ,ref-0 P},{MV1 P ,ref-1 P}) and Q is unidirectionally predictive

[0060] {MV0 Q ,ref-0 Q}, then check whether the following holds:

[0061] ref-0 P ≠ref-0 Q and ref-1 P ≠ref-0 Q (Check 1)

[0062] -(330): P and Q have different reference numbers (unidirectional or bidirectional prediction). In a variant, (330)

[0063] does not exist, and in unidirectional prediction, the MV value of the missing reference is inferred to be zero.

[0064] -(340): P and Q motion vectors with the same reference have different values. Thus, according to a non-limiting example, if P is bi-directionally predicted ({MV0 P ,ref-0 P},{MV1 P ,ref-1 P}) and Q is a unidirectional prediction {MV0 Q ,ref-0 Q}, then check:

[0065] Is the following true: ref-0 P =ref-0 Q and |MV0 P -MV0 Q |>Threshold

[0066] Or is the following true: ref-1P =ref-0 Q and |MV1 P -MV0 Q |>Threshold(check 2)

[0067] A person skilled in the art will have no problem deriving the check for the case of 2 bidirectionally predicted blocks or 2 unidirectionally predicted blocks from the above equation (Check 2).

[0068] In case of a block of luma samples, filtering is performed only on block boundaries where BS is equal to 1 or 2. In case of a block of chroma samples, filtering is performed only on block boundaries where BS is equal to 2.

[0069] Figure 4 An exemplary flow chart of filter strength determination in a deblocking filtering method according to a particular embodiment is shown. Thus, independently of the manner of determining BS (300), an additional test (415) determines whether filtering should be applied based on a set of conditions involving a combination of the absolute difference of the weighted sample values ​​compared to a predetermined threshold β:

[0070] |p20-2p10+p00|+|p23-2p13+p03|+|q20-2q10+q00|+|q23-2q13+q03|>β

[0071] (Equation 1)

[0072] If a filter is applied, the filter process (430) modifies the sample value (410 accesses the sample value) according to the sample value. Figure 2 One or more of the P, Q samples on each side of the boundary. The strength of the filtering process (normal or strong) is determined (420) for each row (corresponding column) of samples on each side of the boundary based on a set of conditions involving a combination of absolute differences of weighted sample values ​​compared to predetermined thresholds. Some thresholds are functions of quantization parameters (QP) (440).

[0073] The number of samples modified by (430) on each side of the block boundary depends on the filter strength. A stronger filter strength affects more pixels on each side of the block boundary.

[0074] At least three new prediction modes are now disclosed, which involve weighted prediction with unequal or spatially variable weights in blocks, and Suitable for Deblocking filter.

[0075] The first prediction mode to which the deblocking filter is adapted: Generalized Bidirectional Prediction (GBI)

[0076] In case of bidirectional prediction, two prediction blocks are computed and combined with a weighted sum to obtain the final block prediction as follows:

[0077] P GBI=(((1<<gs)-gw1).P0+gw1.P1)> >gs(Equation 2)

[0078] In the VVC reference software (VTM), the weight to be used is coded per CU with a “gbi index”, whose values ​​are described in Table 1.

[0079] Table 1: Binarization of the GBi index and associated weights.

[0080] GBi Index <![CDATA[Weight value of w1]]> <![CDATA[gw1]]> gs(shift) Binarization of GBi index 0 -1 / 4 -1 2 0000 1 3 / 8 3 3 001 2 1 / 2 1 1 1 3 5 / 8 5 3 01 4 5 / 4 5 2 0001

[0081] Hereinafter, the weight pair {1 / 2; 1 / 2} is referred to as the default weight.

[0082] Second prediction mode to which the deblocking filter is adapted: Multi-hypothesis

[0083] The general concept of multi-hypothesis MH is to combine an inter prediction P0 (which can be unidirectional or bidirectional) performed in merge mode (where a merge candidate list {reference index, motion value} is constructed and the merge index identifying one candidate is signaled to obtain motion information for inter prediction for motion compensation) with another prediction P1 that is one of an intra prediction (MH-Inter-Intra) or another inter prediction (MH-Inter-Intra: e.g., unidirectional prediction AMVP, Skip and Merge). The final prediction is a weighted average of the prediction of the merge index and the prediction generated by another prediction mode (Intra or Inter), where different weights are applied depending on the combination.

[0084] Figure 5 An example of multi-hypothesis prediction in the case of inter and intra mode combination MH-inter-intra is shown. The intra prediction mode is signaled (which can be a subset of the classic prediction modes (e.g., 4)). The weight w_intra(y) gradually decreases as the region moves away from the intra reference sample. According to a non-limiting example, the current block is partitioned into 4 equal area regions sharing the same weighting. Each weighting set, denoted as (w0=w_intra i ,w1=w_inter i ), where i is 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6), are applied to the corresponding regions, such as Figure 5As described in the example of , for intra vertical direction prediction. When DC or planar mode is selected, or when the width or height of the CU is less than 4, equal weights are applied. In other words, when the intra prediction mode of prediction P1 is DC or planar mode, the relative weights used in the prediction of any sample of the current block are w0=w_intra=1 / 2 and w1=w_inter=1 / 2. Therefore, there is only one region with equal weights (i=1). The decoding mode combined inter-frame merge intra prediction CIIP defined in VVC is a specific example of MH-Inter-Intra, which combines the inter prediction P0 performed in merge mode (which can be unidirectional or bidirectional prediction) with another prediction P1 as a planar intra prediction with equal weight 1 / 2.

[0085] According to another example of multi-hypothesis prediction in the case of inter and inter mode combination MH-inter-intra, the weights are the default (1 / 2; 1 / 2). According to other variant examples of MH-inter-intra mode, one or both of the inter modes are bidirectional prediction {2 MVs, 2 references}, so that the construction of sample prediction requires the calculation of up to 4 motion compensations.

[0086] The third prediction mode for which the deblocking filter is adapted: geometric partitioning

[0087] Geometric (eg, triangle) modes allow more flexibility in partitioning a picture into blocks before decoding.

[0088] Figure 6 An example of a non-rectangular partitioning (top) and an example of overlapping block motion compensated diagonal weighting associated with a triangular partitioning (bottom) are shown. Figure 6 The 2 left examples at the top represent triangles segmentation ,and Figure 6 The top 4 right examples represent more general geometry segmentation If, for example, Figure 6 Additional decoding performance is obtained when the diagonal edge shown on the bottom of the block is mixed with overlapping boundaries. Therefore, the luma samples with weighting factor 4 located on the diagonal of the block use equal weight 1 / 2 (W1=W2=4 / 8=1 / 2), while the luma samples adjacent to the diagonal of the block with weighting factor 2 use weights (W1=2 / 8=1 / 4 and W2=6 / 8=3 / 4). In some variant examples, this weighting process is cascaded with other weightings such as GBI or local illumination compensation LIC, and increases the implementation complexity.

[0089] In the case of triangle segmentation, the samples can be divided into 2 groups:

[0090] - Samples predicted using unidirectional motion compensation (P0 or P1)

[0091] - Samples predicted with bi-directional prediction motion compensation (mixture of uni-directional prediction P0 and uni-directional prediction P1).

[0092] As explained previously, at least these three coding modes utilizing weighted prediction cause problems when evaluating boundary strength or filter strength in a deblocking filter.

[0093] Several embodiments of a modified deblocking filter adapted for new prediction modes

[0094] At least one embodiment of the present principles is directed to a modified deblocking filter DBF in the case where blocks are encoded using a combination of non-square or non-rectangular blocks (e.g., triangles) and / or a combination of unequal (e.g., GBi) and / or spatially variable weighting (e.g., MH). Advantageously, non-square or non-rectangular blocks as shown for triangle partitioning or more generally for geometric partitioning are considered to have Figure 7 Combination of 2 predictions with spatially variable weighting shown above.

[0095] Advantageously, in at least one embodiment, the DBF process (filter strength derivation and filtering) is modified to maintain the original DBF intent and efficiency, and thus increase its efficiency compared to prior art methods.

[0096] Figure 8An example of a method for deblocking a portion of an image according to a general aspect of at least one embodiment is shown. According to a preliminary step S110, information about two adjacent blocks is accessed. The two adjacent blocks are hereinafter referred to as a first block P and a second block Q, and they can be interchanged without restriction. The terms first and second may be used herein to describe two blocks, which should not be limited by these terms. These terms are only used to distinguish one element from another. The first block P and the second block Q belong to a portion of an image to be filtered. According to a non-limiting example, the sample block is one of a decoding unit or a transformation unit. In a first step S120, at least one boundary between the first sample block P and the second sample block Q is determined. In a specific embodiment, for samples at the boundary to be filtered with DBF, a flag EdgeFlag is set to 1, and for samples not to be filtered with DBF, a flag EdgeFlag is set to 0. Samples not to be filtered with DBF include samples at an image boundary or samples at a slice boundary or samples within a block (not boundary samples), such as a decoding unit or a transformation unit. The boundary is a horizontal boundary (horizontal edge) or a vertical boundary (vertical edge). In step S130, at least one parameter of the DBF is determined. As previously disclosed, the at least one parameter of the DBF comprises a boundary strength BS and a filter strength FS of the filter. The boundary strength BS is selected among weak (BS=0), normal (BS=1) or strong (BS=2) values. The filter strength FS refers to the result of a decision process as to whether to apply filtering and affect more or fewer pixels on each side of a block boundary. The decision process comprises at least one condition on the sample values ​​of the first block P and the second block Q. In step S140, the boundary strength BS is determined according to at least one of the prediction mode of the first block and the prediction mode of the second block. Advantageously, the boundary strength BS is suitable for a prediction mode as a weighted prediction mode combining the first prediction with the second prediction. According to a specific characteristic, in the case where the prediction mode of the first block is a weighted prediction mode, the boundary strength BS further depends on the relative weights of the samples used in the first block for predicting the samples according to the weighted prediction mode of the first block. According to specific characteristics, the weighted prediction mode is one of the previously described coding modes, which includes generalized bidirectional prediction GBi also referred to as bidirectional prediction with coding unit level weights BCW, multiple hypothesis prediction combining intra and inter prediction samples MH-inter-intra or combining inter and inter prediction samples MH-inter-inter, combined inter merging intra prediction combining inter merging and intra prediction samples, geometric prediction combining inter prediction samples according to geometric partitioning of blocks, triangular prediction combining inter prediction samples along diagonal edges of blocks. Various embodiments for determining BS will be explained below.According to a specific embodiment, predicting the first block of samples according to the weighted prediction mode of the first block includes predicting the samples of the first block as a weighted combination (w0, w1=1-w0) of a first prediction value P0 determined using the first prediction mode and a second prediction value P1 determined using the second prediction mode, and wherein in the case where the weight w0 of the first prediction value is above a level, the samples of the first block are considered to use the first prediction mode to determine the boundary strength. According to other specific embodiments, the number of samples in the block using the weight w0 of the first prediction value is above the level of the mode determined for the first block in the BS. According to a specific variant, the level is set to zero. According to another specific variant, the boundary strength is determined for a group of samples of blocks sharing the same relative weight. According to a non-limiting example, the sample group includes one sample or 4×4 samples and up to all samples of the block. Then, in step S160, according to the determined boundary strength BS, DBF filtering is applied to the first and second blocks along the boundary.

[0097] In previous methods, such as Figure 2 As shown, for the vertical (corresponding horizontal) boundary {p 3i ,p 2i ,p 1i ,p 0i ,q 0i ,q 1i ,q 2i ,q 3i}, the selection of DBF parameters for a rectangular or square block is performed for each row (or column) of samples on each side of}. However, for ease of implementation, it is usually preferred to manage / set the DBF parameters for each group of samples (e.g., 4x4 sub-block).

[0098] Therefore, in the following, embodiments are described in the case where the DBF parameters of the current block are set for each group of samples of a 4×4 sub-block. However, the present principles are not limited to determining the DBF parameters of a 4×4 sub-block. It will be straightforward for a person skilled in the art to derive parameters for other sizes or even for each single row (or single column) from the following examples. The block is then partitioned into a set of sub-blocks, and the same DBF parameters (e.g., BS) are derived for all samples of the sub-blocks, except for the DBF parameters (e.g., filter strength) associated with the sample values ​​in a row (corresponding column).

[0099] Figure 7 An example of a triangular logical partition (left) and a hybrid mapping (right) according to a particular embodiment is shown. In case of a non-rectangular partition (e.g. a triangle), it may happen that some samples of a sub-block are predicted with unidirectional prediction, while other samples are predicted with bidirectional prediction (a mix of two unidirectional predictions). Figure 7On the left side of , subblock sb2 shows a subblock using unidirectional prediction, while subblock sb3 shows a subblock using bidirectional prediction. In this case, if BS is selected for each sample separately, the BS of each sample may be different, because steps 320, 330, 340 may give different results.

[0100] In the following, we will associate two hybrid maps with each sub-block, which contain, for each sample (x), hybrid weights w0(x) and w1(x) associated with the first prediction value block P0 and the second prediction value block P1, respectively. The purpose of this mapping is to facilitate the description of the embodiment, and therefore it may be virtual, meaning that it may not be calculated or stored in the memory system. In the case of traditional bidirectional prediction, the 2 mappings are consistent and equal to 1 / 2. In the case of GBi, the 2 mappings are consistent and equal to, for example, 3 / 8 and 5 / 8, respectively. In the case of MH, for example, w0(x) = w_intra(x) and w1(x) = w_inter(x).

[0101] Several embodiments for determining a boundary strength BS for use in a general method of deblocking a portion of an image are described, which embodiments may be arranged together according to any combination of the several embodiments.

[0102] First embodiment of the deblocking filter

[0103] The first embodiment comprises a deblocking filter in which the boundary strength is adapted to the number of samples and the relative weights of the samples used in bi-directional prediction or weighted bi-directional prediction. This embodiment is advantageously well suited in the case where one sub-block on one side of the boundary uses geometric partitioning or GBi.

[0104] For a given sub-block, Figure 3 The procedure 320 is modified as follows:

[0105] a) If at least one sample in a sub-block is bi-predicted, the sub-block is considered to be bi-predicted for the BS's determination (e.g., based on Figure 7 The mixed mapping on the right side of sb2 and sb3).

[0106] b) In a variant, if the relative number of samples in a subblock that use bidirectional prediction is greater than a threshold, then the subblock is considered to be bidirectionally predicted for the purpose of determining the BS (e.g., based on Figure 7 The hybrid map on the right has sb3 with 14 bi-predicted samples, which is considered bi-predicted, while according to Figure 7 The hybrid map on the right, sb2 with 3 bi-directionally predicted samples, is treated as uni-directionally predicted).

[0107] c) In a variant, the calculation of the number of bidirectionally predicted samples in a subblock excludes samples for which one blending weight (w0(x) or w1(x)) is below a threshold or level (e.g., th=1 / 4). These samples are considered to be unidirectionally predicted.

[0108] d) In a variation, let n0 be the relative number of samples predicted using reference 0, and let n1 be the relative number of samples predicted using reference 1, if n0 / (n0+n1) and n1 / (n0+

[0109] n1) is above the threshold, the sub-block is considered bi-predicted for determining BS.

[0110] According to non-limiting values, in b), c) and d), the threshold or level is zero.

[0111] Advantageously, for example, the threshold is hard-coded in a look-up table indexed by the size of the block and the segmentation type.

[0112] Second embodiment of the deblocking filter

[0113] The second embodiment comprises a deblocking filter in which the boundary strength is locally adapted to the relative weights of the samples used when combining inter- and intra-predicted samples or when combining inter- and intra-predicted samples. This embodiment is advantageously well suited in the case where one sub-block on one side of the boundary uses MH.

[0114] Consider using MH-Inter-Intra sub-blocks:

[0115] a) The area of ​​the block where w_intra(x) is above a threshold / level is considered intra-frame. Figure 5 In the case of the MH-Inter-Intra example, the first 3 top regions are considered as Intra.

[0116] b) A region of blocks where w_inter(x) is above a threshold / level is considered inter. If inter prediction is bidirectional (or unidirectional), then the region is considered bidirectional (respectively, unidirectional). For example, in Figure 5 In the case of the MH-Inter-Intra example, the last bottom region (w_inter(x)=6 / 8) is considered to be Inter.

[0117] Note that, as previously described, in one variation, the threshold or level is zero.

[0118] Furthermore, in a variation, the threshold is hard-coded in a lookup table indexed by the size of the block and the intra-direction used.

[0119] Thus, according to a), in the case where the first prediction mode is an intra prediction mode and in the case where the weight of the first prediction value (w_intra(x)) is above level, the samples of the first block are considered to use the intra prediction mode to determine the boundary strength. In particular, this applies to any of the intra prediction modes, including directional intra prediction (e.g., as Figure 5 Vertical intra prediction as shown) and non-directional modes such as DC or planar. In addition, in the case of planar mode or DC mode, equal weights are applied to the entire area of ​​the block, resulting in the same intra prediction mode for the entire area of ​​the block. According to the original DBF intention, when the first block P or the second block Q is intra, BS is set to strong (BS=2), so when the first prediction mode is an intra prediction mode and when the weight of the first prediction value (w_intra(x)) is higher than horizontal, the boundary strength is set to strong. In a specific variant corresponding to CIIP (MH-Inter-Intra, where Intra is planar), BS is set to strong (BS=2) for the block.

[0120] Furthermore, according to b), in the case where the first prediction mode is an inter prediction mode, and in the case where the weight of the first prediction value (w_inter(x)) is above level, the samples of the first block are considered to use the inter prediction mode to determine the boundary strength. In particular, this applies to any inter prediction mode including unidirectional and bidirectional inter prediction. Therefore, in the case where the first prediction mode is an inter bidirectional prediction mode, the samples of the first block are considered to use the inter bidirectional prediction mode to determine the boundary strength, and correspondingly in the case where the first prediction mode is an inter unidirectional prediction mode, the samples of the first block are considered to use the inter unidirectional prediction mode to determine the boundary strength. Then, depending on the bidirectional or unidirectional inter prediction mode, different variable checks on the reference index and the motion vector difference are processed.

[0121] Consider using MH-inter-inter subblocks, P0 bidirectional prediction, and P1 unidirectional prediction:

[0122] c) Regions of the block where w0(x) is above a threshold / level are considered bidirectionally predicted, while regions of the block where w1(x) is above the threshold are considered unidirectionally predicted.

[0123] d) In a variant, if at least P0 or P1 is bi-predicted and both {w0(x); w1(x)} are above a threshold, then checks 1 and 2 are changed to:

[0124] Does the following hold: For all pairs of values ​​(i; j): ef-i P ≠ref-j Q (Check 1)

[0125] Does the following hold true: For a P =ref-j Q A pair of values ​​(i; j): |MVi P -MVj Q

[0126] |>Threshold(check 2)

[0127] For (i; j), for example, ref-i P >=0 and ref-j Q >=0

[0128] Note that, as previously mentioned, in one variation, the threshold or level is zero.

[0129] Advantageously, the threshold value may be hard-coded in a lookup table indexed by the size of the block and the segmentation type.

[0130] Third embodiment of the deblocking filter

[0131] A third embodiment comprises a deblocking filter in which the boundary strength is adapted to the weights of the samples used in the prediction. This embodiment is advantageously well adapted in the case where one subblock on one side of the boundary uses generalized bidirectional prediction GBi or BCW.

[0132] Consider that GBi weights (w0(x); w1(w)) associated with reference-0 and reference-1 are not the default sub-blocks:

[0133] a) If w0(x) or w1(x) is below a threshold, the block is considered as unidirectionally predicted {MV1, ref-1} (or MV0, ref-0, respectively) in steps (320, 330) and only {MV1, ref-1} (or MV0, ref-0) is used in (340).

[0134] Fourth embodiment of the deblocking filter

[0135] A fourth embodiment includes a deblocking filter in which the motion vectors are normalized with a POC difference to determine the boundary strength. In the case where the first block P and the second block Q have different reference indices, the test (320) is inferred to be "false" and for (340), the MV value of Q (or P) is rescaled to the same POC as P (or Q, respectively).

[0136] Fifth embodiment of the deblocking filter

[0137] The fifth embodiment includes a deblocking filter in which determining the boundary strength includes a test (additional condition) as to whether a mode of one of the blocks at the boundary is at least one of MH, GBi, bi-directional prediction.

[0138] Fig. 9 FIG. 5 is a flowchart of determining the boundary strength in the deblocking filtering method according to the fifth embodiment. The determination (300) of the BS is modified to include an additional condition (315) regarding the decoding mode (XX) of the first block P or the second block Q. If the determination of the BS follows Figure 3 , then the additional condition can be placed as follows Fig. 9 Before / after / between one of the conditions (310, 320, 330, 340) depicted in one of the non-limiting examples in . According to the non-limiting example, XX is MH and / or GBi and / or bidirectional prediction. According to other non-limiting examples, determining the decoding mode (XX) of the first block P or the second block Q is compatible with any of the variations described for the first, second or third embodiments. For example, the decoding mode of the block is determined based on the number and relative weights of the samples used in the bidirectional prediction, and the decoding mode of the block is determined based on the relative weights of the samples used in the MH. If the decoding mode is determined to be intra-frame, BS is set to 2.

[0139] Sixth embodiment of the deblocking filter

[0140] A sixth embodiment comprises an encoder / decoder in which the coding parameters are modified and stored upon decoding / reconstruction for subsequent use in the deblocking filter (when determining the boundary strength parameter). Fig.10 Two flow charts are shown for storing CU parameters used in determining a boundary strength (BS) parameter in a deblocking filtering method according to a specific embodiment. Fig.10 The method 500 for deblocking a portion of an image on the left and right sides of the image is implemented in an in-loop filter of an encoder or decoder. The method comprises a step 330 of determining BS and a step 400 of applying a filter DBF.

[0141] exist Fig.10 In the embodiment represented on the left side of , the determination of BS parameters depends on a subset of CU (or sub-CU) parameters (e.g., P, Q mode, reference index, MV value) and is modified according to the previously disclosed embodiments. Fig.10 In another embodiment represented on the right side of , the determination 300 of the BS parameters remains unchanged, but some of this subset of CU (or sub-CU) parameters are modified 530 after being used to decode 510 and reconstruct 520 the current CU (corresponding sub-CU) and stored so as to be available by other processes (e.g. DBF process). The modification of this subset of CU parameters is a function of other decoded CU parameters 515. Thus, this function allows making the value of BS (output of 300) depend on other parameters in addition to the decoded subset of CU (or sub-CU) parameters.

[0142] For example, if the decoding mode is MH-Inter-Intra and if w_intra(x) is above a threshold, the decoding mode is modified to intra before storage, so that BS is set to strong (BS=2) corresponding to intra prediction mode in 300. In this way, the same behavior as described in variant a) of the second embodiment with respect to MH-Inter-Intra or CIIP is obtained.

[0143] In another example, if the relative number of samples in the sub-block using bidirectional prediction is greater than a threshold, the decoding mode of the sub-block is modified to perform bidirectional prediction before storage, so that in step 300, the current CU is set to bidirectional prediction, and the same behavior as described in variant b) of the first embodiment is obtained.

[0144] In a variant, these modified CU (or sub-CU) parameters stored for later use are used by other CUs (or sub-CUs) for another process outside of DBF. For example, the modified MV value can be used later for temporal motion prediction (eg, ATMVP).

[0145] Additional Examples and Information

[0146] The present application describes a number of aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described as having specificity, and at least in order to illustrate individual characteristics, are usually described in a manner that may sound limited. However, this is for the purpose of describing clearly, and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide additional aspects. In addition, these aspects can also be combined and interchanged with the aspects described in earlier documents.

[0147] The aspects described and contemplated in this application can be implemented in many different forms. Fig.11 , 12 13 provide some embodiments, but other embodiments are contemplated and are Fig.11 , 12 The discussion of 13 does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0148] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0149] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0150] Various methods and other aspects described in this application can be used to modify modules, such as Fig.11 and Fig.12 In-loop filters (165, 265) of the video encoder 100 and decoder 200 are shown. In addition, aspects of the present invention are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, whether pre-existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0151] Various values ​​are used in this application, for example, the size of a sub-block 4x4 for estimating DBF parameters. The specific values ​​are for example purposes, and the described aspects are not limited to these specific values.

[0152] Fig.11 An encoder 100 is shown. Variations of the encoder 100 are contemplated, but for the sake of clarity, the encoder 100 is described below without describing all contemplated variations.

[0153] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0154] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is divided (102) and processed in units such as CUs. Each unit is encoded using, for example, intra or inter mode. When the unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the unit, and indicates the intra / inter decision by, for example, a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0155] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass the transform and quantization, i.e., encode the residual directly without applying a transform or quantization process.

[0156] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0157] Fig.12 2 shows a block diagram of a video decoder 200. In the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described below. Fig.11 The encoding pass described in is a decoding pass that is the inverse of the encoding pass described in . The encoder 100 also typically performs video decoding as part of encoding the video data.

[0158] In particular, the input to the decoder includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other decoding information. Picture segmentation information indicates how the picture is segmented. The decoder can therefore divide (235) the picture according to the decoded picture segmentation information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined (255) with the prediction block to reconstruct the image block. The prediction block can be obtained (270) according to intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0159] The decoded picture may further undergo post-decoding processing (285), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or performing an inverse remapping of the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0160] Fig.13 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 1000 can be implemented as a device including various components described below, and is configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1000 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed on multiple ICs and / or discrete components. In various embodiments, system 1000 is coupled to one or more other systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 1000 is configured to implement one or more aspects described herein.

[0161] The system 1000 includes at least one processor 1010, which is configured to execute instructions loaded therein, for implementing various aspects described herein, for example. The processor 1010 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device (including a removable and non-removable storage device), and / or a network-accessible storage device.

[0162] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module (one or more) that may be included in the device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated into the processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0163] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of the various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0164] In some embodiments, memory within the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store, for example, an operating system for a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as working memory for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by JVET (Joint Video Team Experts)).

[0165] As shown in block 1130, input to the elements of system 1000 may be provided through various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives an RF signal transmitted over the air, for example, by a broadcaster, (ii) a component (COMP) input terminal (or a collection of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Fig.13 Other examples not shown include composite video.

[0166] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF portion may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF portion of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its relevant input processing element receive the RF signal that sends by wired (for example, cable) medium, and filter to the frequency band of expectation again by filtering, down-conversion and perform frequency selection.Various embodiments rearrange the order of above-mentioned (and other) element, remove some in these elements, and / or add other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example insert amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0167] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, can be implemented as needed, for example, in a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed in a separate interface IC or in the processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 1010 and the encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data stream as needed for presentation on an output device.

[0168] The various components of system 1000 may be disposed within an integrated housing. Within the integrated housing, the various components may be interconnected and data may be transmitted therebetween using a suitable connection arrangement, such as an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and a printed circuit board.

[0169] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0170] In various embodiments, a wireless network, such as a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), is used to stream data or otherwise provide it to the system 1000. The Wi-Fi signals of these embodiments are received by a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is usually connected to an access point or a router, which provides access to an external network including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box that transmits data through an HDMI connection of an input block 1130 to provide streaming data to the system 1000. Still other embodiments use an RF connection of an input block 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use a wireless network other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0171] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 1100 can also be integrated with other components (e.g., as in a smart phone), or separately (e.g., an external monitor for a laptop computer). In each example of each embodiment, other peripheral devices 1120 include one or more of an independent digital video disk (or digital versatile disk) (DVR, two items), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functions based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0172] In various embodiments, control signals are transmitted between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling such as AV link, consumer electronics control (CEC), or other communication protocols that implement device-to-device control with or without user intervention. Output devices can be communicatively coupled to the system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to the system 1000 via communication interface 1050 using communication channel 1060. The display 1100 and speaker 1110 can be integrated into a single unit in an electronic device (e.g., a television) along with other components of the system 1000. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0173] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more of the other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0174] These embodiments may be implemented by computer software implemented by the processor 1010, or hardware, or a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, as a non-limiting example, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0175] Various implementations involve decoding. As used in this application, "decoding" may include, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by decoders of various implementations described in this application, such as reconstructing a picture, determining parameters of a deblocking filter, and then filtering the reconstructed picture using the determined deblocking filter parameters.

[0176] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or generally to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0177] Various implementations involve encoding. In a manner similar to the discussion above regarding "decoding", "encoding" as used in this application may include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also or alternatively include processes performed by encoders of various implementations described in this application, such as reconstructing an encoded picture, determining parameters of a deblocking filter, and then filtering the reconstructed picture using the determined deblocking filter parameters.

[0178] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or generally to a broader encoding process will become clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0179] Note that the grammatical elements used here are descriptive terms such as "grammatical elements". Therefore, they do not exclude the use of other grammatical element names.

[0180] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0181] The implementations and aspects described herein may be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed may also be implemented in other forms (e.g., a device or program). For example, a device may be implemented with appropriate hardware, software, and firmware. The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0182] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations in various places in this application do not necessarily all refer to the same embodiment.

[0183] Additionally, the present application may refer to “determining” various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0184] Furthermore, the present application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0185] Additionally, the present application may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is often involved in one way or another.

[0186] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to multiple items listed, as will be apparent to one of ordinary skill in this and related arts.

[0187] In addition, as used herein, the word "signal" refers in particular to indicating something to a corresponding decoder. For example, in some embodiments, the encoder signals a specific one of a plurality of parameters of a deblocking filter used in de-artifact filtering. In this way, in an embodiment, the same parameters are used on the encoder side and the decoder side. Thus, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters as well as other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various embodiments by avoiding the transmission of any actual function. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the foregoing relates to the verb form of the word "signal / signaling", the word "signal / signaling" can also be used as a noun in this article.

[0188] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0189] We have described a number of embodiments. Features of these embodiments may be provided individually or in any combination across various claim categories and types. In addition, embodiments may include one or more of the following features, devices, or aspects, alone or in any combination across various claim categories and types:

[0190] Modify the deblocking filter process applied in the decoder and / or encoder,

[0191] Enable several improved deblocking filters in the decoder and / or encoder,

[0192] Insertion in the signaling enables the decoder to identify the deblocking filter process

[0193] The syntax elements used,

[0194] based on these syntax elements, selecting the deblocking filter to be applied at the decoder, adapting deblocking filter parameters in the decoder and / or encoder,

[0195] adapting the boundary strength of the deblocking filter in case the blocks at the border are a combination of non-square or non-rectangular blocks and / or a combination of unequal and / or spatially variable weightings, adapting the filter strength of the deblocking filter in case the blocks at the border are a combination of non-square or non-rectangular blocks and / or a combination of unequal and / or spatially variable weightings, determining a hybrid map for each sample of the block representing the weighted combination used in the prediction of each sample,

[0196] adapting the boundary strength to the number of samples and / or sample weights used in bidirectional prediction, adapting the boundary strength to the relative weights of samples used in combined inter and intra predicted samples or in combined intra and intra predicted samples,

[0197] Adapt the boundary strength to the weights of the samples used in the prediction,

[0198] normalizing the motion vector using the POC difference of the reference frame to determine the boundary strength, determining the boundary strength comprises testing whether the prediction mode of one of the blocks at the boundary is at least one of MH, GBi or bi-prediction,

[0199] modifying decoding parameters and storing the modified parameters for subsequent use in a deblocking filter (e.g. in determining a boundary strength parameter),

[0200] a bitstream or signal comprising one or more of the described syntax elements or variants thereof, a bitstream or signal comprising syntax conveying information generated according to any of the described embodiments,

[0201] inserting in the signaling syntax elements enabling the decoder to adapt the deblocking filter in a manner corresponding to that used by the encoder,

[0202] creating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof, ...

[0203] quality, medium or signal for storing data,

[0204] a TV, set-top box, cell phone, tablet or other electronic device that performs the adaptation of deblocking filter parameters according to any of the described embodiments, a TV, set-top box, cell phone, tablet or other electronic device that performs the adaptation of deblocking filter parameters according to any of the described embodiments and displays (e.g. using a monitor, screen or other type of display) the resulting image,

[0205] · A TV, set-top box, cellular phone, tablet or other electronic device, which selects (e.g., using a tuner) a channel to receive a signal comprising an encoded image and performs adaptation of deblocking filter parameters according to any of the described embodiments, · A TV, set-top box, cellular phone, tablet or other electronic device, which receives (e.g., using an antenna) a signal comprising an encoded image over the air and performs adaptation of deblocking filter parameters according to any of the described embodiments.

Claims

1. A method for deblocking a portion of an image, the method comprising: determining at least one boundary between a first block of samples and a second block of samples, the first block and the second block belonging to the portion of the image; determining a boundary strength according to at least one of a prediction mode of the first block and a prediction mode of the second block; filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength; Wherein determining the boundary strength further comprises: the prediction mode for the first block of samples is a geometric prediction mode, performing a weighted combination on a first prediction value determined by a first inter-frame unidirectional prediction and a second prediction value determined by a second inter-frame unidirectional prediction, and using one of an inter-frame bidirectional prediction mode or an inter-frame unidirectional prediction mode as the prediction mode for the samples of the first block according to the weight of the corresponding sample in the first prediction value or according to the weight of the corresponding sample in the second prediction value.

2. The method according to claim 1, wherein when the weight of the corresponding sample in the first prediction value is higher than a level or the weight of the corresponding sample in the second prediction value is higher than a level, the sample of the first block is considered to use the inter-frame bidirectional prediction mode to determine the boundary strength.

3. A device for deblocking a portion of an image, the device comprising at least one processor, the at least one processor being configured to: determining at least one boundary between a first block of samples and a second block of samples, the first block and the second block belonging to the portion of the image; determining a boundary strength according to at least one of a prediction mode of the first block and a prediction mode of the second block; filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength; Wherein determining the boundary strength further comprises: the prediction mode for the first block of samples is a geometric prediction mode, performing a weighted combination on a first prediction value determined by a first inter-frame unidirectional prediction and a second prediction value determined by a second inter-frame unidirectional prediction, wherein one of an inter-frame bidirectional prediction mode or an inter-frame unidirectional prediction mode is used as the prediction mode for the samples of the first block according to the weight of the corresponding samples in the first prediction value and according to the weight of the corresponding samples in the second prediction value.

4. The apparatus according to claim 3, wherein when the weight of the corresponding sample in the first prediction value is higher than a level or the weight of the corresponding sample in the second prediction value is higher than a level, the sample of the first block is considered to use the inter-frame bidirectional prediction mode to determine the boundary strength.

5. A method for decoding an image, comprising: -decode a portion of an image portion; - determining at least one boundary between a first block of samples and a second block of samples, said first block and said second block belonging to the decoded image portion; - determining a boundary strength of a deblocking filter according to at least one of a prediction mode of the first block and a prediction mode of the second block; - filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength of the deblocking filter; Wherein determining the boundary strength further comprises: the prediction mode for the first block of samples is a geometric prediction mode, performing a weighted combination on a first prediction value determined by a first inter-frame unidirectional prediction and a second prediction value determined by a second inter-frame unidirectional prediction, and using one of an inter-frame bidirectional prediction mode or an inter-frame unidirectional prediction mode as the prediction mode for the samples of the first block according to the weight of the corresponding sample in the first prediction value or according to the weight of the corresponding sample in the second prediction value.

6. The method according to claim 5, wherein when the weight of the corresponding sample in the first prediction value is higher than a level or the weight of the corresponding sample in the second prediction value is higher than a level, the sample of the first block is considered to use the inter-frame bidirectional prediction mode to determine the boundary strength.

7. A method for encoding an image, comprising: - reconstruct image parts; - determining at least one boundary between a first block of samples and a second block of samples, said first block and said second block belonging to the reconstructed image portion; - determining a boundary strength of a deblocking filter according to at least one of a prediction mode of the first block and a prediction mode of the second block; - filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength of the deblocking filter; Wherein determining the boundary strength further comprises: the prediction mode for the first block of samples is a geometric prediction mode, performing a weighted combination on a first prediction value determined by a first inter-frame unidirectional prediction and a second prediction value determined by a second inter-frame unidirectional prediction, and using one of an inter-frame bidirectional prediction mode or an inter-frame unidirectional prediction mode as the prediction mode for the samples of the first block according to the weight of the corresponding sample in the first prediction value or according to the weight of the corresponding sample in the second prediction value.

8. A device for decoding an image, the device comprising one or more processors, the one or more processors being configured to: -decode a portion of an image portion; - determining at least one boundary between a first block of samples and a second block of samples, said first block and said second block belonging to the decoded image portion; - determining a boundary strength of a deblocking filter according to at least one of a prediction mode of the first block and a prediction mode of the second block; - filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength of the deblocking filter; Wherein determining the boundary strength further comprises: the prediction mode for the first block of samples is a geometric prediction mode, performing a weighted combination on a first prediction value determined by a first inter-frame unidirectional prediction and a second prediction value determined by a second inter-frame unidirectional prediction, and using one of an inter-frame bidirectional prediction mode or an inter-frame unidirectional prediction mode as the prediction mode for the samples of the first block according to the weight of the corresponding sample in the first prediction value or according to the weight of the corresponding sample in the second prediction value.

9. A device for encoding an image, the device comprising one or more processors, the one or more processors being configured to: - reconstruct image parts; - determining at least one boundary between a first block of samples and a second block of samples, said first block and said second block belonging to the reconstructed image portion; - determining a boundary strength of a deblocking filter according to at least one of a prediction mode of the first block and a prediction mode of the second block; - filtering samples of the first block and the second block adjacent to the at least one boundary according to the boundary strength of the deblocking filter; Wherein determining the boundary strength further comprises: The prediction mode for the first block of samples is a geometric prediction mode, a weighted combination is performed on a first prediction value determined by a first inter-frame unidirectional prediction and a second prediction value determined by a second inter-frame unidirectional prediction, and one of an inter-frame bidirectional prediction mode or an inter-frame unidirectional prediction mode is used as the prediction mode for the samples of the first block according to the weight of the corresponding sample in the first prediction value or according to the weight of the corresponding sample in the second prediction value.

10. A computer-readable medium comprising software code instructions for executing the method according to any one of claims 1, 5 and 7 when the software code instructions are executed by one or more processors.

Images