Method and device for picture encoding and decoding

Abstract

To provide a method and device for picture encoding and decoding.SOLUTION: First, parameters for a current block are decoded. A coding tool is disabled for the current block in the case where one or more of the following conditions is verified: a first coding tool is used for the current block and a given condition on the decoded parameters is verified. Otherwise, the coding tool is enabled for the current block. Finally, the current block is decoded responsive to the first and coding tools.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] At least one of the present aspects relates generally to methods and devices for picture encoding and decoding, and more particularly to methods and devices for picture encoding and decoding using various coding tools, such as Motion Compensation, Local Illumination Compensation, Generalized Bi-prediction, post-reconstruction filters, and the like. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes typically leverage spatial and temporal redundancy in video content using prediction and transformation. In general, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and the difference between an original image block and a predicted image block, often denoted as prediction error, prediction residual, or predictor, is transformed, quantized, and entropy coded. During encoding, the original image block is typically partitioned / divided into sub-blocks, possibly using quadtree partitioning. To reconstruct the video, the compressed data is decoded by inverse processes corresponding to prediction, transformation, quantization, and entropy coding. Summary of the Invention

[0003] According to a general aspect of at least one embodiment, a method for decoding video data is provided and includes: - Decoding the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the parameters being decoded is verified; - if not, enabling said encoding tool for said current block; and - Decoding said current block in response to the encoding tool.

[0004] According to a general aspect of at least one embodiment, a method for encoding video data is provided and includes: -Getting the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the obtained parameters is verified; - if not, enabling said encoding tool for said current block; and - encoding said current block in response to the encoding tool.

[0005] According to a general aspect of at least one embodiment, a decoding apparatus is provided that includes one or more processors configured to: - Decoding the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the parameters being decoded is verified; - if not, enabling said encoding tool for said current block; and - Decoding said current block in response to the encoding tool.

[0006] According to a general aspect of at least one embodiment, a method for encoding video data is provided and includes: -Getting the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the obtained parameters is verified; - if not, enabling said encoding tool for said current block; and - encoding said current block in response to the encoding tool.

[0007] According to a general aspect of at least one embodiment, an encoding apparatus is provided that includes one or more processors configured to: -Getting the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the obtained parameters is verified; - if not, enabling said encoding tool for said current block; and - encoding said current block in response to the encoding tool.

[0008] According to another general aspect of at least one embodiment, a bitstream is formatted to include a signal generated according to the encoding method described above.

[0009] Additionally, one or more of the present aspects also provide a computer-readable recording medium having stored thereon instructions for encoding or decoding video data according to at least a portion of any of the methods described above.

[0010] Furthermore, one or more aspects also provide a computer-readable recording medium having stored thereon a bitstream generated according to the encoding method described above. Furthermore, one or more aspects also provide a method and apparatus for transmitting or receiving a bitstream generated according to the encoding method described above. Furthermore, one or more aspects also provide a computer program product including instructions for performing at least a portion of any of the methods described above. Effect of the Invention

[0011] A method and device for picture encoding and decoding is provided. [Brief description of the drawings]

[0012] [Figure 1a] Draw the general decoding pipeline. [Figure 1b] Another general decoding pipeline is depicted below. [Figure 1c] Another general decoding pipeline is depicted below. [Figure 1d] 1 illustrates a general decoding pipeline according to one embodiment. [Diagram 2] Illustrates the principle of parameter derivation for Local Illumination Compensation. [Diagram 3] 1 illustrates the principle of overlapped block motion compensation. [Figure 4] Illustrates the principle of multiple hypothesis prediction in the case of inter and intra mode combinations. [Diagram 5] 1 illustrates a non-rectangular partitioning of a block and associated OBMC diagonal weighting. [Figure 6] 1 depicts a flowchart of a decoding method according to one embodiment. [Figure 7] 1 depicts a flowchart of an encoding method according to one embodiment. [Figure 8] 1 depicts a flowchart of a decoding process according to various aspects. [Figure 9] 1 is a flowchart of a multiple hypothesis prediction process. [Figure 10] 1 depicts a flowchart of a multiple hypothesis prediction process according to one embodiment. [Figure 11] 1 illustrates a flowchart of a motion compensated prediction process with local illumination compensation. [Figure 12] 1 depicts a flowchart of a prediction process for motion compensation with local illumination compensation according to one aspect. [Figure 13] 1 illustrates a general decoding pipeline with local illumination compensation computed before Decoder Side Motion Vector Refinement according to one embodiment. [Figure 14] Illustrates the principle of multiple hypotheses. [Figure 15] 1 illustrates a block diagram of a video encoder according to an embodiment. [Figure 16] 1 illustrates a block diagram of a video decoder according to an embodiment. [Figure 17] 1 illustrates a block diagram of an example system in which various aspects and aspects may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] In HEVC coding, a picture is divided into square-shaped CTUs with a configurable size, usually 64x64. The size can be 128x128 or 256x256 in other video coding standards. A CTU is the root of a quadtree division into four square coding units (CUs) of equal size, i.e. half the parent block size in width and in height. A quadtree is a tree in which a parent node can be divided into four child nodes, each of which can be a parent node for another four child nodes. In HEVC, a coding block (CB) contains samples of one color component corresponding to one CU. It is divided into one or more prediction blocks (PBs) and forms the root of the quadtree division into transform blocks (TBs). Corresponding to the coding block, prediction block, and transform block, a coding unit (CU) includes a prediction unit (PU) and a tree-structured set of transform units (TUs), where the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The sizes of CB, PB, and TB of the luma components are applied to the corresponding CU, PU, ​​and TU.

[0014] In more recent encoding systems, the CTU is the root of the coding tree division into coding units (CUs). A coding tree is a tree in which a parent node (usually corresponding to a block) can be divided into child nodes (e.g., 2, 3, or 4 child nodes), each of which can be a parent node for further division into another child node. In addition to the quadtree split mode, new split modes are further defined that increase the total number of possible split modes: the binary tree symmetric split mode, the binary tree asymmetric split mode, and the ternary tree split mode. A coding tree has only one root node, e.g., the CTU. The leaves of the coding tree are the terminal nodes of the tree. Each node of the coding tree represents a block that can be further divided into smaller blocks, also called subblocks. Once the division of the CTU into CUs has been determined, the CUs corresponding to the leaves of the coding tree are encoded. The division of the CTU into CUs and the coding parameters used to encode each CU (corresponding to the leaves of the coding tree) may be determined in the encoder by a rate-distortion optimization procedure.

[0015] In this application, the term "block" or "picture block" may be used to refer to any one of CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, the term "block" or "picture block" may be used to refer to macroblocks, partitions, and sub-blocks defined in H.264 / AVC or other video coding standards, and more generally to refer to arrays of samples of any size.

[0016] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture", and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0017] In this application, the terms "enabled" and "used" may be used interchangeably when they apply to coding tools at the block level. Similarly, "disabled" and "not used" may be used interchangeably when they apply to coding tools at the block level.

[0018] In this application, the terms "encoding tool" and "encoding process" may be used interchangeably.

[0019] In modern encoding systems, for example in the Versatile Video Coding (VVC) (known as VTM) reference software, the decoding pipeline includes several cascaded tools. The decoding pipeline for VVC is depicted in Figure 1a. The decoding process is done at the level of the coding unit (CU). A CU contains C square or rectangular blocks (e.g., C=3 for YUV or RGB chroma formats). In the 4:2:0 format, the two chroma blocks are downscaled with respect to the luma block by a ratio of two in both the horizontal and vertical directions.

[0020] The "entropy decoding" process parses the bitstream and decodes syntax elements such as the coding mode (e.g., inter / intra coding mode) and associated parameters (e.g., intra direction, inter motion vectors MV, and reference indexes).

[0021] Motion compensation (MC) uses an MC filter to interpolate one or two (in case of bi-prediction) rectangular block(s) (of the same size as the current CU) samples in the reference picture with a displacement equal to MV(mvX, mvY) in the reference picture in relation to the position of the current block.

[0022] Local Illumination Compensation (LIC) aims to compensate for possible illumination changes between the block being predicted and the reference blocks used in motion compensated (MC) temporal prediction. Typically, this is done using a linear IC model (a,b). S IC =a*S ref +b (formula 1) However, S ref is the reference sample value to be motion compensated, and S IC is the predicted sample value. The LIC parameters (a,b) are calculated for each block for which LIC is enabled.

[0023] There are several ways to calculate the LIC parameters (a,b). In one method, the decoder calculates the LIC parameters based on several reconstructed picture samples, for example, based on a reconstructed sample "y" localized to the left and / or above the current block and a reference picture sample "x" localized to the left and / or above the block to be motion compensated (FIG. 2a). In another method, the LIC parameters are calculated based on a reconstructed sample "y" localized to the left and / or above the current block and the corresponding predicted sample "x" used to reconstruct "y" (FIG. 2b). For the two methods just mentioned, the LIC parameters can be calculated and stored after the reconstruction of each block. The IC parameters (a,b) minimize the difference between sample "y" and "a.x+b" (e.g., least squares method).

[0024]

number

[0025] The two methods just mentioned are hereafter referred to as "conventional LIC". LIC uses reconstructed samples of neighboring CUs and therefore imposes dependencies with neighboring CUs. As a result, the current CU must wait for neighboring blocks to be completely reconstructed before the LIC parameters for the current block can be calculated and thus the current block can be reconstructed (see "feedback rec. sample" in Fig. 1a).

[0026] For bi-prediction, a variant of LIC (called "bi-dir-IC") derives IC parameters (a0, b0) and (a1, b1) directly from the two motion-compensated blocks (block-0 and block-1) used in bi-prediction. It involves estimating the illumination change between the two reference pictures to derive IC parameters (a0, b0) and (a1, b1) for the current block. For example, one calculates IC parameters (a, b) for LIC between motion-compensated block-1 and motion-compensated block-0. The second derives (a0, b0) and (a1, b1) from the values ​​(a, b) and the picture order counts (POC) of the current picture, reference picture 0, and reference picture 1. It is assumed that the values ​​a0, a1, b0, and b1 are linear functions of the temporal distance between the current frame and the reference pictures. The advantage of the variant just described is that it does not introduce any dependencies with nearby CUs.

[0027] In the case of bidirectional prediction, BIO (Bi-directional Optical flow) aims to refine the motion for each sample assuming a linear displacement between the two reference pictures using Hermitian interpolation of the optical flow. Thus, a new term is added to the classical bidirectional prediction.

[0028]

number

[0029] is added.

[0030]

number

[0031] however,

[0032]

number

[0033] are the motion compensated predictions (block-0 and block-1),

[0034]

number

[0035] and

[0036]

number

[0037] are the horizontal and vertical gradients for the i-th reference (k={0,1}), and (v x ,v y ) is the motion vector between block-1 and block-0, and σ is the temporal difference (POC1-POC0).

[0038] In the case of bi-prediction, Decoder Side Motion Vector Refinement (DMVR) aims to refine the motion by testing several displacements around the MV value being decoded (MVinitial). A set of several candidate MV pairs (MVtested(L0),MVtested(L1)) is tested. As in BIO, the tested MV values ​​are symmetric around the initial MV value, i.e. (MVtested(L0)-MVinitial(L0)=MVtested(L1)-MVinitial(L1)), where MVtested(L0) is the motion vector tested for list L0 and MVtested(L1) is the motion vector tested for list L1. The selected MV pair is, for example, the pair that minimizes the difference MC(ref0,MVtested(L0))-MC(ref1,MVtested(L1)) using MRSAD (Mean-Removed Sum of Absolute Difference).

[0039] For bi-prediction, two prediction blocks are calculated and averaged together (aka bi-prediction averaging). For generalized bi-prediction (GBI), they are combined with a weighted sum as follows: P GBI =(((1<<gs)-gw1).P0+gw1.P1)> >gs (Formula 4) where P0, P1 are motion compensated predictions (block-0 and block-1).

[0040] In the VTM, the weight to use is coded per CU by the “gbi-index”, which can take the values ​​listed in Table 1.

[0041] [Table 1]

[0042] Hereinafter, the pair of weights {1 / 2;-1 / 2} will be referred to as the default weights.

[0043] In Figure 1a, the LIC process is applied before the bi-predictive averaging in each unidirectional motion compensated prediction. In a variant, the LIC process (720) is applied after the bi-predictive averaging (710) depicted in Figure 1b.

[0044] According to an overlapping block motion compensation (OBMC) tool (hereinafter referred to as the “OBMC process”), the motion of the neighboring blocks is calculated as N To extend the prediction of (N=A or L), we use the current block prediction P C It is used to gradually weight the A Let (x,y) be the predicted sample (x={0,···W-1},y={0,···H A -1}). The example just described can be expanded by adding an additional B row at the bottom, P A The other prediction samples of are calculated using the same process (MC) and the same motion information. A (x,y)(x={0,···W-1},y=(H A , H A +B-1}. P C (x,y) is the current prediction P OBMC Sample(x={0,···W-1},y={0,···H C -1}). Then, the prediction for the top B rows of samples in the current block is finally computed as P OBMC (x,y)=w N (y).P A (x,y+B)+w C (y).P A (x,y) (Eq. 5) The same process is carried out on the P built from the left block data. L It is applied to the prediction sample in the left column B based on the (x,y) sample.

[0045] If the neighboring block N uses LIC and / or BIO for its prediction, then an additional P N The sample (hereinafter referred to as "OBMC treatment") may be obtained by applying LIC and BIO.

[0046] For example, a typical graduated weight from the border to the center of the current block is P N and P C For each, w C (y)={1 / 4,1 / 8,1 / 16,1 / 32} and w N (y)={3 / 4,7 / 8,15 / 16,31 / 32}.

[0047] The general Multi-Hypothesis concept includes the combination of inter prediction performed in merge mode with intra prediction mode or with another inter mode (e.g., uni-predictive AMVP, skip, and merge), as depicted in FIG. 4. In merge mode, a list of merge candidates (reference index, motion value) is built, and a merge index that identifies one candidate is signaled. Then, in merge mode, the motion information of the block is derived from the candidate identified by the merge index and used for motion compensated prediction. The final prediction is a weighted average of the merge indexed prediction and the prediction generated by the intra prediction mode, but different weights are applied according to the combination. The intra prediction mode is signaled in the bitstream. In a particular aspect, the intra prediction mode is selected in a subset (e.g., subset size equal to 4) of all classical prediction modes. The current block is divided into four equal-area regions. The weights gradually decrease as the regions move away from the intra reference sample. Each set of weights, denoted as (w_intrai,w_interi), where i runs from 1 to 4, (w_intra1,w_inter1)=(6,2), (w_intra2,w_inter2)=(5,3), (w_intra3,w_inter3)=(3,5), and (w_intra4,w_inter4)=(2,6), will be applied to the corresponding region, as depicted in the examples of Figures 4 and 14, for intra-vertical prediction. When DC or planar mode is selected, or the width or height of a CU is less than 4, the exact same weights are applied to the entire CU.

[0048] The geometric (e.g. triangular) mode allows more flexibility for partitioning the picture into blocks before encoding. An example of such partitioning is depicted in the upper part of FIG. 5. Additional coding performance can be obtained by weighting the two predictors on the frontier (gray area in FIG. 5) illustrated in the lower part of FIG. 5. The weighting process just described may be cascaded with other weights, e.g. GBI or LIC, increasing the implementation complexity.

[0049] A “post-reconstruction” filter (730) allows filtering reconstructed samples of a current CU in a CU loop reconstruction, so that the filtered samples may be used for intra prediction or for LIC parameter derivation for subsequent CUs in the same picture (or same slice or same tile), whereas “other post-filters” (740) (e.g., a deblocking filter, a SAO filter, or an ALF filter) filter reconstructed samples after the entire picture (or slice or tile) is reconstructed. Examples of "post-reconstruction" filters are bilateral filters (J. Strom et al., "CE14: Reduced latency, LUT-free bilateral filter", document JVET-L0172, 12th Conference, Macau, CN, 3-12 October 2018), Hadamard filters (S. Ikonen et al., "Non-CE: Hadamard transform domain filter", document JVET-M0468, 13th Conference, Marrakech, MA, 9-18 January 2019), or diffusion filters (J. Rash et al., "CE10: Uniform Directional Diffusion Filters For Video Coding", document JVET-M0042, 13th Conference, Marrakech, MA, 9-18 January 2019).

[0050] In a variant, the "post-reconstruction" filter is only applied to CUs coded in inter mode, depicted in FIG. 1c, and not to CUs coded in intra or combined intra and inter prediction. In another variant, the "post-reconstruction" filter is only applied to CUs coded in intra mode, and not in inter mode. Application of the "post-reconstruction" filter may be conditioned on other parameters, such as the current CU size, QP value, or component type (e.g., the "post-reconstruction" filter is not applied to the chroma component, depicted by the dashed chroma box at 730).

[0051] New video codecs implement an increasing number of coding tools that can be cascaded to improve coding efficiency. In some cases, cascading tools of the same nature can be counterproductive, i.e., reduce coding efficiency, and what has just been mentioned can put a heavy burden on the decoding pipeline.

[0052] At least one aspect is intended to aid in the implementation of a video codec on various platforms by harmonizing the interaction between different encoding tools and ensuring reasonable pipeline complexity. At least one aspect is intended to apply some rules and limitations without degrading the overall codec performance. At least one embodiment reduces both the pipeline size (number of processing stages) and the interleaved pipeline dependencies by using some tools in "inter" coding, e.g., by applying restrictions on mutual co-activation.

[0053] To reduce pipeline dependency, the "inter" decode pipeline is modified.

[0054] FIG. 6 depicts a flowchart of a decoding method according to one embodiment.

[0055] The parameters are decoded for the current block (S100). The coding tool (e.g., LIC) for the current block is disabled (S110) if one or more of the following conditions are verified: a first coding tool is used for said current block (e.g., GBI is used with a weight that is not a default weight), and a condition given for the parameters being decoded (block size is less than a value) is verified. The coding tool is enabled for said current block if not. The current block is decoded taking into account whether the coding tool is enabled (S120). If the coding tool is enabled, the current block is decoded using the coding tool. If not, the current block is decoded without using the coding tool.

[0056] FIG. 7 illustrates a flowchart of an encoding method according to an embodiment. Parameters are obtained for a current block (S200). An encoding tool (e.g., LIC) for the current block is disabled (S110) if one or more of the following conditions are verified: a first encoding tool is used for said current block (e.g., GBI is used with a weight that is not a default weight), and a condition given to the parameters being decoded (block size is less than a value) is verified. An encoding tool is enabled for said current block if not. The current block is encoded taking into account whether the encoding tool is enabled (S220). If the encoding tool is enabled, the current block is encoded using the encoding tool. If not, the current block is encoded without using the encoding tool.

[0057] The disabling of the encoding tool is conditioned on some characteristics of the current encoding unit (CU) and / or on whether another encoding tool "X" is used (406, 408) as illustrated by Figures 8A-8C.

[0058] In Figure 8A, the CU parameters are decoded (400). If a condition on the CU parameters is verified (410), for example, if the CU size is less than a value, then encoding tool / process "Y" is disabled (420).

[0059] In FIG. 8B, the CU parameters are decoded (400). If the first encoding tool "X" is used for the current CU (406), then the first encoding tool is applied (408). If a condition on the CU parameters is further verified (410), for example, if the CU size is less than a value, the second encoding tool / process is disabled. If not, the second encoding tool is used (420).

[0060] In a variant, if a first encoding tool "X" is used for the current CU (406), then the first encoding tool is applied (408) and the second encoding tool "Y" is disabled. In the variant just described, the condition for the CU is not checked.

[0061] In Fig. 8C, the CU parameters are decoded (400). If both the first encoding tool "X" is used for the current CU (406) and the conditions on the CU parameters are verified (410), then the second encoding tool "Y" is disabled. If not, the second encoding tool is used (420). The dotted line means that the application of encoding tool "X" is conditioned by 406.

[0062] A variant is built from a representation of a "condition (CU)" and the processing order of tools "X" and "Y" (Figure 8-B or C). The "condition (CU)" is derived from the previously decoded data (400) and can be a combination of the following rules: -Disable tool "X" for small blocks (CU size less than value), - Disable tool "X" for saturation (CU component is saturation), - Disable tool "X" if the current slice has a TiD (Temporal Identifier) ​​that is above the value, - Disable tool "X" if it requires information (e.g., reconstructed samples) from another reconstructed block located in another CTU or outside the current VPDU (Virtual Pipeline Data Unit).

[0063] Reducing Pipeline Dependence in Multiple Hypothesis Mode FIG. 9 illustrates the inter-prediction pipeline process of the VVC software for MH when the input bit depth or the internal bit depth is equal to 10 bits, and when bi-directional prediction, BIO, and GBI are also used. First, motion compensation is applied to build a prediction (500) of two reference blocks. The result is stored (510) in N bits, where N>10, to preserve more accuracy in further processing (e.g., N=14 bits), i.e., BIO (520) and GBI weighting (530). Next, the sample values ​​are stored (e.g., right shifted) (540) and clipped (550) at the internal bit depth (e.g., 10 bits), and the subsequent MH process (560) is performed with the internal bit depth (e.g., 10 bits).

[0064] "Multiple Hypotheses" (MH) increase the number of processing steps in the inter pipeline because they use an additional "intra" or "inter" prediction calculation and combine it with another prediction, as shown in Figure 9. One may alleviate the pipeline with one or more of the following limitations:

[0065] In one aspect, LIC is disabled for one or all inter modes when MH is used. What has just been said means that if the current inter mode is "merge" and LIC is disabled because MH is used, the IC-flag is inferred to be zero, not merged. The IC-flag is a parameter that indicates whether or not to use LIC for a block.

[0066] In one aspect, inter bi-prediction is disabled when MH is used as shown in Figure 10. Additionally, BIO is also disabled (by design) since it only applies to bi-prediction.

[0067] In one aspect, triangle mode is disabled when MH is used. Indeed, there is a weighting process for the two triangle predictions already depicted in FIG.

[0068] Furthermore, one can improve the MH combining stage when adding intra prediction with increasing bit depth precision to inter prediction and before the clipping operation. In Figure 10, the MH processing is performed at N bits.

[0069] Reducing Pipeline Dependency on LIC: FIG. 11 illustrates the inter prediction pipeline process of the VVC software for LIC when the input bit depth or the internal bit depth is equal to 10 bits and when bidirectional prediction is used. First, motion compensation is applied to build a prediction (600) of two reference blocks. The results are stored (610) in the precision of the internal bit depth (e.g., 10 bits). Next, LIC parameters are estimated or retrieved from previously stored data and illumination compensation is applied (670), which includes weighting, plus adding an offset. Next, a right shift (640) to the internal bit depth (10 bits) and clipping (650) are applied. Next, the samples are converted (680) to N-bit precision before applying GBI weights (630). P LIC =(a.P+b)>>shift LIC (Formula 6) where (a;b) are LIC parameters, P is the block sample being motion compensated, shift LC , P LIC is a right shift of 10 bits, P LIC is the predicted sample by LIC.

[0070] The sample values ​​are then stored (eg, right shifted) (640) and clipped (650) at an internal bit depth (eg, 10 bits).

[0071] Essentially, LIC plugs in the pipeline dependencies due to other nearby reconstructed blocks, and adds a step after motion compensation that changes the lighting, as shown in Figure 11. The dependencies just mentioned can be reduced by using one or more of the following constraints / restrictions:

[0072] In one aspect, LIC is disabled for chroma components and / or small blocks (CU size less than the value).

[0073] In one aspect, for chroma components and / or small blocks (CU size less than the value), the GBI is disabled (GBI weights are set to default weights).

[0074] Since the principle of LIC is to weight the prediction samples obtained by motion compensation, when the weights are not equal to the default weights, there are some similarities with GBI.

[0075] In one aspect, LIC is disabled in bi-prediction if the GBI weight is not the default weight. Therefore, the IC flag is not coded if the GBI-index is different from the default. In merge mode, if the GBI index is inherited, the IC flag is inferred to be false if the GBI weight is not the default. In a variant, if the IC flag is inherited and the value is true, the GBI weight is inferred to be the default weight. In a variant, if the GBI weight is not the default, one uses bi-dir-IC instead of conventional LIC.

[0076] In one embodiment, the IC flag is coded and if the IC flag is true then the GBI index is not coded and the GBI weights are inferred as default weights. In a variant, in merge mode the IC flag is inherited and if the value is true then the GBI weights are inferred as default weights.

[0077] An example syntax is provided in Table 2.

[0078] [Table 2]

[0079] In one aspect, if the GBI weight is not the default weight, the IC flag is coded even if the current CU is coded in "merge" mode.

[0080] In one embodiment, the LIC is a reconstituted sample, “rec N " is disabled if it belongs to one neighboring CU (called N) that is coded with a non-default GBI weight.

[0081] A Virtual Pipeline Data Unit (VPDU) is defined as a non-overlapping MxM luma (L) / NxN chroma (C) unit in a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. Different stages process different VPDUs simultaneously. Typically, a VPDU is a 64x64 luma / 32x32 chroma block. The following three conditions need to be verified: For each VPDU that contains one or more CUs, the CUs are completely contained in the VPDU. For each CU that contains one or more VPDUs, the VPDUs are completely contained in the CU. - The processing order of the CU will not leave a VPDU behind and will not return to it later.

[0082] The block reconstruction dependencies inserted by the LIC may be tolerable for data in the cache memory, typically corresponding to one CTU or one VPDU, which may be related to the hardware and / or implementation memory cache constraints / limitations for decoding elementary CUs, and further related to the maximum sample region size that the hardware can process all-in-one.

[0083] In one embodiment, the LIC is a block of adjacent blocks used to derive the LIC parameters. N In the variant, the LIC may be used but only samples that are reconstructed inside the current CTU or VPDU.

[0084] In one aspect, if the coding mode of the current block is "merge", the merge index indicates that the N neighbors are used to merge, and the block just mentioned has an IC flag equal to true (IC parameter is indicated). N If "rec N If the sample is outside a CTU or VPDU, then the IC flag is true for the current block and one must use the rec N " Instead of using samples, IC N (LIC parameters used by adjacent block N).

[0085] In one embodiment, the LIC is calculated (670) with increasing precision, as depicted in FIG. LIC (Equation 6) is set so that the sample precision is N bits after the LIC process, and samples are clipped only at the end of the prediction process.

[0086] Reduced pipeline dependency for "post-reconstruction" filters: In one aspect, relative to each other, LIC processing and the "post-reconstruction" filtering processing are exclusive for each component (i.e., if LIC is enabled for one component, then the "post-reconstruction" filter is disabled for the just-mentioned component) or exclusive for all components (if LIC is enabled for all components, then the "post-reconstruction" filter is disabled for all components).

[0087] If "post-reconstruction" is not applied to one component (e.g., chroma), then LIC is disabled for the current CU (e.g., the LIC flag is false), "post-reconstruction" may be applied only to the other component (e.g., luma).

[0088] For example, if LIC is enabled and / or applied to the current CU (e.g., the LIC flag is true, inferred, or coded), post-reconstruction filtering is disabled for all components depicted in FIG. 1d.

[0089] In another example, if LIC processing can be enabled per component (e.g., one LIC flag inferred or coded for luma and another LIC flag inferred or coded for chroma), then the “post-reconstruction” filtering process is disabled for a component if the LIC flag is enabled for the component just mentioned.

[0090] Pipeline Reduction for OBMC: OBMC may have smaller coding efficiency for small blocks while the average amount of processing per sample is significantly increased when the number of small blocks in a picture is large. One may ease the pipeline by one or more of the following limitations: In one aspect, OBMC is disabled (i.e., not used) for small blocks (e.g., 4x4). In one aspect, OBMC is not performed on chroma blocks. In one aspect, OBMC is disabled (i.e., not used) if the current block uses LIC. In one aspect, LIC is not used in the OBMC process. In one aspect, BIO is not used in the OBMC process.

[0091] Pipeline reduction for BIO: In one aspect, the BIO is disabled if the GBI weight is not the default. BIO and DMVR are two techniques with the same goal, which is to refine the MV. Therefore, BIO is disabled if DMVR is used for the current block and vice versa (mutually exclusive).

[0092] Pipeline Reduction for DMVR: In the current state of the art, LIC is disabled by the DMVR. Indeed, enabling LIC, which involves the calculation of LIC parameters for each MV to be tested, significantly increases the computational complexity. It has been proposed to perform DMVR without LIC and enable LIC processing after the DMVR stage, as depicted in Figure 1. However, since the DMVR does not include LIC, the accuracy of MV refinement is significantly reduced. To address the limitations just mentioned, one could implement a method to perform DMVR without LIC (before MV refinement by DMVR) and perform LIC processing after the DMVR stage. initial is used to perform the LIC parameter calculation without refining the LIC parameters in the DMVR process, i.e., the LIC parameters are not calculated in or after the DMVR process, as illustrated in FIG.

[0093] Pipeline reduction for the deblocking filter: In one aspect, the strength of the DBF is reduced on an edge if OBMC is applied to the edge and / or if the edge belongs to a block that has LIC or BIO enabled.

[0094] This application describes various aspects, including tools, features, aspects, models, approaches, and the like. Many of the just described aspects are described with particularity and in a manner that may often seem limiting, at least to illustrate their individual characteristics. However, what is just described is for purposes of clarity in description and does not limit the application or scope of the just described aspects. Indeed, all of the different aspects can be combined and substituted to provide further aspects. Furthermore, aspects can be combined and substituted with aspects described in earlier applications as well.

[0095] The aspects described and contemplated in this application can be implemented in many different ways. Below, Figures 15, 16, and 17 provide some aspects, but other aspects are contemplated, and Figures 15, 16, and 17 do not limit the breadth of implementations. In general, at least one of the aspects relates to video encoding and decoding, and in general, at least one other aspect relates to transmitting the generated or encoded bitstream. The just-described aspects and other aspects can be implemented as a method, an apparatus, a computer-readable medium having instructions stored thereon, and / or a computer-readable medium having a bitstream generated according to any of the described methods, for encoding or decoding video data according to any of the described methods.

[0096] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the inherent operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0097] Various methods and other aspects described in the present application may be used to modify, for example, the motion compensation modules (170, 275), the motion refinement modules (190, 290), the LIC modules (195, 295), and the multiple predictions combination modules (105, 270) of the video encoder 100 and the decoder 200 shown in Figures 15 and 16. Furthermore, the aspects are not limited to VVC or HEVC, and may be applied to other standards and recommendations, for example, whether pre-existing or developed in the future, as well as extensions of any of the above standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically prohibited, the aspects described in the present application may be used individually or in combination.

[0098] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes and the aspects described are not limited to the specific values ​​just mentioned.

[0099] 15 illustrates an encoder 100. Although variations of the encoder 100 are envisioned, the encoder 100 is described below for purposes of clarity without describing all contemplated variations.

[0100] Before being encoded, a video sequence may go through a pre-encoding process (101), for example, applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or remapping the input picture components to make the signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata is associated with the pre-processing and can be tied to the bitstream.

[0101] In the encoder 100, a picture is encoded by the encoder elements described below. The picture to be encoded is divided (102) and processed, for example, in units of CUs. Each unit is encoded, for example, using either intra or inter mode. When a unit is encoded in intra mode, intra prediction (160) is performed. In inter mode, motion estimation (175), possibly motion refinement (190), motion compensation (170), and possibly illumination compensation (195) are performed. Motion refinement may use DMVR or BIO. The encoder decides (105) which one of intra or inter modes to use to encode the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. In addition, the encoder may combine (105) several predictions, for example, in MH mode or bidirectional prediction mode. A prediction residual is calculated, for example, by subtracting (110) the block being predicted from the original image block.

[0102] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., the residual is coded directly without applying either the transform or quantization process.

[0103] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual is combined (155) with the predicted block to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed picture, for example to perform deblocking / Sample Adaptive Offset (SAO) filtering to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0104] Figure 16 illustrates a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by the decoder elements described below. In general, the video decoder 200 performs a decoding pass that is the inverse of the encoding pass described in Figure 15. Additionally, the encoder 100 also generally performs video decoding as part of encoding the video data.

[0105] In particular, the decoder input includes a video bitstream that may be generated by the video encoder 100. First, the bitstream is entropy decoded (230) to obtain transform coefficients, motion vectors, and other coding information. Picture partition information indicates how the picture is partitioned. Therefore, the decoder may partition (235) the picture according to the decoded picture partition information. The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the predicted block are combined (255) to reconstruct an image block. The predicted block may be obtained from intra prediction (260), motion compensated prediction (i.e., inter prediction) (275), or a combination (270) of several predictions (e.g., in MH mode or in bidirectional mode). Furthermore, the motion compensated prediction may be compensated for illumination (295). Like the encoder, the decoder also includes a module configured to refine (290) the motion vectors being decoded (e.g., using DMVR or BIO). An in-loop filter (265) is applied to the image being reconstructed. The filtered image is stored in a reference picture buffer (280).

[0106] Additionally, the picture being decoded may go through a post-decoding process (285), such as an inverse color conversion (e.g., YCbCr 4:2:0 to RGB 4:4:4 conversion) or an inverse remapping that reverses the remapping process performed in the pre-encoding process (101). The post-decoding process may use metadata signaled in the bitstream that was derived in the pre-encoding process.

[0107] FIG. 17 illustrates a block diagram of an example of a system in which various aspects and embodiments are implemented. System 1000 can be embodied as a device including various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as, for example, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television broadcast receivers, personal video recording systems, connected consumer electronics appliances, and servers. The elements of system 1000, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or separate components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various aspects, the system 1000 is configured to implement one or more aspects described in this document.

[0108] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein to implement various aspects described herein, for example. The processor 1010 can include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which can include non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash, magnetic disk drives, and / or optical disk drives. The storage devices 1040 may include, by way of non-limiting example, internal storage devices, attached storage devices (including detachable and non-detachable storage devices), and / or network-accessible storage devices.

[0109] The system 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. The encoder / decoder module 1030 represents a module or modules that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as known to those skilled in the art.

[0110] Program code loaded onto the processor 1010 or encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various aspects, one or more of the processors 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during execution of processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing of equations, formulas, operations, and operational logic. In some aspects, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other aspects, memory external to the processing device (e.g., the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of the just-mentioned functions. The external memory can be the memory 1020 and / or the storage device 1040, for example, a dynamic volatile memory and / or a non-volatile flash memory. In some aspects, the external non-volatile flash memory is used to store, for example, the operating system of the television. In at least one embodiment, a high-speed external dynamic volatile memory such as a RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group, which is further called ISO / IEC 13818, which is further known as H.222, and which is further known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, which is further known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a standard being developed by JVET, the Joint Video Experts Team).

[0111] Inputs to the elements of system 1000 can be provided via a variety of input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) an RF section that receives a radio frequency (RF) signal transmitted, for example, over the air by a broadcast station, (ii) a Component (COMP) input terminal (or set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples not shown in FIG. 10 include composite video.

[0112] In various aspects, the input devices of block 1130 are associated with respective input processing elements known in the art. For example, the RF section can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select a single frequency band, which in some aspects can be referred to as a channel (e.g.), (iv) demodulating the down-converted band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various aspects includes one or more elements performing the just-mentioned functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section can include, for example, a tuner performing various functions, including down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to baseband) or to baseband. In one set-top box embodiment, the RF section and associated input processing elements receive, filter, downconvert, and perform frequency selection by refiltering RF signals transmitted over a wired medium (e.g., cable) to the desired frequency band. Various embodiments rearrange the order of the above-mentioned (and other) elements, remove some of the elements just mentioned, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0113] Additionally, the USB and / or HDMI terminals can include respective interface processors to interface the system 1000 to other electronic devices over USB and / or HDMI. It is understood that various aspects of the input processing, e.g., Reed-Solomon error correction, can be implemented, for example, in a separate input processing IC or within the processor 1010, as appropriate. Similarly, aspects of the USB or HDMI interface processing can be implemented in a separate input processing IC or within the processor 1010, as appropriate. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, the processor 1010 and an encoder / decoder 1030, which operates in cooperation with memory and storage elements to process the data stream for presentation at an output device, as appropriate.

[0114] The various elements of system 1000 can be provided within an integrated housing in which the various elements can be interconnected and transmit data therebetween using suitable coupling arrangements, such as internal buses known in the art, including an I2C (Inter-IC) bus, wiring, and printed circuit boards.

[0115] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented within a wired and / or wireless medium, for example.

[0116] Data is streamed or otherwise provided to the system 1000 in various embodiments using a wireless network, such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE citing the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in the embodiment just described is received via a communication channel 1060 and a communication interface 1050 adapted for Wi-Fi communication. Typically, the communication channel 1060 in the embodiment just described is coupled to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to the system 1000 using a set-top box that delivers data via an HDMI connection in the input block 1130. Still other embodiments provide streamed data to the system 1000 using an RF connection in the input block 1130. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0117] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 of various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, a tablet, a laptop, a cellular (mobile) phone, or other device. Furthermore, the display 1100 can be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). In various example embodiments, the other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of the system 1000. For example, a disc player is responsible for playing the output of the system 1000.

[0118] In various aspects, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling such as, for example, AV.Link, CEC (Consumer Electronics Control), or other communication protocols that allow device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be coupled to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and speaker 1110 can be integrated in a single unit with other components of the system 1000, for example, in an electronic device such as a television. In various aspects, the display interface 1070 includes a display driver such as, for example, a timing controller (T Con) chip.

[0119] Alternatively, the display 1100 and speakers 1110 can be separate from one or more other components, for example if the RF portion of the input 1130 is part of a separate set-top box. In various aspects where the display 1100 and speakers 1110 are external components, the output signal can be submitted via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0120] The embodiments can be performed by computer software implemented by the processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. The memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 1010 can be of any type suitable for the technical environment and can include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0121] Various implementations include decoding. As used herein, "decoding" can include all or part of the processing performed on the received encoding sequence to generate a final output suitable for display, for example. In various aspects, the processing includes one or more of the processing typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various aspects, the processing also or alternatively includes the processing performed by the decoder of the various implementations described herein, such as motion and illumination compensation, motion refinement, and multiple predictions combination.

[0122] As a further example, in one embodiment "decoding" refers only to "entropy decoding," in another embodiment "decoding" refers only to differential decoding, and in another embodiment "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or more generally, it is believed that the decoding process will be clear based on the context of the particular description and will be well understood by those of ordinary skill in the art.

[0123] Various implementations include encoding. In a manner similar to the above discussion of "decoding," "encoding" as used herein can include all or part of the processing performed on an input video sequence, for example, to generate an encoded bitstream. In various aspects, the processing includes one or more of the processing typically performed by an encoder, such as partitioning, differential encoding, transforming, quantization, and entropy encoding. In various aspects, in addition to or as an alternative, the processing includes the processing performed by an encoder in various implementations described herein, such as motion and illumination compensation, motion refinement, and multiple predictions combination.

[0124] As a further example, in one embodiment "encoding" refers only to "entropy decoding," while in another embodiment "encoding" refers only to differential de-encoding, while in another embodiment "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding" is intended to refer specifically to a subset of operations or more generally, it is believed that the encoding process will be clear based on the context of the particular description and will be well understood by those of ordinary skill in the art. Note that the syntax elements used herein, e.g., IC flags, are descriptive terms. As noted above, they do not preclude the use of other syntax element names.

[0125] It should be understood that when a diagram is provided as a flow diagram, it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is provided as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0126] Various aspects refer to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, often subject to computational complexity constraints. Typically, rate-distortion optimization is formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate-distortion optimization problem. For example, an approach may be based on an extensive test of all encoding options, including all considered modes or coding parameter values, together with a full evaluation of the coding cost and associated distortion of the reconstructed signal after encoding and decoding. Furthermore, faster approaches may be used, in particular with a calculation of the approximated distortion based on a prediction or prediction residual signal rather than the reconstructed one, in order to reduce the encoding complexity. A mixture of the two approaches just mentioned may also be used, for example by using approximated distortion for only some of the possible encoding options and full distortion for other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches use any of a variety of techniques to perform an optimization, but the optimization is not necessarily a complete evaluation of both the coding cost and the associated distortion.

[0127] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if described only in the context of a single form of implementation (e.g., described only as a method), the implementation of the described features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. A method may be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Additionally, a processor also includes a communication device, such as, for example, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate communication of information between end users.

[0128] Reference to "one embodiment" or "embodiment" or "one implementation" or "implementation" means that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment, as well as other variations. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" appearing in various places throughout this application are not necessarily all referring to the same embodiment, as well as any other variations.

[0129] Additionally, the application may refer to "determining" various portions of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0130] Additionally, the application may refer to "accessing" various portions of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0131] Additionally, the application may refer to "receiving" various portions of information. Receiving, as with "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" is typically included among operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information in some way or another.

[0132] It should be understood that the use of any of the following " / ", "and / or" and "at least one of" is intended to include, for example, in the case of "A / B", "A and / or B" and "at least one of A and B", the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", the phrase is intended to include the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). What has just been said may be expanded beyond those items merely listed, as would be apparent to one of ordinary skill in the art and related arts.

[0133] Furthermore, as used herein, the word "signaling" refers to, among other things, indicating something to a corresponding decoder. For example, in an aspect, an encoder signals a particular one of the encoding modes. In this way, in an aspect, the same parameters are used on both the encoder side and the decoder side. Thus, for example, an encoder can transmit a particular parameter to a decoder (explicit signaling), and the decoder can use the same particular parameter. Conversely, if the decoder already has a particular parameter as well as others, signaling can be used without transmission (implicit signaling) to simply enable the decoder to know and select the particular parameter. By avoiding the transmission of any actual function, bit savings are realized in various aspects. It is understood that signaling can be accomplished in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various aspects. Although the foregoing has concerned the verb form of the word "signaling," the word "signaling" may also be used herein as a noun.

[0134] As would be apparent to one skilled in the art, the implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. For example, the information can include instructions for performing a method or data generated by one of the described implementations. For example, the signals can be formatted to carry a bit stream of the described aspects. For example, the signals can be formatted as electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or as baseband signals. For example, the formatting can include encoding a data stream and modulating a carrier wave with the encoded data stream. For example, the information the signals carry can be analog or digital information. The signals can be transmitted over a variety of separate wired or wireless links, as is known. The signals can be stored in a processor-readable medium.

[0135] We describe a number of embodiments. The features of the embodiments just described may be provided alone or in any combination. In addition, an embodiment may include one or more of the following features, devices, or aspects, alone or in any combination, across the various claim categories and types. A television, set-top box, mobile phone, tablet, or other electronic device that performs motion and lighting compensation, motion refinement, and multi-prediction combining adaptation according to any of the described aspects. A television, set-top box, mobile phone, tablet, or other electronic device that performs the motion and lighting compensation, motion refinement, and multi-prediction combining adaptation in accordance with any of the described aspects and displays the resulting images (e.g., using a monitor, screen, or other type of display). A television, set-top box, mobile phone, tablet, or other electronic device that tunes (e.g., using a tuner) to receive a signal including an encoded image and performs motion and illumination compensation, motion refinement, and multi-prediction combining adaptation in accordance with any of the described aspects. A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal over the air (e.g., using an antenna) that includes the encoded image and performs motion and illumination compensation, motion refinement, and multi-prediction combining adaptation according to any of the described aspects.

[0136] According to a general aspect of at least one embodiment, a decoding method is disclosed that includes: - Decoding the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the parameters being decoded is verified; - if not, enabling said encoding tool for said current block; and - Decoding said current block in response to the encoding tool.

[0137] According to a general aspect of at least one embodiment, a method of encoding is disclosed that includes: -Getting the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the parameters being decoded is verified; - if not, enabling said encoding tool for said current block; and - encoding said current block in response to the encoding tool.

[0138] According to a general aspect of at least one embodiment, a decoding device is disclosed that includes one or more processors configured to: - Decoding the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the parameters being decoded is verified; - if not, enabling said encoding tool for said current block; and - Decoding said current block in response to the encoding tool.

[0139] According to a general aspect of at least one embodiment, an encoding device is disclosed that includes one or more processors configured to: -Getting the parameters for the current block; - disabling the encoding tool for said current block if one or more of the following conditions are verified: a first encoding tool is used for said current block, and a condition provided for the parameters being decoded is verified; - if not, enabling said encoding tool for said current block; and - encoding said current block in response to the encoding tool.

[0140] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling inter bidirectional prediction if a multiple hypothesis encoding mode is used for the current block.

[0141] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling local illumination compensation if a multiple hypothesis encoding mode is used for the current block:

[0142] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling triangulation if a multiple hypothesis encoding mode is used for the current block:

[0143] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling local lighting compensation or OBMC for one or more of the sub-blocks and chroma components:

[0144] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling local illumination compensation if generalized bi-prediction weights are not equal to default weights.

[0145] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling generalized bi-prediction (or setting generalized bi-prediction weights to default weights) and not encoding a generalized bi-prediction index if local illumination compensation is enabled for the current block.

[0146] In one aspect, disabling the encoding tool for the current block when one or more of the following conditions are verified includes disabling local lighting compensation when a neighboring sample of the current block used by local lighting compensation belongs to a block that uses generalized bidirectional prediction weights that are not equal to default weights.

[0147] In one aspect, disabling the encoding tool for the current block if one or more of the following conditions are verified includes disabling overlap block motion compensation if local illumination compensation is used.

[0148] In one aspect, disabling the encoding tool for the current block when one or more of the following conditions are verified includes disabling local illumination compensation and / or generalized bidirectional prediction when overlap block motion compensation is used and weights are not equal to default weights in the overlap block motion compensation process.

Claims

1. decoding a first block level parameter for the current block; disabling a bidirectional optical flow mode for the current block in response to a first condition being verified, the bidirectional optical flow mode enabling refining a motion of samples of the block based on optical flow, the first condition being verified in response to the decoded first block level parameters indicating that a multiple hypothesis coding mode combining inter prediction and intra prediction is used for the current block, the multiple hypothesis coding mode including obtaining a predictor for the current block, the predictor being a weighted average between an inter predictor of the current block and an intra predictor of the current block; decoding the current block; A decoding method comprising:

2. 2. The method of claim 1, wherein the first condition is further verified in response to a decoded second block level parameter indicating that generalized bi-prediction weights are not equal to default weights for the current block.

3. 2. The method of claim 1 , further comprising disabling a triangular mode for the current block in response to the first condition being verified, the triangular mode including dividing a predictor block into two partitions separated by a diagonal edge, each partition being associated with a different inter predictor block.

4. 2. The method of claim 1, further comprising disabling a generalized bidirectional prediction mode for the current block or setting a weight of the generalized bidirectional prediction mode to a default weight for the current block in response to a second condition being verified, the second condition being verified in response to a size of the block falling below a value.

5. obtaining a first block level parameter for a current block; disabling a bidirectional optical flow mode for the current block in response to a first condition being verified, the bidirectional optical flow mode enabling a motion of samples of the block to be refined based on optical flow, the first condition being verified in response to the obtained first block level parameters indicating that a multiple hypothesis coding mode combining inter prediction and intra prediction is used for the current block, the multiple hypothesis coding mode including obtaining a predictor for the current block, the predictor being a weighted average between an inter predictor of the current block and an intra predictor of the current block; encoding the current block; An encoding method comprising:

6. 6. The method of claim 5, wherein the first condition is further verified in response to a second block level parameter indicating that generalized bi-prediction weights are not equal to default weights for the current block.

7. 6. The method of claim 5, further comprising disabling a triangular mode for the current block in response to the first condition being verified, the triangular mode including dividing a predictor block into two partitions separated by a diagonal edge, each partition being associated with a different inter predictor block.

8. 6. The method of claim 5, further comprising disabling a generalized bidirectional prediction mode for the current block or setting a weight of the generalized bidirectional prediction mode to a default weight for the current block in response to a second condition being verified, the second condition being verified in response to the size of the block falling below a value.

9. Decoding a first block level parameter for the current block; disabling a bidirectional optical flow mode for the current block in response to a first condition being verified, the bidirectional optical flow mode enabling a motion of samples of the block to be refined based on optical flow, the first condition being verified in response to the decoded first block level parameters indicating that a multiple hypothesis coding mode combining inter prediction and intra prediction is used for the current block, the multiple hypothesis coding mode including obtaining a predictor for the current block, the predictor being a weighted average between an inter predictor of the current block and an intra predictor of the current block; Decoding the current block; One or more processors configured to implement A decoding device comprising:

10. 10. The decoding apparatus of claim 9, wherein the first condition is further verified in response to a decoded second block level parameter indicating that generalized bidirectional prediction weights are not equal to default weights for the current block.

11. 10. The decoding apparatus of claim 9, wherein the one or more processors are further configured to, in response to the first condition being verified, implement disabling a triangular mode for the current block, the triangular mode including dividing a predictor block into two partitions separated by a diagonal edge, each partition being associated with a different inter predictor block.

12. 10. The decoding device of claim 9, wherein the one or more processors are further configured to disable a generalized bidirectional prediction mode for the current block or set a weight of the generalized bidirectional prediction mode to a default weight for the current block in response to a second condition being verified, the second condition being verified in response to the size of the block falling below a value.

13. Obtaining a first block level parameter for a current block; disabling a bidirectional optical flow mode for the current block in response to a first condition being verified, the bidirectional optical flow mode enabling a motion of samples of the block to be refined based on optical flow, the first condition being verified in response to the obtained first block level parameters indicating that a multiple hypothesis coding mode combining inter prediction and intra prediction is used for the current block, the multiple hypothesis coding mode including obtaining a predictor for the current block, the predictor being a weighted average between an inter predictor of the current block and an intra predictor of the current block; encoding said current block One or more processors configured to implement An encoding device comprising:

14. 14. The encoding apparatus of claim 13, wherein the first condition is further verified in response to a second block level parameter indicating that generalized bi-prediction weights are not equal to default weights for the current block.

15. 14. The encoding apparatus of claim 13, wherein the one or more processors are further configured to, in response to the first condition being verified, implement disabling a triangular mode for the current block, the triangular mode including dividing a predictor block into two partitions separated by a diagonal edge, each partition being associated with a different inter predictor block.

16. 14. The encoding apparatus of claim 13, wherein the one or more processors are further configured to disable a generalized bidirectional prediction mode for the current block or set a weight of the generalized bidirectional prediction mode to a default weight for the current block in response to a second condition being verified, the second condition being verified in response to the size of the block falling below a value.

Images