Method and apparatus for picture encoding and decoding

Abstract

Disclosed are a method and apparatus for picture encoding and decoding. First, a parameter of a current block is decoded. If one or more of the following conditions are validated, a coding tool is disabled for the current block: a first coding tool is used for the current block and a given condition regarding the decoded parameter is validated. Otherwise, the coding tool is enabled for the current block. Finally, in response to the first sum coding tool, the current block is decoded.

Description

[0001] This application is a divisional application of Chinese patent application No. 201980076913.2, filed on November 15, 2019, entitled “Method and Device for Picture Encoding and Decoding”, and the contents of the parent application are incorporated herein by reference. 1. Technical Field

[0002] At least one embodiment of the present invention generally relates to a method and apparatus for picture encoding and decoding, and more specifically, to a method and apparatus for picture encoding and decoding using various decoding tools (e.g., motion compensation, local illumination compensation, generalized bi-prediction, post-reconstruction filter, etc.). 2. Background technology

[0003] In order to achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame frame correlation, and then the difference between the original image block and the predicted image block (which is usually expressed as a prediction error, prediction residual or prediction value) is transformed, quantized and entropy encoded. During encoding, the original image block may usually be partitioned / divided into sub-blocks using quadtree partitioning. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to the prediction, transformation, quantization and entropy decoding. 3. Summary of the invention

[0004] According to a general aspect of at least one embodiment, a method for decoding video data is proposed, comprising:

[0005] – Decode the parameters of the current block;

[0006] - disabling the decoding tool for the current block if one or more of the following conditions are verified: a first decoding tool is used for the current block, and a given condition on the decoded parameter is verified;

[0007] - otherwise, enabling the decoding tool for the current block; and

[0008] - In response to the decoding tool, decoding the current block.

[0009] According to a general aspect of at least one embodiment, a method for encoding video data is proposed, comprising:

[0010] – Get the parameters of the current block;

[0011] - disabling the decoding tool for the current block if one or more of the following conditions are verified: the first decoding tool is used for the current block, and the given condition on the obtained parameter is verified;

[0012] - otherwise, enabling the decoding tool for the current block; and

[0013] - In response to the decoding tool, encoding the current block.

[0014] According to a general aspect of at least one embodiment, a decoding apparatus is provided, comprising one or more processors, wherein the one or more processors are configured to execute:

[0015] – Decode the parameters of the current block;

[0016] - disabling the decoding tool for the current block if one or more of the following conditions are verified: a first decoding tool is used for the current block, and a given condition on the decoded parameter is verified;

[0017] - otherwise, enabling the decoding tool for the current block; and

[0018] - In response to the decoding tool, decoding the current block.

[0019] According to a general aspect of at least one embodiment, a method for encoding video data is proposed, comprising:

[0020] – Get the parameters of the current block;

[0021] - disabling the decoding tool for the current block if one or more of the following conditions are verified: the first decoding tool is used for the current block, and the given condition on the obtained parameter is verified;

[0022] - otherwise, enabling the decoding tool for the current block; and

[0023] - In response to the decoding tool, encoding the current block.

[0024] According to a general aspect of at least one embodiment, there is provided an encoding apparatus comprising one or more processors configured to perform:

[0025] – Get the parameters of the current block;

[0026] - disabling the decoding tool for the current block if one or more of the following conditions are verified: the first decoding tool is used for the current block, and the given condition on the obtained parameter is verified;

[0027] - otherwise, enabling the decoding tool for the current block; and

[0028] - In response to the decoding tool, encoding the current block.

[0029] According to another general aspect of at least one embodiment, a bitstream is formatted to include a signal generated according to the above-described encoding method.

[0030] One or more of the embodiments of the present invention also provide a computer-readable storage medium on which instructions for encoding or decoding video data according to at least part of any of the above methods are stored. One or more embodiments also provide a computer-readable storage medium on which a bit stream generated according to the above encoding method is stored. One or more embodiments also provide a method and apparatus for sending or receiving a bit stream generated according to the above encoding method. One or more embodiments also provide a computer program product including instructions for executing at least part of any of the above methods. 4. Brief description of the accompanying drawings

[0032] FIG. 1A depicts an overall decoding pipeline, FIG. 1B and FIG. 1C depict other overall decoding pipelines, and FIG. 1D depicts an overall decoding pipeline according to one embodiment.

[0033] Figure 2 The principle of derivation of local illumination compensation parameters is shown;

[0034] Figure 3 The principle of overlapping block motion compensation is shown;

[0035] Figure 4 The principle of multi-hypothesis prediction in case of combination of inter- and intra-mode is shown;

[0036] Figure 5 Non-rectangular partitioning of blocks and associated OBMC diagonal weighting are shown;

[0037] Figure 6 depicts a flowchart of a decoding method according to one embodiment;

[0038] Figure 7 depicts a flow chart of an encoding method according to one embodiment;

[0039] Figure 8 depicts a flow chart of a decoding process according to various embodiments;

[0040] Fig. 9 A flowchart depicting the multi-hypothesis forecasting process;

[0041] Fig.10depicts a flow chart of a multi-hypothesis prediction process according to one embodiment;

[0042] Fig.11 depicts a flow chart of the motion compensated prediction process with local illumination compensation;

[0043] Fig.12 depicts a flow chart of a motion compensated prediction process with local illumination compensation according to one embodiment;

[0044] Fig.13 depicts an overall decoding pipeline with local illumination compensation calculated before decoder-side motion vector refinement according to one embodiment;

[0045] Fig.14 The principle of multiple hypotheses is shown;

[0046] Fig.15 shows a block diagram of a video encoder according to an embodiment;

[0047] Fig.16 A block diagram of a video decoder according to an embodiment is shown; and

[0048] Fig.17 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. 5. Specific implementation methods

[0049] In HEVC coding, pictures are partitioned into square-shaped CTUs with a configurable size, typically 64×64. In other video coding standards, the size can be 128×128 or 256×256. CTU is the root of a quadtree partitioned into 4 equally sized square coding units (CUs), i.e., the size is half the size of the parent block in width and height. A quadtree is a tree in which a parent node can be divided into four child nodes, each of which can become another parent node divided into four child nodes. In HEVC, a coding block (CB) contains samples corresponding to one color component of a CU. It is partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree partitioned into transform blocks (TBs). Corresponding to the coding blocks, prediction blocks, and transform blocks, a coding unit (CU) includes a prediction unit (PU) and a tree structure set with a transform unit (TU), the PU includes prediction information for all color components, and the TU includes a residual coding syntax structure for each color component. The sizes of CB, PB, and TB of the luma component are applicable to the corresponding CU, PU, ​​and TU.

[0050] In recent coding systems, CTU is the root of a coding tree partitioned into coding units (CUs). A coding tree is a tree in which a parent node (usually corresponding to a block) can be divided into child nodes (e.g., divided into 2, 3, or 4 child nodes), each of which can become a parent node for another partition into child nodes. In addition to the quadtree partition mode, new partition modes (binary tree symmetric partition mode, binary tree asymmetric partition mode, and ternary tree partition mode) are defined, which increases the total number of possible partition modes. The coding tree has a unique root node, such as CTU. The leaves of the coding tree are the terminal nodes of the tree. Each node of the coding tree represents a block, which can be further partitioned into smaller blocks (also called sub-blocks). Once the partition of CTU to CU is determined, the CU corresponding to the leaf of the coding tree is encoded. The partitioning of CTU into CUs and the coding parameters for encoding each CU (corresponding to the leaf of the coding tree) can be determined on the encoder side by a rate-distortion optimization process.

[0051] In this application, the term "block" or "picture block" may be used to refer to any of CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, the term "block" or "picture block" may be used to refer to macroblocks, partitions, and sub-blocks as specified in H.264 / AVC or other video coding standards, and more generally to arrays of samples of various sizes.

[0052] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0053] In this application, the terms "enable" and "use" can be used interchangeably when they apply to block-level decoding tools. In the same way, "disable" and "not use" can be used interchangeably when they apply to block-level decoding tools.

[0054] In this application, the terms "coding tool" and "coding process" are used interchangeably.

[0055] In recent coding systems, such as in the reference software for Versatile Video Coding (VVC), called VTM, the decoding pipeline consists of several cascaded tools. Figure 1a The decoding pipeline for VVC is depicted in . The decoding process is done at the coding unit (CU) level. A CU consists of C square or rectangular blocks (e.g., C=3 for YUV or RGB chroma formats). In the 4:2:0 format, the two chroma blocks are scaled down relative to the luma block by a ratio of two in both the horizontal and vertical directions.

[0056] The "entropy decoding" process parses the bitstream and decodes syntax elements, such as coding modes (eg, inter / intra coding modes) and associated parameters (eg, intra direction, inter motion vectors MV, and reference indices).

[0057] Motion compensation (MC) interpolates samples of one or two (in case of bi-prediction) rectangular blocks (same size as current CU) in the reference picture using MC filters, where the displacement is equal to the MV (mvX, mvY) relative to the position of the current block in the reference picture.

[0058] Local illumination compensation (LIC) aims to compensate for illumination changes that may occur between a predicted block and its reference block used in motion compensated (MC) temporal prediction. This is usually done using a linear IC model (a, b):

[0059] S IC =a*S ref +b(Equation 1)

[0060] Where: S ref is the motion compensation reference sample value, and S IC is the predicted sample value. LIC parameters (a, b) are calculated for each block with LIC enabled.

[0061] There are several ways to calculate the LIC parameters (a, b). In one method, the decoder calculates the LIC parameters (a, b) based on some reconstructed picture samples (e.g., based on reconstructed samples "y" positioned to the left and / or top of the current block and reference picture samples "x" positioned to the left and / or top of the motion compensated block, see Figure 2 a), calculate LIC parameters. In another method, the LIC parameters are calculated based on the reconstructed sample "y" located at the left and / or top of the current block and the corresponding prediction sample "x" used to reconstruct "y". Figure 2 b). For both methods, the LIC parameters may be calculated and stored after each block reconstruction. The IC parameters (a, b) minimize the difference (e.g., least squares) between sample "y" and "a.x+b":

[0062]

[0063] These two methods are referred to as "traditional LIC" in the following. LIC introduces dependencies with neighboring CUs because it uses reconstructed samples of neighboring CUs. Therefore, before the LIC parameters of the current block can be calculated and the current block can therefore be reconstructed, the current CU must wait for the neighboring blocks to be fully reconstructed (see Figure 1a "Feedback Reconstruction Sample (feedback rec.sample)" in .

[0064] In the case of bi-directional prediction, a variant of LIC (called "bi-dir-IC") derives IC parameters (a0, b0) and (a1, b1) directly from the two motion compensated blocks (block 0 and block 1) used in bi-directional prediction. It includes estimating the brightness change between the two reference pictures to derive IC parameters (a0, b0) and (a1, b1) of the current block. For example, the IC parameters (a, b) of the LIC between motion compensated block 1 and motion compensated block 0 are calculated. Next, (a0, b0) and (a1, b1) are derived from the values ​​(a, b) and the picture order counts (POC) of the current picture, reference picture 0, and reference picture 1. It is assumed that the values ​​a0, a1, b0, b1 are linear functions of the temporal distance between the current frame and the reference picture. The advantage of this variant is that it does not introduce any dependency with neighboring CUs.

[0065] In the case of bidirectional prediction, bidirectional optical flow (BIO) aims to refine the motion of each sample assuming a linear displacement between two reference pictures using Hermite interpolation of optical flow. Therefore, a new term is added to the traditional bidirectional prediction:

[0066]

[0067] in:

[0068] I 0 ,I 1 is the motion compensated prediction (block 0 and block 1),

[0069] and are the horizontal and vertical gradients of the ith reference (k = {0, 1}),

[0070] (v x ,v y ) is the motion vector between block 1 and block 0,

[0071] σ is the time difference (POC1-POC0).

[0072] In the case of bidirectional prediction, decoder-side motion vector refinement (DMVR) aims to refine the motion by testing some displacements around the decoded MV value (MVinitial). A set of several MV pairs of candidates is tested (MVtested(L0), MVtested(L1)). As in BIO, the tested MV values ​​are symmetric about the initial MV values, i.e., (MVtested(L0)-MVinitial(L0)=MVtested(L1)-MVinitial(L1)), where MVtested(L0) is the motion vector tested for list L0 and MVtested(L1) is the motion vector tested for list L1. The selected MV pair is the one that minimizes the difference MC(ref0,MVtested(L0))-MC(ref1,MVtested(L1)) using, for example, the mean removed sum of absolute differences (MRSAD).

[0073] In the case of bi-directional prediction, two prediction blocks are calculated and averaged together (also called bi-prediction averaging). In the case of generalized bi-directional prediction (GBI), they are combined with a weighted sum as follows:

[0074] P GBI =(((1<<gs)-gw1).P0+gw1.P1)> >gs(Equation 4)

[0075] Where P0, P1 are motion compensated predictions (block 0 and block 1).

[0076] In VTM, the weight to be used is encoded for each CU by a “gbi index” which can take the values ​​mentioned in Table 1.

[0077]

[0078] Table 1. Binarization of GBi index and associated weights.

[0079] Hereinafter, the weight pair {1 / 2; 1 / 2} will be referred to as the default weights.

[0080] exist Figure 1a In , the LIC process is applied before bidirectional prediction averaging is performed on each unidirectional motion compensated prediction. In a variant, as Figure 1b As depicted in , after the bidirectional prediction averaging ( 710 ), the LIC process ( 720 ) is applied.

[0081] According to the Overlapped Block Motion Compensation (OBMC) tool (hereinafter referred to as the "OBMC process"), the motion of a neighboring block is used to extend the prediction P for that neighboring block. N (N = A or L) and use the current block to predict PC To perform weighted ( Figure 3 ). For example, let P A (x, y) represents the upper prediction sample of the upper block (x={0,…W-1}, y={0,..H A -1}). These samples are extended by an additional B rows at the bottom: P A (x,y)(x={0,…W-1},y={H A ,..H A +B-1}, which is used with P A The other prediction samples are calculated using the same process (MC) and the same motion information. Assume that P C (x,y) represents the current prediction P OBMC Sample (x={0,…W-1},y={0,..H C -1}), then the prediction for the top B rows of samples in the current block is finally calculated as:

[0082] P OBMC (x,y)=w N (y).P A (x,y+B)+w C (y).P A (x,y) (Equation 5)

[0083] Based on P constructed from the left block data L (x,y) samples, and apply the same process to the prediction samples in column B on the left.

[0084] In the case where the neighboring block N uses LIC and / or BIO for its prediction, additional P can also be obtained by applying LIC and BIO N Sample (hereinafter referred to as "OBMC process").

[0085] For example, for P N and P C , the typical step-by-step weights from the boundary to the current block center are w C (y) = {1 / 4, 1 / 8, 1 / 16, 1 / 32} and w N (y)={3 / 4,7 / 8,15 / 16,31 / 32}.

[0086] The general concept of multiple assumptions includes Figure 4The inter-frame prediction performed in the merge mode as depicted above is combined with the intra-frame prediction mode or with another inter-frame mode (for example, unidirectional prediction AMVP, skip and merge). In the merge mode, a list of merge candidates (reference index, motion value) is constructed and a merge index identifying a candidate is signaled. Then, from the candidate identified by the merge index, the motion information of the block in the merge mode is derived and used for motion compensated prediction. The final prediction is a weighted average of the prediction about the merge index and the prediction generated by the intra-frame prediction mode, where different weights are applied depending on the combination. The intra-frame prediction mode is signaled in the bitstream. In a specific embodiment, the intra-frame prediction mode is selected from a subset of all classic prediction modes (for example, a subset size equal to 4). The current block is divided into 4 equal-area regions. As the region moves away from the intra-frame reference sample, the weight decreases gradually. As Figure 4 and 14 As shown in the example, each weighting set (which is represented as (w_intrai, w_interi), where i is 1 to 4, and (w_intra1, w_inter1) = (6, 2), (w_intra2, w_inter2) = (5, 3), (w_intra3, w_inter3) = (3, 5), and (w_intra4, w_inter4) = (2, 6)) will be applied to the corresponding region for intra vertical prediction. When DC or planar mode is selected or the width or height of the CU is less than 4, the same weight is applied to the entire CU.

[0087] Geometric (e.g., triangle) modes allow for more flexibility in partitioning pictures into blocks before decoding. Examples of such partitioning are Figure 5 As shown at the top. Figure 5 As shown at the bottom of Figure 5 Additional decoding performance can be obtained by weighting the two prediction values ​​(predictor) on the gray area of ​​​​the gray area. This weighting process can be cascaded with other weightings such as GBI or LIC, and increases the complexity of the implementation.

[0088] The "post-reconstruction" filter (730) allows filtering of the reconstructed samples of the current CU in CU loop reconstruction so that the filtered samples can be used for intra prediction or for LIC parameter derivation of subsequent CUs in the same picture (or the same slice or the same tile). Whereas the "other post-filters" (740) (e.g., deblocking filter, SAO filter, or ALF filter) filter the reconstructed samples after the entire picture (or slice or tile) has been reconstructed. Examples of “reconstructed” filters are bilateral filters (J. Strom et al., “CE14: Reduced latency, LUT-free bilateral filter”, document JVET-L0172, 12th conference: Macau, China, October 3-12, 2018), Hadamard filters (S. Ikonen et al., Non-CE: Hadamard transform domain filter”, document JVET-M0468, 13th conference: Marrachec, MA, January 9-18, 2019), or diffusion filters (J. Rash et al., “CE10: Uniform Directional Diffusion Filters For Video Coding”, document JVET-M0042, 13th conference: Marrakech, MA, January 9-18, 2019).

[0089] In one variation, the “post-reconstruction” filter is applied only to CUs coded in inter mode, and not to CUs coded in intra or with combined intra and inter prediction, such as Figure 1c In another variation, the "post-reconstruction" filter is applied to CUs that are coded only in intra mode and not in inter mode. The application of the "post-reconstruction" filter may be conditioned on other parameters, such as the current CU size, QP value, or component type (e.g., as shown by the dashed chroma box in 730, the "post-reconstruction" filter is not applied to chroma components).

[0090] New video codecs implement a greater number of decoding tools that can be cascaded to improve decoding efficiency. In some cases, cascading tools of the same nature may be counterproductive, i.e., reduce decoding efficiency, and this may also increase the burden on the pipeline of the decoding process.

[0091] At least one embodiment proposes to coordinate the interaction between different decoding tools and help implement video codecs on various platforms by ensuring reasonable pipeline complexity.At least one embodiment proposes to apply some rules and restrictions without degrading the overall codec performance.

[0092] At least one embodiment proposes reducing pipeline size (number of processing stages) and pipeline dependencies introduced by some tools in "inter-frame" decoding, which can be achieved, for example, by adding restrictions on their mutual co-activation.

[0093] To reduce pipeline dependencies, modify the "inter" decoding pipeline.

[0094] Figure 6 A flowchart of a decoding method according to one embodiment is described.

[0095] Parameters are decoded for a current block (S100). A decoding tool (e.g., LIC) for the current block is disabled (S110) if one or more of the following conditions are verified: a first decoding tool is used for the current block (e.g., GBI is used with weights other than default weights), and a given condition (block size is less than a value) about the decoded parameters is verified. Otherwise, the decoding tool is enabled for the current block. Consider whether the decoding tool is enabled to decode the current block (S120). If the decoding tool is enabled, the current block is decoded using the decoding tool. Otherwise, the current block is not decoded using the decoding tool.

[0096] Figure 7 A flow chart depicting an encoding method according to one embodiment. Parameters of a current block are obtained (S200). A decoding tool (e.g., LIC) of the current block is disabled (S110) if one or more of the following conditions are verified: a first decoding tool is used for the current block (e.g., GBI is used with weights that are not default weights), and a given condition (block size is below a value) regarding the decoded parameters is verified. Otherwise, the decoding tool is enabled for the current block. Considering whether the decoding tool is enabled to encode the current block (S220). If the decoding tool is enabled, the current block is encoded using the decoding tool. Otherwise, the current block is not encoded using the decoding tool.

[0097] like Figure 8 As shown in 8A to 8C, the disabling of a coding tool may be adjusted according to some properties of the current coding unit (CU) and / or whether another coding tool "X" (406, 408) is used.

[0098] exist Figure 8In A, CU parameters are decoded (400). In case the condition of the CU parameters is verified (410), for example, in case the CU size is below a value, the coding tool / process "Y" is disabled (420).

[0099] exist Figure 8 In B, CU parameters are decoded (400). In the case that a first decoding tool "X" is used for the current CU (406), then the first decoding tool is applied (408). In the case that the condition of the CU parameters is further verified (410), for example, in the case that the CU size is below a value, the second decoding tool / process is disabled. Otherwise, the second decoding tool is used (420).

[0100] In a variation, where a first coding tool "X" is used for the current CU (406), then the first coding tool is applied (408) and the second coding tool "Y" is disabled. In this variation, conditions on the CU are not checked.

[0101] exist Figure 8 In C, CU parameters are decoded (400). If the first coding tool "X" is used for the current CU (406) and the condition on the CU parameters is verified (410), then the second coding tool "Y" is disabled. Otherwise, the second coding tool is used (420). The dotted line indicates that the application of the coding tool "X" is regulated by 406.

[0102] The variant is derived from the expression of the “condition (CU)” and the process sequence of tools “X” and “Y” ( Figure 8 -B or C). The "condition (CU)" is derived from previously decoded data (400) and can be a combination of the following rules:

[0103] – For small blocks (CU size below one value), disable tool “X”,

[0104] – For Chroma (CU components are Chroma), disable tool “X”,

[0105] – Disable tool "X" if the current slice has a time identifier (TiD) higher than a value,

[0106] - If a tool "X" needs information (eg, reconstructed samples) from another reconstructed block located in another CTU or outside the current VPDU (Virtual Pipeline Data Unit), then the tool "X" is disabled.

[0107] Reducing pipeline dependencies for multi-hypothesis mode

[0108] Fig. 9The VVC software inter prediction pipeline process for MH is shown when the input bit depth or internal bit depth is equal to 10 bits and when bidirectional prediction, BIO and GBI are also used. First, motion compensation is applied to construct two reference block predictions (500). The result is stored in N bits, where N>10 (510) to maintain higher accuracy (e.g., N=14 bits) in further processes, namely BIO (520) and GBI weighting (530). Next, the sample values ​​are stored (e.g., right shifted) (540) and cropped (550) in the internal bit depth (e.g., 10 bits) so that the subsequent MH process (560) is performed at the internal bit depth (e.g., 10 bits).

[0109] Since "Multiple Hypothesis" (MH) uses an additional "Intra" or "Inter" prediction calculation and combines it with another prediction, such as Fig. 9 As shown, it therefore increases the number of processing steps in the inter-frame pipeline. One can mitigate the pipeline by imposing one or more of the following restrictions.

[0110] In one embodiment, LIC is disabled for one or all inter-modes in case of MH use. This means that if the current inter-mode is "Merge" and LIC is disabled because MH is used, the IC flag is not merged but inferred to be zero. The IC flag is a parameter that indicates for a block whether it uses LIC.

[0111] In one embodiment, in Fig.10 In the case of MH shown, inter bi-prediction is disabled. BIO is also disabled (by design) as it only works with bi-prediction.

[0112] In one embodiment, when using MH, triangle mode is disabled. In fact, you already have a weighting process for two triangle predictions, such as Figure 5 Depicted above.

[0113] Furthermore, the MH combining level can be improved when intra prediction is added to inter prediction with increased bit depth accuracy and before the cropping operation. Fig.10 In , MH processing is performed with N bits.

[0114] Reduce LIC's pipeline dependency:

[0115] Fig.11The VVC software inter prediction pipeline process for LIC is depicted when the input bit depth or internal bit depth is equal to 10 bits and when bidirectional prediction is used. First, motion compensation is applied to construct two reference block predictions (600). The results are stored at the internal bit depth precision (e.g., 10 bits) (610). Next, LIC parameters are estimated or retrieved from previously stored data and illumination compensation is applied (670). It includes weighting plus offsets. Next, a right shift to the internal bit depth (10 bits) (640) and cropping (650) are applied. The samples are then transformed to N-bit precision (680) before applying the GBI weights (630).

[0116] P LIC =(a.P+b)>>shift LIC (Equation 6)

[0117] Where: (a; b) are LIC parameters,

[0118] P is the motion compensation block sample,

[0119] shift LC is a right shift, so that P LIC is 10 bits,

[0120] P LIC is the prediction sample using LIC.

[0121] Next, the sample values ​​are stored (eg, right-shifted) (640) and clipped (650) at the internal bit depth (eg, 10 bits).

[0122] Basically, LIC introduces pipeline dependencies with other neighboring reconstructed blocks and adds a step after motion compensation to change the brightness, e.g. Fig.11 This dependency can be reduced by using one or more of the following constraints / restrictions.

[0123] In one embodiment, LIC is disabled for chroma components and / or small blocks (CU size below a value).

[0124] In one embodiment, GBI is disabled (or GBI weights are set to default weights) for chroma components and / or small blocks (CU size below a value).

[0125] Since the principle of LIC is to weight the prediction samples obtained by motion compensation, it has some similarities with GBI when the weight is not equal to the default weight.

[0126] In one embodiment, LIC is disabled in bi-directional prediction when the GBI weight is not the default weight. Thus, if the GBI index is different from the default value, the IC flag is not decoded. In merge mode, if the GBI index is inherited, the IC flag is inferred to be false if the GBI weight is not the default value. In a variant, if the IC flag is inherited and its value is true, the GBI weight is inferred to be the default weight. In a variant, if the GBI weight is not the default, bi-directional IC (bi-dir-IC) is used instead of the traditional LIC.

[0127] In one embodiment, when the IC flag is decoded and the IC flag is true, the GBI index is not decoded and the GBI weight is inferred as the default weight. In a variant, in the merge mode, when the IC flag is inherited and its value is true, the GBI weight is inferred as the default weight.

[0128] An example of the syntax is provided in Table 2.

[0129]

[0130] Table 2 Example decoding unit syntax, where the GBI index is decoded conditionally based on the IC_flag value

[0131] In one embodiment, in case the GBI weight is not the default weight, the IC flag is coded even if the current CU is coded in the "merge" mode.

[0132] In one embodiment, in the reconstruction sample "rec N LIC is disabled in the case of a neighboring CU (called N) that has been coded with non-default GBI weights.

[0133] A Virtual Pipeline Data Unit (VPDU) is defined as a non-overlapping MxM-luma(L) / NxN-chroma(C) unit in a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. Different stages process different VPDUs simultaneously. Typically, a VPDU is a 64×64 luma / 32×32 chroma block. The following three conditions must be verified:

[0134] – For each VPDU containing one or more CUs, the CU is completely contained in the VPDU.

[0135] - For each CU containing one or more VPDUs, the VPDU is fully contained in the CU.

[0136] – The processing order of the CU shall not leave the VPDU and revisit it later.

[0137] The block reconstruction dependency introduced by using LIC is acceptable for data in the cache memory, which typically corresponds to one CTU or to one VPDU, which is related to the hardware and / or implementation memory cache constraints / limitations for decoding the basic CU and also to the maximum sample region size that the hardware can handle in one piece.

[0138] In one embodiment, the reconstructed samples “rec N "Outside a CTU or VPDU, LIC is disabled. In a variant, LIC may be used, but only using reconstructed samples within the current CTU or VPDU.

[0139] In one embodiment, in the following cases: the decoding mode of the current block is "merge", the merge index indicates that the neighbor N is used for merging, and the block has an IC flag equal to true (the IC parameter is denoted as IC N ), and "rec N "The sample is outside the CTU or VPDU, then the IC flag is true for the current block and the IC is reused N (LIC parameters used by neighboring blocks N) to build the prediction of the current block, instead of using “rec N " sample to derive the current LIC parameters.

[0140] In one embodiment, the LIC is calculated (670) with increased precision, i.e., setting the shift LIC (Equation 6), so that the sample accuracy is N bits after the LIC process, and the samples are only clipped at the end of the prediction process, as Fig.12 shown.

[0141] Reduce pipeline dependency of "post-reconstruction" filter:

[0142] In one embodiment, the LIC process and the “post-reconstruction” filtering process are mutually exclusive for each component (i.e., if LIC is enabled for one component, the “post-reconstruction” filter is disabled for that component) or mutually exclusive for all components (i.e., if LIC is enabled for all components, the “post-reconstruction” filter is disabled for all components).

[0143] If “after reconstruction” does not apply to one component (eg, chroma), then “after reconstruction” may only apply to other components (eg, luma) if LIC is disabled for the current CU (eg, LIC flag is false).

[0144] For example, if LIC is enabled and / or applied to the current CU (eg, LIC flag is true, inferred, or decoded), then Figure 1d As depicted in , post-reconstruction filtering is disabled for all components.

[0145] In another example, if the LIC process can be enabled per component (e.g., one LIC flag is inferred or decoded for luma and another LIC flag is inferred or decoded for chroma), then if the LIC flag is enabled for a component, the “post-reconstruction” filtering process is disabled for that component.

[0146] Reduce pipeline for OBMC:

[0147] OBMC may have lower decoding efficiency for small blocks, while the average processing per sample increases significantly when the number of small blocks in the picture is high. One can mitigate the pipeline by one or more of the following restrictions.

[0148] In one embodiment, for small blocks (e.g., 4×4), OBMC is disabled (i.e., not used). In one embodiment, chroma block OBMC is not performed. In one embodiment, OBMC is disabled (i.e., not used) in the case where the current block uses LIC. In one embodiment, LIC is not used for the OBMC process. In one embodiment, BIO is not used for the OBMC process.

[0149] Reduce the pipeline used for BIO:

[0150] In one embodiment, if the GBI weight is not the default value, the BIO is disabled.

[0151] BIO and DMVR are two technologies with the same purpose, i.e., thinning MV. Therefore, in the case where DMVR is used for the current block, BIO is disabled, and vice versa (mutually exclusive).

[0152] Reduce the pipeline for DMVR:

[0153] In the prior art, LIC is disabled by DMVR. In fact, enabling LIC (including the calculation of LIC parameters for each MV to be tested) significantly increases the computational effort. It has been proposed to perform DMVR without LIC and, after the DMVR stage, enable the LIC process, as depicted in FIG1 . However, since DMVR does not include LIC, the accuracy of MV refinement is significantly reduced. To cope with this limitation, it is proposed to utilize MV initial LIC parameter calculation is performed (before MV refinement using DMVR) and LIC parameters are not refined during the DMVR process, i.e., LIC parameters are not calculated during or after the DMVR process, such as Fig.13 As depicted in .

[0154] Reduce the pipeline for the deblocking filter:

[0155] In one embodiment, DBF strength is reduced on an edge if OBMC is applied on the edge, and / or DBF strength is reduced if the edge belongs to a LIC or BIO enabled block.

[0156] The present application describes a number of aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described as having specificity, and at least in order to illustrate individual characteristics, are usually described in a manner that may sound limited. However, this is for the purpose of describing clearly, and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide other aspects. In addition, these aspects can also be combined and interchanged with the aspects described in earlier documents.

[0157] The aspects described and contemplated in this application can be implemented in many different forms. Fig.15 , 16 17 provide some embodiments, but other embodiments are contemplated and are Fig.15 , 16 The discussion of 17 does not limit the breadth of implementation. At least one of the aspects is generally related to video encoding and decoding, and at least one other aspect is generally related to transmitting the generated or encoded bitstream. These and other aspects can be implemented as methods, devices, computer-readable storage media having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having stored thereon bitstreams generated according to any of the described methods.

[0158] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0159] Various methods and other aspects described in this application can be used to modify modules, e.g. Fig.15 and Fig.16 The motion compensation module (170, 275), motion refinement module (190, 290), LIC module (195, 295), and multi-prediction combination module (105, 270) are shown. In addition, the present invention is not limited to VVC or HEVC, and can be applied to, for example, other standards and proposals (whether pre-existing or developed in the future) and extensions of any such standards and proposals (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0160] Various numerical values ​​are used in this application. The specific values ​​are for illustrative purposes, and the described aspects are not limited to these specific values.

[0161] Fig.15 An encoder 100 is shown. Variations of the encoder 100 are contemplated, but for the sake of clarity, the encoder 100 is described below without describing all contemplated variations.

[0162] Before being encoded, the video sequence may undergo a pre-encoding process (101), for example, applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input picture components in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0163] In encoder 100, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (102) and processed in units such as CUs. Each unit is encoded using, for example, intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175), possible motion refinement (190), motion compensation (170) and possible illumination compensation (195) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the unit and indicates the intra / inter decision by, for example, a prediction mode flag. The encoder can also combine (105) several predictions, for example in MH mode or bidirectional prediction mode. For example, the prediction residual is calculated by subtracting (110) the prediction block from the original image block.

[0164] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145) to output a bitstream. The encoder may skip the transform and directly apply quantization to the untransformed residual signal. The encoder may bypass both transform and quantization, i.e., directly code the residual without applying the transform or quantization process.

[0165] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0166] Fig.16 2 shows a block diagram of a video decoder 200. In the decoder 200, a bitstream is decoded by decoder elements as described below. The video decoder 200 generally performs the same operations as described below. Fig.15 The encoding process described in 1 is a decoding process that is the inverse of the encoding process described in 1. The encoder 100 also typically performs video decoding as part of encoding the video data.

[0167] In particular, the input of the decoder comprises a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors and other decoding information. Picture partition information indicates how the picture is partitioned. The decoder can therefore divide (235) the picture according to the decoded picture partition information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined (255) with a prediction block to reconstruct the image block. The prediction block may be obtained from intra prediction (260) or motion compensated prediction (i.e., inter prediction) (275) or from a combination of several predictions (270) (e.g., in MH mode or bidirectional mode). Motion compensated prediction may further compensate in illumination (295). As with the encoder, the decoder may also include a module (290) configured to refine (e.g., using DMVR or BIO) the decoded motion vector. An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0168] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or performing an inverse remapping of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0169] Fig.17A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 1000 can be implemented as a device including various components described below, and is configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 1000 can be implemented in a single integrated circuit (IC), multiple ICs and / or discrete components individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed on multiple ICs and / or discrete components. In various embodiments, the system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more aspects described herein.

[0170] The system 1000 includes at least one processor 1010, which is configured to execute instructions loaded therein, for implementing various aspects described herein, for example. The processor 1010 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical disk drive. As a non-limiting example, the storage device 1040 may include an internal storage device, an attached storage device (including a removable storage device and a non-removable storage device) and / or a network accessible storage device.

[0171] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding module and the decoding module. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated into the processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0172] Program code to be loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of the various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of the decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0173] In some embodiments, memory within the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store, for example, an operating system for a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as working memory for MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard developed by Joint Video Team experts JVET).

[0174] As shown in block 1130, input to the elements of system 1000 may be provided through various input devices. Such input devices include, but are not limited to: (i) an RF portion that receives a radio frequency (RF) signal transmitted, for example, over the air by a broadcaster, (ii) a component (COMP) input terminal (or a set of component input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Fig.10 Other examples not shown include composite video.

[0175] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band limiting the frequency band to a narrower frequency band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its relevant input processing element receive the RF signal that sends by wired (for example, cable) medium, and carry out the frequency selection to the desired frequency band by filtering, down-conversion and filtering again.Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example insert amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0176] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Solomon error correction) may be implemented as needed, for example, in a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed in a separate interface IC or in processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on output devices.

[0177] The various components of system 1000 may be disposed in an integrated housing in which the various components may be interconnected and transmit data using a suitable connection arrangement (e.g., an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and a printed circuit board).

[0178] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0179] In various embodiments, data is streamed or otherwise provided to the system 1000 using a wireless network (e.g., a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)). The Wi-Fi signals of these embodiments are received through a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to an external network including the Internet to allow streaming applications and other cloud communications. Other embodiments use a set-top box that transmits data through an HDMI connection of an input box 1130 to provide streaming data to the system 1000. Still other embodiments use an RF connection of an input box 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use a wireless network other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0180] The system 1000 can provide output signals to various output devices (including a display 1100, a speaker 1110, and other peripheral devices 1120). The display 1100 of various embodiments includes one or more of the following: for example, a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone), or other devices. The display 1100 can also be integrated with other components (for example, as in a smart phone), or it can be separate (for example, an external monitor for a laptop computer). In various examples of various embodiments, the other peripheral devices 1120 include one or more of the following: a stand-alone digital video disk (or digital versatile disk) (DVR, for both), a disk player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functions based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0181] In various embodiments, control signals are transmitted between the system 1000 and the display 1100, speaker 1110, or other peripheral device 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. The output devices may be communicatively coupled to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices may be connected to the system 1000 via a communication interface 1050 using a communication channel 1060. The display 1100 and speaker 1110 may be integrated into a single unit in an electronic device (e.g., a television) along with other components of the system 1000. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0182] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separate from one or more of the other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0183] These embodiments may be implemented by the processor 1010 or by computer software implemented by hardware or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, as a non-limiting example, one or more of the following: a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0184] Various implementations involve decoding. As used in this application, "decoding" may include, for example, all or part of the processing performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of the various implementations described in this application, such as motion and illumination compensation, motion refinement, and multi-prediction combining.

[0185] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0186] Various implementations involve encoding. In a manner similar to the discussion above about "decoding", "encoding" as used in this application can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy decoding. In various embodiments, such processes also or alternatively include processes performed by encoders of the various implementations described in this application, such as motion and illumination compensation, motion refinement, and multi-prediction combining.

[0187] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy decoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally refer to a broader encoding process will become clear based on the context of the specific description and is believed to be fully understood by those skilled in the art.

[0188] Note that the syntax elements (eg, IC-flags) as used herein are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0189] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0190] Various embodiments relate to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, usually given a constraint on computational complexity. The rate-distortion optimization is usually formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options, including all considered modes or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to save coding complexity, in particular, to calculate approximate distortion based on a prediction or prediction residual signal rather than a reconstructed signal. A mixture of these two methods can also be used, for example, by using approximate distortion only for some possible coding options and full distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete evaluation of both decoding costs and associated distortions.

[0191] The implementations and aspects described herein can be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). For example, the device can be implemented with appropriate hardware, software, and firmware. The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0192] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations mean that a particular feature, structure, characteristic, etc. described in conjunction with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations in various places in this application do not necessarily all refer to the same embodiment.

[0193] Additionally, the present application may refer to “determining” various information. Determining the information may include, for example, one or more of: estimating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0194] In addition, the present application may involve "accessing" various information. Accessing the information may include, for example, one or more of: receiving the information, retrieving the information (e.g., retrieving the information from a memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0195] Additionally, the present application may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving the information may include, for example, one or more of: accessing the information or retrieving the information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is generally involved in one way or another.

[0196] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to include selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to multiple items listed, as will be apparent to one of ordinary skill in this and related arts.

[0197] In addition, as used herein, the word "signal" especially refers to a corresponding decoder indicating something. For example, in some embodiments, the encoder signals a specific one of the decoding modes. Thus, in one embodiment, the same parameters are used on the encoder side and the decoder side. Therefore, for example, the encoder can send (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. On the contrary, if the decoder already has the specific parameter and other parameters, signaling can be used without sending (implicitly signaling) to simply allow the decoder to know and select the specific parameter. By avoiding the transmission of any actual function, bit saving is achieved in various embodiments. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to send information to the corresponding decoder with a signal. Although the foregoing relates to the verb form of the word "signal", the word "signal" can also be used as a noun in this article.

[0198] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0199] We have described a number of embodiments. Features of these embodiments may be provided individually or in any combination. In addition, across various claim categories and types, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination:

[0200] • A TV, set-top box, cell phone, tablet or other electronic device that performs motion and illumination compensation, motion refinement and adaptation of multiple prediction combining according to any of the described embodiments.

[0201] A TV, set-top box, cell phone, tablet or other electronic device that performs motion and illumination compensation, motion refinement and adaptation of multiple prediction combining according to any of the described embodiments, and displays (e.g., using a monitor, screen or other type of display) the resulting image.

[0202] A TV, set-top box, cellular phone, tablet or other electronic device that tunes (e.g., using a tuner) a channel to receive a signal comprising the encoded image and performs motion and illumination compensation, motion refinement and adaptation of multiple prediction combining according to any of the described embodiments.

[0203] A TV, set-top box, cellular phone, tablet or other electronic device that receives over the air (e.g., using an antenna) a signal comprising the encoded image and performs motion and illumination compensation, motion refinement and adaptation of multiple prediction combining according to any of the described embodiments.

[0204] According to a general aspect of at least one embodiment, a decoding method is disclosed, comprising:

[0205] – Decode the parameters of the current block;

[0206] - disabling the decoding tool for the current block if one or more of the following conditions are verified: a first decoding tool is used for the current block, and a given condition on the decoded parameter is verified;

[0207] - otherwise, enabling the decoding tool for the current block; and

[0208] - In response to the decoding tool, decoding the current block.

[0209] According to a general aspect of at least one embodiment, there is disclosed an encoding method comprising:

[0210] – Get the parameters of the current block;

[0211] - disabling the decoding tool for the current block if one or more of the following conditions are verified: a first decoding tool is used for the current block, and a given condition on the decoded parameter is verified;

[0212] - otherwise, enabling the decoding tool for the current block; and

[0213] - In response to the decoding tool, encoding the current block.

[0214] According to a general aspect of at least one embodiment, a decoding apparatus is disclosed, comprising one or more processors configured to execute:

[0215] – Decode the parameters of the current block;

[0216] - disabling the decoding tool for the current block if one or more of the following conditions are verified: a first decoding tool is used for the current block, and a given condition on the decoded parameter is verified;

[0217] - otherwise, enabling the decoding tool for the current block; and

[0218] - In response to the decoding tool, decoding the current block.

[0219] According to a general aspect of at least one embodiment, a coding apparatus is disclosed that includes one or more processors configured to perform:

[0220] – Get the parameters of the current block;

[0221] - disabling the decoding tool for the current block if one or more of the following conditions are verified: a first decoding tool is used for the current block, and a given condition on the decoded parameter is verified;

[0222] - otherwise, enabling the decoding tool for the current block; and

[0223] - In response to the decoding tool, encoding the current block.

[0224] In one embodiment, disabling coding tools for the current block if one or more of the following conditions are verified comprises disabling inter bi-directional prediction if a multiple hypothesis coding mode is used for the current block.

[0225] In one embodiment, disabling coding tools for the current block if one or more of the following conditions are verified comprises disabling local illumination compensation if a multiple hypothesis coding mode is used for the current block.

[0226] In one embodiment, disabling the coding tools for the current block if one or more of the following conditions are verified comprises: disabling triangle partitioning if a multiple hypothesis coding mode is used for the current block.

[0227] In one embodiment, disabling coding tools for the current block if one or more of the following conditions are verified includes disabling local illumination compensation or OBMC for one or more of small blocks and chroma components.

[0228] In one embodiment, disabling coding tools for the current block if one or more of the following conditions are verified comprises disabling local illumination compensation if the generalized bi-directional prediction weight is not equal to the default weight.

[0229] In one embodiment, disabling the decoding tools for the current block when one or more of the following conditions are verified includes: disabling generalized bidirectional prediction (or setting the generalized bidirectional prediction weight to a default weight) and not decoding the generalized bidirectional prediction index when local illumination compensation is enabled for the current block.

[0230] In one embodiment, disabling decoding tools for the current block when one or more of the following conditions are verified includes: disabling local illumination compensation when neighboring samples of the current block used by local illumination compensation belong to blocks using generalized bidirectional prediction weights that are not equal to default weights.

[0231] In one embodiment, disabling the coding tools of the current block if one or more of the following conditions are verified includes disabling overlapped block motion compensation if local illumination compensation is used.

[0232] In one embodiment, disabling the decoding tools for the current block when one or more of the following conditions are verified includes: when overlapping block motion compensation is used, disabling local illumination compensation and / or generalized bidirectional prediction when the weight is not equal to the default weight in the overlapping block motion compensation process.

Claims

1. A decoding method, comprising: -decode the parameters of the current block; - disabling the first decoding tool for the current block if one or more of the following conditions are verified: the second decoding tool is used for the current block and a given condition on a decoding parameter is verified; - otherwise, enabling the first decoding tool for the current block; as well as - If the first decoding tool is enabled, decoding the current block using the first decoding tool.

2. The method of claim 1 , wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: Disable bidirectional optical flow in cases where the generalized bidirectional prediction weights are not equal to the default weights.

3. The method of claim 1 , wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: In case bidirectional optical flow is used for the block, decoder side motion vector refinement is disabled.

4. The method of claim 1 , wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: In case a multiple hypothesis coding mode combining inter and intra prediction is used for the current block, triangle partitioning is disabled.

5. The method of claim 1 , wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: In case the block size is below a predefined value, generalized bi-directional prediction is disabled or the generalized bi-directional prediction weights are set to default weights.

6. A coding method comprising: - Get the parameters of the current block; - disabling the first decoding tool for the current block if one or more of the following conditions are verified: the second decoding tool is used for the current block and the given condition on the parameter is verified; - otherwise, enabling the first decoding tool for the current block; as well as - If the first decoding tool is enabled, encoding the current block using the first decoding tool.

7. The method of claim 6, wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: Disable bidirectional optical flow in cases where the generalized bidirectional prediction weights are not equal to the default weights.

8. The method of claim 6, wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: In case bidirectional optical flow is used for the block, decoder side motion vector refinement is disabled.

9. The method of claim 6, wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: In case a multiple hypothesis coding mode combining inter and intra prediction is used for the current block, triangle partitioning is disabled.

10. The method of claim 6, wherein disabling a first decoding tool for the current block if one or more of the following conditions are verified comprises: In case the block size is below a predefined value, generalized bi-directional prediction is disabled or the generalized bi-directional prediction weights are set to default weights.

11. A decoding device, comprising one or more processors, wherein the one or more processors are configured to execute: -decode the parameters of the current block; - disabling the first decoding tool for the current block if one or more of the following conditions are verified: the second decoding tool is used for the current block and a given condition on the decoded parameter is verified; - otherwise, enabling the first decoding tool for the current block; as well as - If the first decoding tool is enabled, decoding the current block using the first decoding tool.

12. The decoding device of claim 11, wherein disabling the first decoding tool for the current block if one or more of the following conditions are verified comprises: Disable bidirectional optical flow in cases where the generalized bidirectional prediction weights are not equal to the default weights.

13. The decoding device of claim 11 , wherein disabling the first decoding tool for the current block if one or more of the following conditions are verified comprises: In case bidirectional optical flow is used for the block, decoder side motion vector refinement is disabled.

14. The decoding device of claim 11, wherein disabling the first decoding tool of the current block if one or more of the following conditions are verified comprises: In case a multiple hypothesis coding mode combining inter and intra prediction is used for the current block, triangle partitioning is disabled.

15. The decoding device of claim 11, wherein disabling the first decoding tool for the current block if one or more of the following conditions are verified comprises: In case the block size is below a predefined value, generalized bi-directional prediction is disabled or the generalized bi-directional prediction weights are set to default weights.

16. An encoding device comprising one or more processors configured to: - Get the parameters of the current block; - disabling the first decoding tool for the current block if one or more of the following conditions are verified: the second decoding tool is used for the current block and the given condition on the parameter is verified; - otherwise, enabling the first decoding tool for the current block; as well as - in case the first decoding tool is enabled, using the first decoding tool to encode the current block.

17. The encoding device of claim 16, wherein disabling the first decoding tool for the current block if one or more of the following conditions are verified comprises: Disable bidirectional optical flow in cases where the generalized bidirectional prediction weights are not equal to the default weights.

18. The encoding device of claim 16, wherein disabling the first decoding tool of the current block if one or more of the following conditions are verified comprises: In case bidirectional optical flow is used for the block, decoder side motion vector refinement is disabled.

19. The encoding device of claim 16, wherein disabling the first decoding tool for the current block if one or more of the following conditions are verified comprises: In case a multiple hypothesis coding mode combining inter and intra prediction is used for the current block, triangle partitioning is disabled.

20. The encoding device of claim 16, wherein disabling the first decoding tool for the current block if one or more of the following conditions are verified comprises: In case the block size is below a predefined value, generalized bi-directional prediction is disabled or the generalized bi-directional prediction weights are set to default weights.

21. A computer program comprising program code instructions for implementing the method according to any one of claims 1 or 6.

22. Information storage means for storing program code instructions for implementing the method according to any one of claims 1 or 6.

Images