Methods and devices for picture encoding and decoding

Abstract

This invention provides a method and device for picture encoding and decoding. [Solution] The decoding method comprises the steps of: decoding parameters for the current block; disabling the first encoding tool for the current block in response to the verification of one or more conditions, such as the use of a second encoding tool for the current block and the verification of given conditions for the parameters; otherwise, enabling the first encoding tool for the current block; and, if the first encoding tool is enabled, decoding the current block using the first encoding tool.

Description

[Technical Field]

[0001] At least one of these embodiments relates in general to a method and device for picture encoding and decoding, and more particularly to a method and device for picture encoding and decoding using various encoding tools, such as motion compensation, local illumination compensation, generalized bi-prediction, and post-reconstruction filters. [Background technology]

[0002] To achieve high compression efficiency, image and video encoding schemes typically leverage spatial and temporal redundancy in video content using prediction and transformation. Generally, intra or inter-prediction is used to leverage intra or inter-frame correlation, and the difference between the original image block and the predicted image block, often expressed as prediction error, prediction residual, or predictor, is transformed, quantized, and entropy-encoded. During encoding, the original image block is typically divided into sub-blocks, possibly using quadtree partitioning. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to prediction, transformation, quantization, and entropy coding. [Overview of the Initiative]

[0003] A method for decoding video data is given according to a general aspect of at least one of its embodiments, which includes: - Decode parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the decoded parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Decode the aforementioned current block in response to the encoding tool.

[0004] A method for encoding video data is given according to a general aspect of at least one of its embodiments, which includes: -Retrieving parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the retrieved parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Encode the aforementioned current block in response to the encoding tool.

[0005] A decoding device is provided according to a general aspect of at least one embodiment, and includes one or more processors configured to do the following: - Decode parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the decoded parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Decode the aforementioned current block in response to the encoding tool.

[0006] According to a general aspect of at least one embodiment, a method for encoding video data is provided and includes the following: - obtaining parameters for a current block; - disabling an encoding tool for the current block above if one or more of the following conditions are verified: the first encoding tool is used for the current block above and the conditions given to the obtained parameters are verified; - otherwise enabling the encoding tool above for the current block above, and - encoding the current block above in response to the encoding tool.

[0007] According to a general aspect of at least one embodiment, an encoding apparatus is provided and includes one or more processors configured to perform the following: - obtaining parameters for a current block; - disabling an encoding tool for the current block above if one or more of the following conditions are verified: the first encoding tool is used for the current block above and the conditions given to the obtained parameters are verified; - otherwise enabling the encoding tool above for the current block above, and - encoding the current block above in response to the encoding tool.

[0008] According to another general aspect of at least one embodiment, a bitstream is formatted to include a signal generated according to the encoding method described above.

[0009] Further, one or more of the present embodiments also provide a computer-readable recording medium storing instructions for encoding or decoding video data according to at least a portion of any of the methods described above.

[0010] Furthermore, one or more aspects also provide a computer-readable recording medium storing a bitstream generated according to the encoding method described above. Further, one or more aspects also provide a method and an apparatus for transmitting or receiving a bitstream generated according to the encoding method described above. Further, one or more aspects also provide a computer program product including instructions for performing at least a part of any of the methods described above.

Effect of the Invention

[0011] Provide a method and a device for picture encoding and decoding.

Brief Description of the Drawings

[0012] [Figure 1a] Draw a general decoding pipeline. [Figure 1b] Draw another general decoding pipeline. [Figure 1c] Draw another general decoding pipeline. [Figure 1d] Draw a general decoding pipeline according to one aspect. [Figure 2] Illustrate the principle of parameter derivation for local illumination compensation. [Figure 3] Illustrate the principle of overlapping block motion compensation. [Figure 4] Illustrate the principle of multiple hypothesis prediction in the case of a combination of inter and intra modes. [Figure 5] Illustrate non-rectangular partitioning of blocks and related OBMC diagonal weighting. [Figure 6] Draw a flowchart of a decoding method according to one aspect. [Figure 7] Draw a flowchart of an encoding method according to one aspect. [Figure 8] A flowchart of the decoding process in various forms is drawn. [Figure 9] Draw a flowchart for the prediction process using multiple hypotheses. [Figure 10] A flowchart is drawn for the prediction process of multiple hypotheses related to one aspect. [Figure 11] Draw a flowchart for the motion compensation prediction process with local illumination compensation. [Figure 12] A flowchart is drawn for the prediction process of motion compensation with local illumination compensation according to one embodiment. [Figure 13] This diagram illustrates the overall decoding pipeline with local illumination compensation calculated prior to decoder side motion vector refinement according to one embodiment. [Figure 14] This illustrates the principle of multiple hypotheses. [Figure 15] An example block diagram of a video encoder relating to the embodiment is shown. [Figure 16] An example block diagram of a video decoder relating to the configuration is shown. [Figure 17] Block diagrams illustrating examples of systems that implement various aspects and configurations are provided. [Modes for carrying out the invention]

[0013] In HEVC encoding, a picture is divided into square CTUs of a configurable size, typically 64x64. The size can be 128x128 or 256x256 in other video encoding standards. A CTU is the root of a quadtree division into four equal-sized square coding units (CUs), i.e., half the size of the parent block in both width and height. A quadtree is a tree where a parent node can be divided into four child nodes, each of which can become a parent node for another four child nodes. In HEVC, a coding block (CB) contains a sample of one color component corresponding to one CU. It is divided into one or more prediction blocks (PBs), forming the root of a quadtree division into transform blocks (TBs). Corresponding to coding blocks, prediction blocks, and transformation blocks, a coding unit (CU) includes a tree structure of prediction units (PUs) and transformation units (TUs), where the PU contains prediction information for all color components, and the TU contains a residual coding syntax structure for each color component. The sizes of the luminance components CB, PB, and TB are applied to the corresponding CU, PU, ​​and TU.

[0014] In more recent encoding systems, the CTU is the root of the coding tree partition into coding units (CUs). A coding tree is a tree where a parent node (usually corresponding to a block) can be divided into child nodes (e.g., two, three, or four child nodes), each of which can become a parent node for further division into other child nodes. In addition to the quadtree split mode, newer split modes are defined that increase the total number of possible split modes (symmetric split mode in binary trees, asymmetric split mode in binary trees, and split mode in ternary trees). A coding tree has only one root node, e.g., the CTU. The leaves of a coding tree are the terminal nodes of the tree. Each node in a coding tree represents a block that can be further divided into smaller blocks, also called subblocks. Once the partition of the CTU into CUs is decided, the CU corresponding to the leaf of the coding tree is encoded. The division of a CTU into CUs, and the encoding parameters used to encode each CU (corresponding to a leaf in the encoding tree), may be determined in the encoder by a rate-distortion optimization procedure.

[0015] In this application, the terms “block” or “picture block” may be used to refer to any one of CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, the terms “block” or “picture block” may be used more generally to refer to a sequence of samples of a very large size, in addition to referring to macroblocks, divisions, and subblocks as defined in H.264 / AVC or other video encoding standards.

[0016] In this application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image,” “picture,” and “frame” may be used interchangeably. Typically, though not always, the term “reconstructed” is used on the encoder side, while “decoded” is used on the decoder side.

[0017] In this application, the terms “enabled” and “used” may be used interchangeably when they are applied to an encoding tool at the block level. Similarly, the terms “deactivated” and “not used” may be used interchangeably when they are applied to an encoding tool at the block level.

[0018] In this application, the terms “encoding tool” and “encoding process” may be used interchangeably.

[0019] In modern encoding systems, for example, in the reference software for Versatile Video Coding (VVC), known as VTM, the decoding pipeline includes several cascaded tools. The decoding pipeline for VVC is illustrated in Figure 1a. The decoding process is performed at the level of coding units (CUs). A CU contains C square or rectangular blocks (for example, C=3 for YUV or RGB chroma formats). In the 4:2:0 format, two chroma blocks are downscaled with respect to the lumen block by two ratios in both the horizontal and vertical directions.

[0020] The "entropy decoding" process parses the bitstream and decodes syntax elements such as the encoding mode (e.g., inter / intra encoding mode) and associated parameters (e.g., intra-direction, inter-motion vector MV, and reference index).

[0021] Motion compensation (MC) interpolates one or more rectangular blocks (in the case of bi-prediction) (same size as the current CU) samples in the reference picture (same size as the current CU) using an MC filter, with displacements equal to MV(mvX,mvY) in the reference picture in correlation with the position of the current block.

[0022] Local illumination compensation (LIC) aims to compensate for possible illumination changes between the predicted block and the reference block used in the motion-compensated (MC) time prediction. Typically, this is done using a linear IC model (a,b). S IC =a*S ref +b (formula 1) However, S ref This is a motion-compensated reference sample value, S IC These are the predicted sample values. The LIC parameters (a,b) are calculated for each block where LIC is enabled.

[0023] There are several ways to calculate the LIC parameters (a,b). In one method, the decoder calculates the LIC parameters based on several reconstructed picture samples, for example, a reconstructed sample "y" localized to the left and / or above the current block, and a local reference picture sample "x" to the left and / or above the motion-compensated block (Figure 2a). In another method, the LIC parameters are calculated based on a reconstructed sample "y" localized to the left and / or above the current block, and the corresponding predicted sample "x" used to reconstruct "y" (Figure 2b). For the two methods described above, the LIC parameters can be calculated and stored after the reconstruction of each block. The LIC parameters (a,b) minimize the difference between sample "y" and "a.x+b" (for example, by least squares).

[0024]

number

[0025] The two methods described above will hereafter be referred to as "conventional LIC" in this specification. Since LIC uses a reconfigured sample of a neighboring CU, it inserts dependencies with the neighboring CU. As a result, the current CU must wait for the neighboring block to be fully reconfigured before it is possible to calculate the LIC parameters for the current block and therefore the current block can be reconstructed (see "feedback rec. sample" in Figure 1a).

[0026] In the case of bidirectional prediction, the LIC variant (called "bi-dir-IC") derives the IC parameters (a0,b0) and (a1,b1) directly from the two motion-compensated blocks (block-0 and block-1) used in bidirectional prediction. This involves estimating the illumination change between the two reference pictures to derive the IC parameters (a0,b0) and (a1,b1) for the current block. For example, one calculates the IC parameters (a,b) for the LIC between motion-compensated block-1 and motion-compensated block-0. Another derives (a0,b0) and (a1,b1) from the values ​​(a,b) and the POC (picture order count) of the current picture, reference picture 0, and reference picture 1. The values ​​a0,a1,b0,b1 are assumed to be linear functions of the temporal distance between the current frame and the reference pictures. The advantage of the variant I just described is that it does not impose any dependencies on nearby CUs.

[0027] In the case of bidirectional prediction, BIO (Bi-directional Optical Flow) aims to refine the motion for each sample, assuming a linear displacement between two reference pictures, using Hermitian interpolation of optical flow. Therefore, the new term is different from conventional bidirectional prediction.

[0028]

number

[0029] It will be added.

[0030]

number

[0031] however,

[0032]

number

[0033] These are motion-compensated predictions (block-0 and block-1),

[0034]

number

[0035] and

[0036]

number

[0037] The horizontal and vertical slopes are (k={0,1}) and (v x ,v y ) is the motion vector between block-1 and block-0, and σ is the time difference (POC1-POC0).

[0038] In bidirectional prediction, DMVR (Decoder Side Motion Vector Refinement) aims to refine the motion by testing several displacements around the decoded MV value (MVinitial). A set of several candidate MV pairs (MVtested(L0), MVtested(L1)) are tested. As in BIO, the tested MV values ​​are symmetric around the initial MV value, i.e., (MVtested(L0) - MVinitial(L0) = MVtested(L1) - MVinitial(L1)), where MVtested(L0) is the motion vector tested for list L0 and MVtested(L1) is the motion vector tested for list L1. The selected MV pair is, for example, the pair that minimizes the difference MC(ref0,MVtested(L0))-MC(ref1,MVtested(L1)) using MRSAD (Mean-Removed Sum of Absolute Difference).

[0039] In bidirectional prediction, two prediction blocks are calculated and averaged together (also known as bi-prediction averaging). In generalized bidirectional prediction (GBI), they are combined with a weighted sum as follows: P GBI =(((1<<gs)-gw1).P0+gw1.P1)> >gs (Equation 4) However, P0 and P1 are motion-compensated predictions (block-0 and block-1).

[0040] In VTM, the weights used are encoded for each CU using the "gbi-index," which can take the values ​​listed in Table 1.

[0041] [Table 1]

[0042] Hereinafter in this specification, the weight pair {1 / 2; -1 / 2} will be referred to as the default weight.

[0043] In FIG. 1a, the LIC process is applied before the bidirectional prediction average in the prediction with motion compensation in each single direction. In FIG. 1b, the LIC process (720) is applied after the bidirectional prediction average (710) depicted.

[0044] According to the overlap block motion compensation (OBMC) tool (hereinafter referred to as the "OBMC process"), the motion of adjacent blocks is gradually weighted using the current block prediction P N (N = A or L) to extend the prediction (FIG. 3). For example, if P C is used to gradually weight, for example, P A (x, y) represents the above-mentioned prediction samples (x = {0, ··· W - 1}, y = {0, ··· H A - 1}) for the above-mentioned blocks. The samples described above are extended by an additional B rows at the bottom and calculated with the same process (MC) and the same motion information as the other prediction samples of P A (x, y) (x = {0, ··· W - 1}, y = (H A , ··· H A , ··· H A + B - 1). If P C (x, y) represents the current prediction P OBMC samples (x = {0, ··· W - 1}, y = {0, ··· H C - 1}), then the prediction for the B rows above the samples of the current block is finally calculated as follows. P OBMC (x, y)=w N (y).P A (x, y + B)+w C (y).P A (x, y) (Equation 5) The same process is applied to the prediction samples of the left B columns based on the P L (x, y) samples built from the left block data.

[0045] If a neighboring block N uses LIC and / or BIO for its prediction, then an additional P N Samples (hereinafter referred to as "OBMC processing" in this specification) may also be obtained by applying LIC and BIO.

[0046] For example, a typical stepwise weight from the border towards the center of the current block is P N and P C For each of them, w C (y) = {1 / 4, 1 / 8, 1 / 16, 1 / 32} and w N (y) = {3 / 4, 7 / 8, 15 / 16, 31 / 32}.

[0047] The general concept of multi-hypotheses, as illustrated in Figure 4, involves combinations of inter-predictions performed in merge mode with intra-prediction modes or with other inter-modes (e.g., AMVP, skip, and merge for single prediction). In merge mode, a list of merge candidates (reference index, movement value) is built, and a merge index is signaled to identify one candidate. Next, in merge mode, the block's movement information is derived from the candidate identified by the merge index and used for a movement-compensated prediction. The final prediction is a weighted average of the merge-indexed prediction and the prediction generated by the intra-prediction mode, but with different weights applied according to the combination. The intra-prediction mode is signaled in a bitstream. In a particular aspect, the intra-prediction mode is selected from a subset of all classical prediction modes (e.g., a subset size equal to 4). The current block is divided into four equal-area regions. The weights gradually decrease as the region moves away from the intra-reference sample. Each weight set, denoted as (w_intra,w_interi), where i ranges from 1 to 4, and represented as (w_intra1,w_inter1)=(6,2), (w_intra2,w_inter2)=(5,3), (w_intra3,w_inter3)=(3,5), and (w_intra4,w_inter4)=(2,6), will be applied to the intra-vertical predictions for the corresponding regions, as depicted in the examples in Figures 4 and 14. The exact same weights are applied to all CUs when DC or planar mode is selected, or when the width or height of a CU is less than 4.

[0048] Geometric (e.g., triangular) modes allow for greater flexibility in dividing the picture into blocks before encoding. An example of the above division is shown at the top of Figure 5. Additional encoding performance can be achieved by weighting two predictors on the frontier (gray area in Figure 5), as illustrated at the bottom of Figure 5. The weighting process described above can be cascaded with other weighting methods, such as GBI or LIC, which can greatly increase implementation complexity.

[0049] A “post-reconstruction” filter (730) allows filtering out the reconstructed sample of the current CU in CU loop reconstruction, and the filtered sample may be used for intra-prediction or LIC parameter derivation for subsequent CUs in the same sample (or same slice or same tile). However, “other post-filters” (740) (e.g., deblocking filters, SAO filters, or ALF filters) filter out the reconstructed sample after the entire picture (or slice or tile) has been reconstructed. Examples of "post-reconstruction" filters include bilateral filters (J. Strom et al., "CE14: Reduced latency, LUT-free bilateral filter", document JVET-L0172, 12th Conference, Macau, CN, October 3-12, 2018), Hadamard filters (S. Ikonen et al., "Non-CE: Hadamard transform domain filter", document JVET-M0468, 13th Conference, Marrakech, MA, January 9-18, 2019), or diffusion filters (J. Rash et al., "CE10: Uniform Directional Diffusion Filters For Video Coding", document JVET-M0042, 13th Conference, Marrakech, MA, January 9-18, 2019).

[0050] In one variant, the "post-reconstruction" filter is applied only to CUs encoded in intermode, as depicted in Figure 1c, and not to CUs encoded intra, or to CUs that combine intra and interprediction. In another variant, the "post-reconstruction" filter is applied only to CUs encoded intra, and not to intermode. The application of the "post-reconstruction" filter may be conditional on other parameters, such as the current CU size, QP value, or component type (for example, the "post-reconstruction" filter is not applied to saturation components depicted by the saturation box in the dash in 730).

[0051] New video codecs implement an increasing number of encoding tools that can be cascaded to improve encoding efficiency. In some cases, cascading tools of the same nature can have a counterproductive effect, namely reducing encoding efficiency, and furthermore, as mentioned above, can place a heavy burden on the decoding pipeline.

[0052] At least one aspect intends to assist in the implementation of video codecs on various platforms by harmonizing the interactions between different encoding tools and ensuring reasonable pipeline complexity. At least one aspect intends to apply certain rules and limitations without degrading the overall codec performance. At least one approach reduces both the pipeline size (number of processing stages) and the inserted pipeline dependencies by using several tools in "inter" coding, for example, by imposing restrictions on mutual co-activation.

[0053] To reduce pipeline dependency, the "inter" decode pipeline will be modified.

[0054] Figure 6 shows a flowchart of a decoding method according to one embodiment.

[0055] The parameters are decoded for the current block (S100). The encoding tool for the current block (e.g., LIC) is disabled if one or more of the following conditions are met: the first encoding tool is used for the aforementioned current block (e.g., GBI is used with weights other than the default weights), and the conditions given to the parameter being decoded (block size is less than the value) are verified (S110). The encoding tool is enabled for the aforementioned current block otherwise. The current block is decoded, taking into account whether the encoding tool is enabled (S120). If the encoding tool is enabled, the current block is decoded using the encoding tool. Otherwise, the current block is decoded without using the encoding tool.

[0056] Figure 7 shows a flowchart of an encoding method according to one embodiment. Parameters are obtained for the current block (S200). The encoding tool for the current block (e.g., LIC) is disabled if one or more of the following conditions are met: the first encoding tool is used for the aforementioned current block (e.g., GBI is used with weights other than the default weights), and the conditions given to the decoded parameters (block size is less than the value) are verified (S110). The encoding tool is enabled for the aforementioned current block otherwise. The current block is encoded, taking into account whether the encoding tool is enabled (S220). If the encoding tool is enabled, the current block is encoded using the encoding tool. Otherwise, the current block is encoded without using the encoding tool.

[0057] The disabling of an encoding tool is conditional on certain characteristics of the current encoding unit (CU) and / or whether another encoding tool "X" illustrated by Figures 8A-8C is used (406, 408).

[0058] In Figure 8A, the CU parameter is decoded (400). When the conditions for the CU parameter are verified (410), for example, if the CU size is less than the value, the encoding tool / process "Y" is disabled (420).

[0059] In Figure 8B, the CU parameters are decoded (400). If the first encoding tool "X" is used for the current CU (406), then the first encoding tool is applied (408). If further conditions regarding the CU parameters are checked (410), for example, if the CU size is less than the value, the second encoding tool / process is disabled. Otherwise, the second encoding tool is used (420).

[0060] In the variant, if the first encoding tool "X" is used for the current CU (406), then the first encoding tool is applied (408), and the second encoding tool "Y" is disabled. In the variant described above, the conditions for CU are not checked.

[0061] In Figure 8C, the CU parameters are decoded (400). If both the first encoding tool "X" is used for the current CU (406) and the conditions for the CU parameters are verified (410), then the second encoding tool "Y" is disabled. Otherwise, the second encoding tool is used (420). The dotted line indicates that the application of encoding tool "X" is conditional on 406.

[0062] The variant is built from the expression of the "condition (CU)" and the processing order of tools "X" and "Y" (Figure 8-B or C). The "condition (CU)" is derived from the previously decoded data (400) and can be a combination of the following rules: - Disable tool "X" for small blocks (CU size less than the value). - Disable tool "X" for saturation (where the CU component is saturation). - If the current slice has a TiD (Temporal Identifier) ​​that is above a certain value, disable tool "X". -Disable tool "X" if you need information (e.g., a reconfigured sample) from another reconfigured block located in a different CTU or outside the current VPDU (Virtual Pipeline Data Unit).

[0063] Reducing pipeline dependencies in multiple hypothesis modes Figure 9 illustrates the inter-prediction pipeline processing of VVC software for MH when the input bit depth or internal bit depth is equal to 10 bits, and when bidirectional prediction, BIO, and GBI are also used. First, motion compensation is applied to build the predictions (500) of two reference blocks. The results are stored (510) in N bits for N>10 to maintain greater accuracy in further processing (e.g., N=14 bits), namely BIO (520) and GBI weighting (530). Next, the sample values ​​are stored (e.g., right-shifted) (540) and clipped (550) at the internal bit depth (e.g., 10 bits) so that the subsequent MH process (560) is performed by the internal bit depth (e.g., 10 bits).

[0064] Multiple Hypothesis (MH), as shown in Figure 9, increases the number of processing steps in the inter-pipeline by using additional intra- or inter-predictive calculations and combining them with other predictions. One limitation may mitigate the pipeline by one or more of the following:

[0065] In one aspect, LIC is disabled for one or all intermodes when MH is used. What has been stated means that if the current intermode is "merge" and LIC is disabled because MH is used, then the IC flag (IC-flag) is inferred to be zero, not merge. The IC flag is a parameter that indicates whether to use LIC for a block.

[0066] In one embodiment, inter-bidirectional prediction is disabled when the MH shown in Figure 10 is used. Furthermore, BIO is also disabled (depending on the design) as it is only applicable to bidirectional prediction.

[0067] In one embodiment, triangle mode is disabled when MH is used. Indeed, there is a weighting process for the two triangle predictions already shown in Figure 5.

[0068] Furthermore, one way to improve the MH combination stage is to add intra-prediction with increasing bit depth accuracy to inter-prediction, and before the clipping operation. In Figure 10, the MH process is performed in N bits.

[0069] Reducing pipeline dependency on LIC: Figure 11 illustrates the inter-prediction pipeline processing of the VVC software for LIC when the input bit depth or internal bit depth is equal to 10 bits and bidirectional prediction is used. First, motion compensation is applied to build predictions (600) for two reference blocks. The results are stored (610) with the precision of the internal bit depth (e.g., 10 bits). Next, LIC parameters are estimated or retrieved from previously stored data and illumination compensation is applied (670). This includes weighting plus an offset. Next, a right shift (640) and clipping (650) to the internal bit depth (10 bits) are applied. Then, the samples are converted (680) to N-bit precision before GBI weights are applied (630). P LIC =(a.P+b)>>shift LIC (Formula 6) However, (a;b) are LIC parameters, P is a motion-compensated block sample, shift LC P LIC This is a right shift because it is 10 bits. P LIC This is a prediction sample by LIC.

[0070] Next, the sampled value is stored (e.g., right-shifted) (640) and clipped (650) within an internal bit depth (e.g., 10 bits).

[0071] Essentially, the LIC inserts pipeline dependencies by other nearby reconfigured blocks, as shown in Figure 11, and adds a subsequent step of motion compensation to change the lighting. The dependencies described above can be reduced by using one or more of the following constraints / limitations.

[0072] In one embodiment, LIC is disabled for the saturation component and / or small blocks (CU size is less than a value).

[0073] In one embodiment, GBI is disabled for the saturation component and / or small blocks (CU size is less than the value) (GBI weights are set to default weights).

[0074] The principle of LIC is to weight the predicted samples obtained through motion compensation, so there are some similarities with GBI when the weights are not equal to the default weights.

[0075] In one mode, the LIC is disabled in bidirectional prediction if the GBI weights are not the default weights. Therefore, the IC flag is not encoded if the GBI index (GBI-index) is different from the default. In merge mode, if the GBI index is inherited, the IC flag is inferred to be false if the GBI weights are not the default. In variant mode, if the IC flag is inherited and its value is true, the GBI weights are inferred to be the default weights. In variant mode, if the GBI weights are not the default, one uses a bi-dir IC instead of the conventional LIC.

[0076] In one mode, the IC flag is encoded, and if the IC flag is true, then the GBI index is not encoded, and the GBI weights are inferred to be the default weights. In another mode, if the IC flag is inherited in merge mode and its value is true, then the GBI weights are inferred to be the default weights.

[0077] An example of the syntax is provided in Table 2.

[0078] [Table 2]

[0079] In one embodiment, if the GBI weight is not the default weight, the IC flag is encoded even if the current CU is encoded in "merge" mode.

[0080] In one aspect, the LIC is a reconstructed sample "rec N This is disabled if it belongs to one neighboring CU (called N) encoded with non-default GBI weights.

[0081] A virtual pipeline data unit (VPDU) is defined as a non-overlapping MxM luminance (L) / NxN saturation (C) unit in a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. Different stages process different VPDUs simultaneously. Typically, a VPDU is a 64x64 luminance / 32x32 saturation block. Three conditions must be verified: For each VPDU containing one or more CUs, the CUs are fully contained within the VPDU. For each CU containing one or more VPDUs, the VPDUs are fully contained within the CU. -CU will be processed in an order that does not leave VPDUs behind and does not return to them later.

[0082] Block reconfiguration dependencies introduced by LIC may be acceptable for data in cache memory, typically corresponding to one CTU or one VPDU, which is related to hardware and / or implementation memory cache constraints / limitations for decoding elementary CUs, and further related to the maximum sample area size that hardware can handle all-in-one.

[0083] In one embodiment, the LIC is a nearby block used to derive the LIC parameters. N The reconfigured sample "recN" is invalidated if it is outside the CTU or VPDU. In other words, LIC may be used, but only if the sample is reconfigured inside the current CTU or VPDU.

[0084] In one embodiment, when the encoding mode of the current block is "merge", the block in question has an IC flag equal to true (the IC parameter is shown) if the merge index indicates that the neighboring N is used for merging. N (if it is) and "rec N If the sample is outside the CTU or VPDU, then the IC flag is true for the current block, and one derives the current LIC parameter "rec N Instead of using the sample, build the prediction for the current block using IC N Reuse the LIC parameters (used by the neighboring block N).

[0085] In one embodiment, LIC is calculated with increasing precision (670), as shown in Figure 12, i.e., shift LIC (Equation 6) is set so that the sample accuracy is N bits after LIC processing and the sample is clipped only at the end of the prediction process.

[0086] Reducing pipeline dependencies on "post-reconstruction" filters: In one embodiment, the LIC process and the "post-reconstruction" filtering process are mutually exclusive with respect to each component (i.e., if LIC is enabled for one component, the "post-reconstruction" filter is disabled for that component), or they are mutually exclusive with respect to all components (if LIC is enabled for all components, the "post-reconstruction" filter is disabled for all components).

[0087] If "Post-Reconstruction" is not applied to one component (e.g., saturation), then, if LIC is invalid for the current CU (e.g., the LIC flag is false), then "Post-Reconstruction" may be applied only to other components (e.g., luminance).

[0088] For example, if LIC is enabled and / or applied to the current CU (e.g., the LIC flag is true, inferred, or encoded), post-reconstruction filtering is disabled for all components shown in Figure 1d.

[0089] In another example, if LIC processing can be enabled for each component (for example, one LIC flag inferred or encoded for luminance, and another LIC flag inferred or encoded for saturation), then the "post-reconstruction" filtering process will be disabled for one component if the LIC flag is enabled for the component mentioned above.

[0090] Reducing the pipeline for OBMC: OBMC may have lower encoding efficiency for small blocks, while the average amount of processing per sample increases significantly when there are many small blocks in the picture. One may reduce pipeline efficiency by one or more of the following limitations: In one embodiment, OBMC is disabled (i.e., not used) for small blocks (e.g., 4x4). In one embodiment, OBMC is not performed on saturation blocks. In one embodiment, OBMC is disabled (i.e., not used) if the current block uses LIC. In one embodiment, LIC is not used in OBMC processing. In one embodiment, BIO is not used in OBMC processing.

[0091] Reducing pipelines for BIO: In one embodiment, BIO is disabled if the GBI weights are not the default values. BIO and DMVR are two techniques with the same objective: to refine the MV. Therefore, BIO is invalidated when DMVR is used on the current block, and vice versa (mutually exclusive).

[0092] Reducing the pipeline for DMVR: In the current situation, LIC is disabled by DMVR. Indeed, enabling LIC, which involves calculating LIC parameters for each MV being tested, significantly increases the computational cost. It has been proposed to perform DMVR without LIC and allow LIC processing after the DMVR stage, as depicted in Figure 1. However, since DMVR does not include LIC, the accuracy of MV refinement is significantly reduced. To address the limitations mentioned above, one option is to perform MV refinement (before MV refinement by DMVR) initial Using this method, the LIC parameters are calculated without refining them during the DMVR process; that is, the LIC parameters are not calculated during or after the DMVR process, as shown in Figure 13.

[0093] Reducing the pipeline for deblocking filters: In one embodiment, the strength of DBF is reduced if OBMC is applied to an edge and / or if that edge belongs to a block having an enabled LIC or BIO.

[0094] This application describes various aspects, including tools, features, embodiments, models, and approaches. Many of these aspects are described in a specific manner, often in a way that may give the impression of limitation, at least in order to illustrate their individual characteristics. However, this is for the purpose of clarity in the description and does not limit the application or scope of these aspects. Indeed, all the different aspects can be combined and interchangeable to provide further aspects. Furthermore, embodiments can likewise be combined and interchangeable with aspects described in earlier applications.

[0095] The aspects described and envisioned in this application can be implemented in many different ways. Figures 15, 16, and 17 below provide some embodiments, but other embodiments are envisioned, and Figures 15, 16, and 17 do not limit the scope of implementation. Generally, at least one of the embodiments relates to video encoding and decoding, and generally, at least one other embodiment relates to transmitting the generated or encoded bitstream. The embodiments described above and the other embodiments can be implemented as computer-readable recording media storing methods, apparatus, and instructions for encoding or decoding video data according to any of the described methods, and / or computer-readable recording media storing the bitstream generated according to any of the described methods.

[0096] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a particular order of steps or actions is required for the inherent operation of the method, any particular order and / or use of steps and / or actions may be modified or combined.

[0097] The various methods and other aspects described herein can be used, for example, to modify the motion compensation modules (170, 275), motion refinement modules (190, 290), LIC modules (195, 295), and multiple predictions combination modules (105, 270) of the video encoder 100 and decoder 200, as shown in Figures 15 and 16. Furthermore, these aspects are not limited to VVC or HEVC, but can be applied to other standards and recommendations, whether existing or to be developed in the future, as well as extensions to any of the above standards and recommendations (including VVC and HEVC). Unless otherwise specified or particularly technically impeded, the aspects described herein can be used individually or in combination.

[0098] Various numerical values ​​are used in this application. Certain values ​​are for illustrative purposes only, and the aspects described are not limited to the specific values ​​mentioned herein.

[0099] Figure 15 illustrates encoder 100. While variations of encoder 100 are anticipated, encoder 100 is described below for clarity without explaining all expected variations.

[0100] Before encoding, a video sequence may undergo pre-encoding (101), which may involve, for example, applying color conversions to the input color picture (e.g., a conversion from RGB4:4:4 to YCbCr4:2:0) or remapping input picture components to make the signal distribution more resilient to compression (e.g., using histogram equalization on one of the color components). Metadata can be associated with pre-processing and linked to the bitstream.

[0101] In encoder 100, the picture is encoded by the encoder elements described below. The picture to be encoded is divided (102) and processed, for example, in units of CU. Each unit is encoded using either intra or intermode. When a unit is encoded in intramode, intra prediction (160) is performed. In intermode, motion estimation (175), possibly motion refinement (190), motion compensation (170), and possibly illumination compensation (195) are performed. Motion refinement may use DMVR or BIO. The encoder determines (105) either intramode or intermode to use to encode the unit, indicating the intra / inter determination, for example, by a prediction mode flag. Furthermore, the encoder may combine several predictions (105), for example, in MH mode or bidirectional prediction mode. The prediction residual is calculated, for example, by subtracting the predicted block from the original image block (110).

[0102] Next, the predicted residual is transformed (125) and quantized (130). The quantized transformed coefficients, likewise the motion vector and other syntax elements, are entropy coded (145) to output the bitstream. The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transformation and quantization, i.e., the residual is coded directly without applying either the transformation or quantization process.

[0103] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transformation coefficients are inversely quantized (140) and inversely transformed (150) to decode the prediction residuals. The decoded prediction residuals and the predicted blocks are combined (155) to reconstruct the image blocks. An in-loop filter (165) is applied to the reconstructed picture to perform deblocking / SAO (Sample Adaptive Offset) filtering, for example, to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0104] Figure 16 illustrates a block diagram of the video decoder 200. In the decoder 200, the bitstream is decoded by the decoder elements described below. Generally, the video decoder 200 performs a decoding path that is the reverse of the encoding path described in Figure 15. Furthermore, generally, the encoder 100 also performs video decoding as part of encoding the video data.

[0105] In particular, the decoder input includes a video bitstream that can be generated by the video encoder 100. First, the bitstream is entropy-decoded (230) to obtain transformation coefficients, motion vectors, and other encoded information. Picture segmentation information indicates how the picture is segmented. Therefore, the decoder may segment (235) the picture according to the decoded picture segmentation information. The transformation coefficients are inversely quantized (240) and inversely transformed (250) to decode the prediction residuals. The image blocks are reconstructed by combining (255) the decoded prediction residuals and the predicted blocks. The predicted blocks can be obtained from intra-predictions (260), motion-compensated predictions (i.e., inter-predictions) (275), or from a combination (270) of several predictions (e.g., in MH mode or in bidirectional mode). Furthermore, motion-compensated predictions may be compensated (295) by illumination. Like the encoder, the decoder also includes a module configured to refine (290) the decoded motion vector (for example, using DMVR or BIO). An in-loop filter (265) is applied to the image being reconstructed. The filtered image is stored in a reference picture buffer (280).

[0106] Furthermore, the decoded picture can go through post-decoding (285), for example, inverse color conversion (e.g., conversion from YCbCr4:2:0 to RGB4:4:4), or reverse remapping, which is the reverse of the remapping process performed in pre-encoding (101). Post-decoding can use metadata signaled in the bitstream derived in pre-encoding.

[0107] Figure 17 illustrates a block diagram of an example of a system in which various aspects and configurations are implemented. System 1000 can be materialized as a device including various components described below and configured to perform one or more aspects described in this document. Examples of the above device include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home electrical appliances, and servers. The elements of System 1000 can be materialized individually or in combination as a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one configuration, the processing and encoder / decoder elements of System 1000 are distributed across multiple ICs and / or individual components. In various configurations, System 1000 is communicated to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various forms, the system 1000 is configured to implement one or more aspects described in this document.

[0108] System 1000 includes, for example, at least one processor 1010 configured to execute instructions loaded to implement various aspects described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 1000 includes at least one memory 1020 (for example, a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040 which may include non-volatile memory and / or volatile memory, including, but not limited to, EEPROM (Electrically Erasable Programmable Read-Only Memory), ROM (Read-Only Memory), PROM (Programmable Read-Only Memory), RAM (Random Access Memory), DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), flash, magnetic disk drives, and / or optical disk drives. Storage device 1040 may include, in non-limiting examples, internal storage devices, attached storage devices (including detachable and non-detachable storage devices), and / or network-accessible storage devices.

[0109] System 1000 includes, for example, an encoder / decoder module 1030 configured to process data that provides encoded or decoded video, the encoder / decoder module 1030 which may include its own processor and memory. Encoder / decoder module 1030 represents a module(s) which may be included in a device that performs encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or it may be incorporated into the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0110] Program code loaded onto the processor 1010 or encoder / decoder 1030 performing the various aspects described herein can be stored in the storage device 1040 and subsequently loaded into memory 1020 for execution by the processor 1010. Depending on the manner in which they are performed, one or more processors 1010, memory 1020, storage device 1040, and encoder / decoder modules 1030 can store one or more various items during the execution of the processes described herein. The stored items may include, but are not limited to, input video, decoded video, or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic. In some embodiments, the internal memory of the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for the processing required during encoding or decoding. However, in other embodiments, external memory of the processing device (for example, the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of the functions described above. The external memory can be memory 1020 and / or storage device 1040, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one aspect, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as MPEG-2 (MPEG refers to the Moving Picture Experts Group, and MPEG-2 is further called ISO / IEC 13818, and 13818-1 is known as H.222, and 13818-2 is known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, JVET, a standard developed by the Joint Video Experts Team).

[0111] Inputs to the elements of system 1000 can be provided via various input devices, as shown in block 1130. These input devices include, but are not limited to, (i) an RF section for receiving, for example, RF (radio frequency) signals transmitted via radio waves by a broadcasting station, (ii) a COMP (Component) input terminal (or a set of COMP input terminals), (iii) a USB (Universal Serial Bus) input terminal, and / or (iv) an HDMI (High Definition Multimedia Interface) input terminal. Other examples not shown in Figure 10 include composite video.

[0112] In various embodiments, the input device of block 1130 associates with various input processing elements known in the art. For example, the RF portion can be associated with elements suitable for (i) selecting a desired frequency (also said to be selecting a signal or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting a single frequency band, which in some embodiments may be called a channel, to a narrower frequency band of the selected signal, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream for data packets. Various embodiments of the RF portion include one or more elements that perform the functions described above, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error collectors, and demultiplexers. The RF portion can include tuners that perform various functions, such as down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or to the baseband. In one set-top box configuration, the RF section and associated input processing elements receive, filter, down-convert, and filter again to a desired frequency band to perform frequency selection. Various configurations involve rearranging the order of the elements described above (and others), removing some of the elements mentioned, and / or adding other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various configurations, the RF section includes an antenna.

[0113] Additionally, USB and / or HDMI terminals may include their respective interface processors to connect system 1000 to other electronic devices via USB and / or HDMI. It is understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or, if necessary, in processor 1010. Similarly, aspects of USB or HDMI interface processing may be implemented in a separate input processing IC or, if necessary, in processor 1010. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, for example, processor 1010 and encoder / decoder 1030, which works in cooperation with memory and storage elements to process the data stream for submission to output devices as necessary.

[0114] Various elements of system 1000 can be provided within an integrated housing, where the various elements are interconnected and can transmit data between them using an internal bus known in the art, such as an I2C (Inter-IC) bus, wiring, and printed circuit boards.

[0115] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may, but is not limited to, include a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may, but is not limited to, include a modem or a network card, and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0116] Data is provided to system 1000 in various ways, for example, by streaming or otherwise using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in the aforementioned manner is received via a communication channel 1060 and a communication interface 1050, which are modified to accommodate Wi-Fi communication. Typically, the communication channel 1060 in the aforementioned manner is connected to an access point or router that provides access to an external network, including the Internet, to enable streaming applications and other over-the-top communications. In other aspects, the streamed data is provided to system 1000 using a set-top box that distributes the data via the HDMI connection of input block 1130. Still, in other aspects, the streamed data is provided to system 1000 using the RF connection of input block 1130. As shown above, various aspects provide data in ways other than streaming. Additionally, various aspects use wireless networks other than Wi-Fi, e.g., cellular networks or Bluetooth networks.

[0117] System 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. Various embodiments of the display 1100 include, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be for a television, tablet, laptop, cellular phone (mobile phone), or other device. Furthermore, the display 1100 can be integrated with other components (e.g., in a smartphone) or separate (e.g., with an external monitor for a laptop). In various embodiments, the other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functions based on the output of System 1000. For example, a disc player plays the role of playing the output of System 1000.

[0118] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral devices 1120 using signaling such as AV.Link, CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be connected to the system 1000 via dedicated connections through their respective interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to the system 1000 via communication interface 1050 using communication channel 1060. The display 1100 and speaker 1110 can be integrated into a single unit with other components of the system 1000, for example, in an electronic device such as a television. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (TCon) chip.

[0119] Alternatively, the display 1100 and speaker 1110 can be isolated from one or more other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments where the display 1100 and speaker 1110 are external components, the output signals can be submitted via a dedicated output connection, such as an HDMI port, a USB port, or a COMP output.

[0120] The embodiments can be implemented by computer software implemented by processor 1010, by hardware, or by a combination of hardware and software. In a non-limiting example, the embodiments can be implemented by one or more integrated circuits. Memory 1020 can be any type appropriate for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, for example, in a non-limiting example. Processor 1010 can be any type appropriate for the technical environment and can include, in a non-limiting example, one or more of microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures.

[0121] Various implementations include decoding. As used in this application, “decoding” can include, for example, all or part of processing performed on the receiving encoding sequence to produce a final output suitable for display. In various embodiments, the processing includes one or more of the processing typically performed by a decoder, e.g., entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, further or alternatively, the processing includes processing performed by the decoders of the various implementations described in this application, e.g., motion and illumination compensation, motion refinement, and multiple predictions combination.

[0122] As further examples, in one aspect, “decoding” refers only to “entropy decoding,” in another aspect, “decoding” refers only to differential decoding, and in yet another aspect, “decoding” refers to a combination of entropy decoding and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or to refer more broadly in general, it is believed that the decoding process will become clear from the context of a particular description and will be well understood by those skilled in the art.

[0123] Various implementations include encoding. In a manner similar to the above discussion of "decoding," "encoding" as used in this application may include, for example, all or part of the processing performed on the input video sequence to generate the encoded bitstream. In various embodiments, the processing includes one or more of the processing typically performed by an encoder, e.g., splitting, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, further or alternatively, the processing includes processing performed by the encoders of the various implementations described in this application, e.g., motion and illumination compensation, motion refinement, and multiple predictions combination.

[0124] As further examples, in one aspect, “encoding” refers only to “entropy decoding,” in another aspect, “encoding” refers only to differential decoding, and in yet another aspect, “encoding” refers to a combination of differential encoding and entropy encoding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to refer more broadly in general, it is believed that encoding processes will become clear from the context of a particular description and will be well understood by those skilled in the art. Note that the syntax elements used herein, such as IC flags, are descriptive terms. As mentioned above, they do not preclude the use of other syntax element names.

[0125] When a diagram is given as a flowchart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is given as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0126] Various aspects cite rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often given constraints on computational complexity. Typically, rate-distortion optimization is formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving rate-distortion optimization problems. For example, an approach may be based on extensive testing of all encoding options, including all considered mode or encoding parameter values, along with a full evaluation of the encoding cost and associated distortion of the signal being reconstructed after encoding and decoding. Furthermore, faster approaches may be used, in order to eliminate the complexity of encoding, with the calculation of approximated distortion based on the predicted or predicted residual signal rather than one being reconstructed. A mixture of the two approaches described above can also be used, for example, by using approximated distortion for only some of the possible encoding options and full distortion for the other encoding options. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ one of several optimization techniques, but optimization does not necessarily provide a complete evaluation of both the coding cost and the associated distortions.

[0127] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even when described only in the context of a single form of implementation (e.g., only as a method), the implementation of the described features can also be implemented in other forms (e.g., apparatus or program). Apparatus can be implemented, for example, in appropriate hardware, software, and firmware. Methods can be implemented, for example, in processors, which generally refer to processing devices, including computers, microprocessors, integrated circuits, or programmable logic devices. Furthermore, processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users.

[0128] References to “one aspect,” “aspect,” “one implementation,” or “implementation” mean, as with other variations, that certain features, structures, characteristics, etc., described in relation to an aspect are included in at least one aspect. Therefore, the appearance of the phrase “in one aspect,” “in one aspect,” “in one implementation,” or “in implementation,” which appears in various places throughout this application, does not necessarily refer to all of the same aspects, as with any other variation.

[0129] In addition, this application may use the term “determining” various parts of information. Determining information can include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory.

[0130] Furthermore, this application may refer to “accessing” various parts of information. Accessing information can include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, computing information, determining information, predicting information, or estimating information.

[0131] In addition, this application may use the term “receiving” various parts of information. Receiving is intended to be a broad term, as with respect to “accessing.” Receiving information can include, for example, accessing information or retrieving information (for example, from memory). Furthermore, “receiving” usually includes, in some way or another, actions such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0132] It should be understood that the use of any of the following " / ", "and / or", and "at least one of" is intended to include, for example, "A / B", "A and / or B", and "at least one of A and B", which include selecting only the first enumerated option (A), or only the second enumerated option (B), or both options (A and B). As further examples, in the case of "A, B, and / or C" and "at least one of A, B, and C", the above phrases are intended to include selecting only the first enumerated option (A), or only the second enumerated option (B), or only the third enumerated option (C), or only the first and second enumerated options (A and B), or only the first and third enumerated options (A and C), or the second and third enumerated options (B and C), or all three options (A, B, and C). What has been stated can be extended to the matters listed, as will be obvious to those skilled in the art and related businesses.

[0133] Furthermore, as used herein, the word “signaling” primarily refers to indicating something to a corresponding decoder. For example, in one aspect, an encoder signals one particular encoding mode. In this way, the same parameters are used on both the encoder and decoder sides. Thus, for example, an encoder can transmit certain parameters to a decoder (explicit signaling), and the decoder can use the same specific parameters. Conversely, if the decoder already has certain parameters, signaling can be used without transmission (implicit signaling), simply to allow the decoder to know and select those parameters. Bit saving is achieved in various ways by avoiding transmission in any real-world function. It is understood that signaling can be performed in various ways. For example, one or more syntax elements, flags, etc., are used in various ways to signal information to a corresponding decoder. The above concerns the verb form of the word "signaling," but the word "signaling" can also be used as a noun in this specification.

[0134] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be stored or transmitted. For example, the information can include instructions for performing a method or data generated by one of the implementations described. For example, a signal can be formatted to carry a bitstream in the manner described. For example, the above signal can be formatted as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. For example, formatting can include encoding a data stream and modulating a carrier wave with the encoded data stream. For example, the information carried by the signal can be analog or digital information. The signal can be transmitted over various separate wired or wireless links, as is known. The signal can be stored in a processor-readable medium.

[0135] We will describe many embodiments. The features of the embodiments described above can be provided individually or in any combination. Furthermore, embodiments can include one or more of the following features, devices, or aspects, individually or in any combination, across various claim categories and types. Televisions, set-top boxes, mobile phones, tablets, or other electronic devices that perform motion and lighting compensation, motion refinement, and multiple predictive combination fitting in any of the described embodiments. Televisions, set-top boxes, mobile phones, tablets, or other electronic devices that perform motion and lighting compensation, motion refinement, and multiple predictive combination fitting in any of the described embodiments, and display the resulting image (for example, using a monitor, screen, or other type of display). A television, set-top box, mobile phone, tablet, or other electronic device that tunes to a channel (for example, using a tuner) to receive a signal containing an encoded image and performs motion and lighting compensation, motion refinement, and multiplex predictive combination fitting in any of the described manner. A television, set-top box, mobile phone, tablet, or other electronic device that receives a signal via radio waves containing an encoded image (for example, using an antenna) and performs motion and lighting compensation, motion refinement, and multiplex predictive combination fitting in any of the described embodiments.

[0136] A decoding method is disclosed in accordance with the general aspects of at least one embodiment, which includes the following: - Decode parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the decoded parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Decode the aforementioned current block in response to the encoding tool.

[0137] An encoding method is disclosed according to a general aspect of at least one embodiment, which includes the following: -Retrieving parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the decoded parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Encode the aforementioned current block in response to the encoding tool.

[0138] A decoding device is disclosed that includes one or more processors configured to do the following, according to a general aspect of at least one embodiment: - Decode parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the decoded parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Decode the aforementioned current block in response to the encoding tool.

[0139] An encoding device is disclosed that includes one or more processors configured to do the following, according to a general aspect of at least one embodiment: -Retrieving parameters for the current block, -Disable the encoding tool for the current block if one or more of the following conditions are met: the first encoding tool is used for the current block, and the conditions given to the decoded parameters are verified. -Otherwise, enable the aforementioned encoding tool for the current block, and - Encode the aforementioned current block in response to the encoding tool.

[0140] In one embodiment, disabling the coding tool for the current block described above when one or more of the following conditions are met includes disabling inter-bidirectional prediction when a multiple hypothesis coding mode is used for the current block.

[0141] In one embodiment, disabling the coding tool for the current block described above when one or more of the following conditions are verified includes disabling local illumination compensation when a multiplexed hypothesis coding mode is used for the current block.

[0142] In one embodiment, disabling the coding tool for the current block described above when one or more of the following conditions are met includes disabling triangulation when the multiplexed hypothetical coding mode is used for the current block.

[0143] In one embodiment, disabling the encoding tool for the current block described above when one or more of the following conditions are verified includes disabling local illumination compensation or OBMC for one or more of the subblocks and saturation components.

[0144] In one embodiment, disabling the coding tool for the aforementioned current block when one or more of the following conditions are met includes disabling local illumination compensation when the generalized bi-prediction weights are not equal to the default weights.

[0145] In one embodiment, disabling the coding tool for the aforementioned current block when one or more of the following conditions are met includes disabling generalized bi-prediction (or setting the weights of generalized bi-prediction to default weights) and not coding the generalized bi-prediction index when local illumination compensation is enabled for the aforementioned current block.

[0146] In one embodiment, disabling the coding tool for the aforementioned current block when one or more of the following conditions are verified includes disabling local illumination compensation if the adjacent samples of the aforementioned current block used by local illumination compensation belong to a block that uses generalized bidirectional prediction weights that are not equal to the default weights.

[0147] In one embodiment, disabling the encoding tool for the current block described above when one or more of the following conditions are verified includes disabling overlap block motion compensation when local illumination compensation is used.

[0148] In one embodiment, disabling the coding tool for the aforementioned current block when one or more of the following conditions are verified includes disabling local illumination compensation and / or generalized bidirectional prediction when overlapping block motion compensation is used and the weights are not equal to the default weights in the overlapping block motion compensation process.

Claims

1. The steps include: decoding the parameters for the current block, A step of disabling the first encoding tool for the current block in response to one or more conditions being verified, such as the second encoding tool being used for the current block and the given conditions for the parameters being verified, Otherwise, the steps include enabling the first encoding tool for the current block, If the first encoding tool is enabled, the steps include: decoding the current block using the first encoding tool; A decoding method comprising the following features.

2. The method of claim 1, wherein the step of disabling the first coding tool for the current block in response to one or more conditions being verified includes disabling bidirectional optical flow if the generalized bidirectional prediction weights are not equal to the default weights for the current block.

3. The method of claim 1, wherein the step of disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling decoder-side motion vector refinement when bidirectional optical flow is used for the current block.

4. The method of claim 1, wherein the step of disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling triangulation in response to a multiple hypothesis coding mode combining interprediction and intraprediction being used for the current block.

5. The method of claim 1, wherein the step of disabling the first coding tool for the current block in response to one or more conditions being verified includes disabling generalized bidirectional prediction or, if the block size is less than a predetermined value, setting the generalized bidirectional prediction weights to default weights.

6. Steps to obtain parameters for the current block, A step of disabling the first encoding tool for the current block in response to one or more conditions being verified, such as the second encoding tool being used for the current block and the given conditions for the parameters being verified, Otherwise, the steps include enabling the first encoding tool for the current block, If the first encoding tool is enabled, the steps include: encoding the current block using the first encoding tool; An encoding method that includes the following features.

7. The method of claim 6, wherein the step of disabling the first coding tool for the current block in response to one or more conditions being verified includes disabling bidirectional optical flow if the generalized bidirectional prediction weights are not equal to the default weights for the current block.

8. The method of claim 6, wherein the step of disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling decoder-side motion vector refinement when bidirectional optical flow is used for the current block.

9. The method of claim 6, wherein the step of disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling triangulation in response to a multiple hypothesis coding mode combining interprediction and intraprediction being used for the current block.

10. The method of claim 6, wherein the step of disabling the first coding tool for the current block in response to one or more conditions being verified includes disabling generalized bidirectional prediction or, if the block size is less than a predetermined value, setting the generalized bidirectional prediction weights to default weights.

11. Decoding parameters for the current block, In response to one or more conditions being verified, such as the use of a second encoding tool for the current block and the verification of given conditions for the parameters, the first encoding tool is disabled for the current block. Otherwise, enable the first encoding tool for the current block, When the first encoding tool is enabled, the current block is decoded using the first encoding tool. One or more processors configured to perform A decoding device provided.

12. Decoding apparatus of claim 11, wherein disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling bidirectional optical flow if the generalized bidirectional prediction weights are not equal to the default weights for the current block.

13. Decoding apparatus of claim 11, wherein disabling a first encoding tool for the current block in response to the verification of one or more conditions includes disabling decoder-side motion vector refinement when bidirectional optical flow is used for the current block.

14. Decoding apparatus of claim 11, wherein disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling triangulation in response to a multiple hypothesis coding mode combining interprediction and intraprediction being used for the current block.

15. Decoding apparatus of claim 11, wherein disabling the first coding tool for the current block in response to the verification of one or more conditions includes disabling generalized bidirectional prediction or setting the generalized bidirectional prediction weights to default weights if the block size is below a predetermined value.

16. To obtain parameters for the current block, In response to one or more conditions being verified, such as the use of a second encoding tool for the current block and the verification of given conditions for the parameters, the first encoding tool is disabled for the current block. Otherwise, enable the first encoding tool for the current block, When the first encoding tool is enabled, the current block is encoded using the first encoding tool. One or more processors configured to perform An encoding device equipped with an encoding device.

17. The encoding apparatus of claim 16, wherein the step of disabling a first encoding tool for the current block in response to one or more conditions being verified includes disabling bidirectional optical flow if the generalized bidirectional prediction weights are not equal to the default weights for the current block.

18. The encoding apparatus of claim 16, wherein the step of disabling a first encoding tool for the current block in response to the verification of one or more conditions includes disabling decoder-side motion vector refinement when bidirectional optical flow is used for the current block.

19. The encoding apparatus of claim 16, wherein the step of disabling a first coding tool for the current block in response to one or more conditions being verified includes disabling triangulation in response to a multiple hypothesis coding mode combining interprediction and intraprediction being used for the current block.

20. The encoding apparatus of claim 16, wherein the step of disabling the first encoding tool for the current block in response to the verification of one or more conditions includes disabling generalized bidirectional prediction or, if the block size is less than a predetermined value, setting the generalized bidirectional prediction weights to default weights.

Images