System and method for general multi-hypothesis prediction for video coding

Abstract

Systems and methods for video encoding using generalized bi-prediction are described. In an exemplary embodiment, to encode a current block of a video in a bitstream, a first reference block is selected from a first reference picture and a second reference block is selected from a second reference picture. Each reference block is associated with a weight, wherein the weight is an arbitrary weight ranging, for example, between 0 and 1. The current block is predicted using a weighted sum of the reference blocks. The weight may be selected from a plurality of candidate weights. The candidate weights may be signaled within the bitstream or may be implicitly derived based on a template. The candidate weights may be pruned to avoid out-of-range or substantially duplicate candidate weights. Generalized bi-prediction may additionally be used in frame rate up-conversion.

Description

[0001] This application is a divisional application of Chinese patent application No. 201780042257.5, entitled “System and method for general multi-hypothesis prediction for video coding”, filed on May 11, 2017, the contents of which are incorporated herein by reference in their entirety.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is a non-provisional application of, and claims the benefit under 35 U.S.C. §119(c) of, the following U.S. provisional patent applications: Application No. 62 / 336,227, filed May 13, 2016, entitled “System and Method for Generalized Multi-Hypothesis Prediction for Video Coding”; Application No. 62 / 342,772, filed May 27, 2016, entitled “System and Method for Generalized Multi-Hypothesis Prediction for Video Coding”; Application No. 62 / 399,234, filed September 23, 2016, entitled “System and Method for Generalized Multi-Hypothesis Prediction for Video Coding”; and Application No. 62 / 415,187, filed October 31, 2016, entitled “System and Method for Generalized Multi-Hypothesis Prediction for Video Coding”. All of these applications are hereby incorporated by reference in their entireties. Background Art

[0004] Video coding systems are widely used to compress digital video signals to reduce the storage requirements and / or transmission bandwidth of such signals. Among various types of video coding systems, such as block-based, wavelet-based, and object-based systems, block-based hybrid video coding systems are the most widely used and deployed today. Examples of block-based video coding systems include international video coding standards (such as MPEG-1 / 2 / 4 Part 2, H.264 / MPEG-4 Part 10 AVC, VC-1) and the latest video coding standard known as High Efficiency Video Coding (HEVC), which was developed by ITU-T / SG16 / Q.6 / VCEG and JCT-VC (Joint Collaboration Team on Video Coding) of ISO / IEC / MPEG.

[0005] Video encoded using block-based coding accounts for a major portion of data transmitted electronically, for example, over the Internet. It is desirable to increase video compression efficiency so that high-quality video content can be stored and transmitted using fewer bits. Summary of the Invention

[0006] In an exemplary embodiment, a system and method for performing generalized bi-prediction (GBi) are described. The exemplary method includes encoding and decoding (collectively, "coding") a video comprising a plurality of pictures, the plurality of pictures comprising a current picture, a first reference picture, and a second reference picture, each picture comprising a plurality of blocks. In the exemplary method, for at least a current block within the current picture, a block-level index is encoded to identify a first weight and a second weight within a set of weights, wherein at least one weight within the set of weights has a value not equal to 0, 0.5, or 1. The current block is predicted as a weighted sum of a first reference block within the first reference picture and a second reference block within the second reference picture, wherein the first reference block is weighted by the first weight and the second block is weighted by the second weight.

[0007] In some embodiments (or for some blocks), the block-level information identifying the first and second weights may be encoded for the current block in a manner other than encoding an index for the block. For example, the block may be encoded in merge mode. In this case, the block-level information may be information identifying a candidate block from a plurality of merge candidate blocks. Thus, the first and second weights may be identified based on the weights used to encode the identified candidate block.

[0008] In some embodiments, the first reference block and the second reference block may be further scaled by at least one scaling factor signaled within the bitstream for the current picture.

[0009] In some embodiments, the set of weights is encoded in the bitstream, allowing different sets of weights to be applied to different slices, pictures, or sequences. In other embodiments, the set of weights is predetermined. In some embodiments, only one of the two weights is signaled in the bitstream, and the other weight is derived by subtracting the signaled weight from 1.

[0010] In some embodiments, codewords are assigned to corresponding weights, and the weights are identified by using the corresponding codewords. The assignment of codewords to weights may be a predetermined assignment, or the assignment may be adaptively adjusted based on weights used in previously encoded blocks.

[0011] Also described herein are exemplary encoders and decoders for performing generalized bi-prediction.

[0012] The systems and methods described herein provide a novel technique for predicting blocks having sample values. This technique can be used by encoders and decoders. In the encoding method, the prediction of a block can result in the block containing sample values ​​being subtracted from the original input block to determine a residual encoded in the bitstream. In the decoding method, the residual can be decoded from the bitstream and added to the predicted block to obtain a reconstructed block that is identical or similar to the original input block. Thus, by reducing the number of bits required to encode and decode video in at least some implementations, the prediction method described herein can improve the operation of video encoders and decoders. Further advantages of the exemplary prediction method for the operation of video encoders and decoders will be described in the detailed description below. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] A more detailed understanding may be obtained from the following description given by way of examples in conjunction with the accompanying drawings, which are first briefly described below.

[0014] Figure 1 is a functional block diagram illustrating an example of a block-based video encoder.

[0015] Figure 2 is a functional block diagram illustrating an example of a block-based video decoder.

[0016] Figure 3 Is to use template T c And a schematic diagram of the prediction of the associated prediction blocks T0 and T1.

[0017] Figure 4 is a graph that provides a schematic representation of illuminance changes over time.

[0018] Figure 5 is a functional block diagram illustrating a video encoder configured to use generalized bi-prediction according to some embodiments.

[0019] Figure 6 is a functional block diagram of an exemplary generalized bi-prediction module for use within a video encoder.

[0020] Figure 7 is a diagram illustrating an exemplary decoder-side derivation of implicit weight values ​​for generalized bi-prediction.

[0021] Figure 8 is a schematic diagram of a tree structure for binarized weight indexing, where each circle represents a bit to be signaled.

[0022] Figure 9 is a functional block diagram illustrating a video decoder configured to use generalized bi-prediction according to some embodiments.

[0023] Figure 10is a functional block diagram of an exemplary generalized bi-prediction module for use within a video decoder.

[0024] Figure 11A and 11B The following codeword assignment methods are provided: Constant assignment ( Figure 11A ) and alternative assignments ( Figure 11B ).

[0025] Figure 12A and 12B A schematic diagram is provided for an example of block adaptive codeword assignment: the weight value field ( Figure 12A ) and the final codeword assignment updated from the constant assignment ( Figure 12B ).

[0026] Figure 13 It is a schematic diagram of merging candidate positions.

[0027] Figure 14 is a schematic diagram of an example of OBMC, where m is a basic processing unit for performing OBMC, N1 to N8 are causally adjacent sub-blocks, and B1 to B7 are sub-blocks within the current block.

[0028] Figure 15 An example regarding frame rate up conversion (FRUC) is shown, where v0 is a given motion vector corresponding to reference list L0, and v1 is a scaled MV based on v0 and temporal distance.

[0029] Figure 16 is a diagram showing an example of the structure of a bit stream after encoding.

[0030] Figure 17 is a diagram illustrating an example communication system.

[0031] Figure 18 is a diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used as an encoder or decoder in some embodiments. DETAILED DESCRIPTION

[0032] Block-based encoding

[0033] Figure 11 is a block diagram of a general block-based hybrid video coding system 100. An input video signal 102 is processed block by block. In HEVC, extended block sizes (called "coding units" or CUs) are used to efficiently compress high-resolution (1080p and above) video signals. In HEVC, a CU can be up to 64x64 pixels. A CU can be further partitioned into prediction units or PUs, for which a separate prediction method can be applied. For each input video block (MB or CU), spatial prediction (160) and / or temporal prediction (162) can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from already coded neighboring blocks within the same video picture / slice to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion compensated prediction") uses pixels from already coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given video block may be signaled by one or more motion vectors indicating the direction and amount of motion between the current block and its reference blocks. In addition, if multiple reference pictures are supported (e.g., for recent video coding standards such as H.264 / AVC or HEVC), a reference index of the reference picture may also be sent for each video block. The reference index is used to identify which reference picture within the reference picture library (164) the temporal prediction signal comes from. After spatial and / or temporal prediction, a mode decision block (180) within the encoder may select the best prediction mode, for example, based on a rate-distortion optimization approach. The prediction block may then be extracted from the current video block (116), and the prediction residuals decorrelated using a transform (104) and quantization (106) to achieve a target bit rate. The quantized residual coefficients may be inverse quantized (110) and inverse transformed (112) to form a reconstructed residual, which may then be added back to the prediction block (126) to form a reconstructed video block. Further in-loop filtering (such as a deblocking filter and an adaptive loop filter) may be applied (166) to the reconstructed video block before it is placed in a reference picture library (164) and used to encode future video blocks. To form the output video bitstream 120, the coding mode (intra-frame or inter-frame), prediction mode information, motion information, and quantized residual coefficients may be fed into an entropy coding unit (108) for further compression and packing to form a bitstream.

[0034] Block-based decoding

[0035] Figure 2The main block diagram of the block-based video decoder 200 is shown. At the entropy decoding unit 208, the video bitstream 202 is unpacked and entropy decoded. The coding mode and prediction information are sent to the spatial prediction unit 260 (if intra-coded) or spatial prediction unit 262 (if inter-coded) to form a prediction block. The residual transform coefficients are sent to the inverse quantization unit 210 and the inverse transform unit 212 to reconstruct the residual block. Thereafter, at 226, the prediction block and the residual block are added together. The reconstructed block can be further filtered in-loop before being stored in the reference picture library 264. The reconstructed video in the reference picture library can then be sent to drive a display device and used to predict future video blocks.

[0036] In modern video codecs, bidirectional motion-compensated prediction (MCP) is well-known for its efficiency in removing temporal redundancy by exploiting temporal correlation between pictures and has been widely used in most state-of-the-art video codecs. However, the bi-prediction signal is simply formed by combining the single prediction signals with a weight value equal to 0.5. This is not necessarily optimal from the perspective of combining single prediction signals, especially in certain situations where the illumination changes rapidly from one reference picture to another. Therefore, several prediction techniques have been developed that aim to compensate for the temporal variations in illumination by applying some global or local weights and offsets to each sample value within the reference picture.

[0037] Weighted Dual Prediction

[0038] Weighted bi-prediction is a coding tool primarily used to compensate for temporally varying illumination (such as degenerate transitions) when performing motion compensation. For each slice, two sets of multiplicative weights and additive offsets are explicitly indicated and applied separately to the motion compensated prediction, one set at a time for each reference list. This technique is most effective when illumination changes linearly and evenly from one picture to another.

[0039] Local illumination compensation

[0040] In local illumination compensation, parameters (two pairs of multiplicative weight values ​​and additive offset values) can be adaptively modulated block by block. Unlike weighted bi-prediction, which indicates these parameters at the slice level, this technique relies on adapting the optimized parameters to the template (T C The final parameters can be obtained by minimizing T C and T0 (for the first pair of weights and biases) and T C The illumination difference between T1 and T2 (for the second pair of weights and offsets) is optimized. The obtained parameters can then be used to perform motion compensation similar to weighted bi-prediction.

[0041] The impact of illumination changes

[0042] Spatial and temporal illumination changes can severely affect the performance of motion compensation prediction. Figure 4 As shown in Figure 2, when the illumination degenerates along the time direction, motion compensation prediction cannot provide good performance. For example, when the object sample moves along the time period from t-3 to t, the intensity value of the sample will change from v to v along its motion trajectory. t-3 becomes v t Assuming that the sample will be predicted at the tth picture, its predicted value will be limited to v t-3 With v t-1 , which results in poor motion-compensated predictions. The weighted bi-prediction and local illumination compensation techniques described above do not fully address this problem. Weighted bi-prediction fails because illumination can fluctuate wildly within the image. Local illumination compensation sometimes produces poor estimates of weights and offsets due to low illumination correlation between a block and its associated template block. These examples show that both global and template-based local descriptions are insufficient to capture illumination variations in space and time.

[0043] Exemplary embodiments

[0044] Example embodiments described herein can improve the prediction efficiency of weighted motion compensated prediction. In some embodiments, systems and methods for general multi-hypothesis prediction are proposed, which linearly combine multi-hypothesis prediction signals using motion compensated prediction and block-level weight values. In some embodiments, a general dual prediction framework is described using weight values. In some embodiments, a finite set of weights is used at the sequence level, picture level, and slice level, and a process for constructing the weight sets is described. In some embodiments, the weight values ​​are determined based on a given set of weights and optimized taking into account the signals of the current block and its reference blocks. Example encoding methods for signaling weight values ​​are described. Example encoder search criteria are described for use in a motion estimation process for the proposed prediction, and the proposed prediction process for use in conjunction with the disclosed temporal prediction techniques is described.

[0045] In this disclosure, systems and methods for temporal prediction using generalized multi-hypothesis prediction are described. Figure 5 and Figure 9An exemplary encoder and decoder using generalized bi-prediction are described. The systems and methods disclosed herein are organized according to the following sections. Section "Generalized Multi-Hypothesis Prediction" describes an exemplary embodiment using generalized multi-hypothesis prediction. Section "Generalized Bi-Prediction" describes an exemplary framework and prediction process for generalized bi-prediction. Sections "Construction of Weight Sets" and "Weight Index Coding" describe an exemplary construction process for a weight set and an exemplary technique for signaling weight selection within the set, respectively. In section "Extensions to Advanced Temporal Prediction Techniques," systems and methods are described for combining the exemplary proposed prediction method with advanced inter-frame prediction techniques (including local illumination compensation and weighted bi-prediction, merge mode, overlapped block motion compensation, affine prediction, bidirectional optical flow, and a decoder-side motion vector derivation technique known as rate-up conversion bi-prediction). In section "GBi Prediction Search Strategy," an exemplary encoder-only method for enhancing the efficiency of the exemplary prediction method is described.

[0046] Generalized multi-hypothesis forecasting

[0047] The exemplary systems and methods described herein employ generalized multi-hypothesis prediction. Generalized multi-hypothesis prediction is described as a generalized form of multi-hypothesis prediction to provide an estimate of the intensity value of a pixel based on a linear combination of multiple motion compensated prediction signals. Generalized multi-hypothesis prediction can take advantage of the advantages of multiple predictions with different quantities by combining them together. To achieve accurate estimation, the motion compensated prediction signals can be processed by a predefined function f(·) (e.g., gamma correction, local illumination correction, dynamic range conversion) and then linearly combined. Generalized multi-hypothesis prediction can be described with reference to formula (1):

[0048]

[0049] Where P[x] represents the final prediction signal of sample x at position x in the picture, w i represents the weight value applied to the i-th motion hypothesis from the i-th reference picture, P i [x+v i ] is to use motion vector (MV) v i is the motion compensated prediction signal of x, and n is the total number of motion hypotheses.

[0050] For motion compensated prediction, one factor that needs to be considered is how to balance the accuracy of the motion field with the required motion overhead to achieve maximum rate-distortion performance. An accurate motion field means better prediction, while the required motion overhead is sometimes more important than the prediction accuracy benefit. Therefore, in an exemplary embodiment, the proposed video encoder is able to adaptively switch between different numbers of motion hypotheses n and find the value of n that provides the best rate-distortion performance for each corresponding PU. To help explain how general multi-hypothesis prediction works, the value of n=2 is selected as an example in the following section because two motion hypotheses are the most commonly used in current video coding standards. However, other values ​​of n may be used interchangeably. To simplify the understanding of the exemplary embodiments, the formula f(·) can be regarded as an identity function and is therefore not explicitly given. For those skilled in the art, it is obvious after understanding the present disclosure that the system and method disclosed herein can be applied to cases where f(·) is not an identity function.

[0051] Universal Bi-Prediction

[0052] The term generalized bi-prediction (GBi) used herein refers to a special case of generalized multi-hypothesis prediction where the number of motion predictions is limited to 2, i.e., n = 2. In this case, the prediction signal at sample x given by equation (1) can be simplified to:

[0053] P[x]=w0*P0[x+v0]+w1*P1[x+v1] (2)

[0054] where w0 and w1 are weight values ​​shared across all samples in the block. Based on this formula, a wide variety of prediction signals can be generated by adjusting the weight values ​​w0 and w1. Some configurations of w0 and w1 can result in predictions that are identical to conventional uni- and bi-prediction, such as (w0, w1) = (1, 0) for uni-prediction using reference list L0, (w0, w1) = (0, 1) for uni-prediction using reference list L1, and (w0, w1) = (0.5, 0.5) for bi-prediction using two reference lists. In the cases where (w0, w1) = (1, 0) and (w0, w1) = (0, 1), only one set of motion information is signaled, since the other set associated with the weight value equal to 0 does not contribute to the prediction signal P[x].

[0055] The flexibility of the values ​​of w0 and w1 may result in high signaling overhead costs, especially at high precision. To save signaling overhead, in some embodiments, a unity gain constraint may be applied, that is, w0+w1=1, so that only one weight value for the block is explicitly indicated for the PU encoded by GBi. To further reduce the weight signaling overhead, the weight value may be signaled at the CU level rather than the PU level. For simplicity of explanation, in the discussion of this disclosure, w1 is signaled, so that formula (2) can be further simplified to:

[0056] P[x]=(1-w1)*P0[x+v0]+w1*P1[x+v1]. (3)

[0057] In an example embodiment, to further limit signaling overhead, frequently used weight values ​​may be arranged in a set (hereinafter referred to as W L1 ), so that within a limited range, each weight value can be indicated by an index value, that is, the weight_index (weight_idx) points to its L1 Items occupied within.

[0058] In exemplary embodiments, generalized bi-prediction does not introduce additional decoding overhead by supporting the generation of a weighted average of two reference blocks. Since most current video standards (e.g., AVC, HEVC) support weighted bi-prediction, the same prediction model can be adapted for GBi prediction. In exemplary embodiments, generalized bi-prediction can be applied not only to traditional uni-prediction and bi-prediction, but also to other advanced temporal prediction techniques such as affine prediction, advanced temporal motion vector derivation, and bidirectional optical flow. These techniques all aim to derive motion field representations at finer units (e.g., 4x4) at the expense of very low motion overhead. Affine prediction is a model-based motion field coding method in which the motion of each unit within a PU is derived based on model parameters. Advanced temporal motion vector derivation involves deriving the motion of each unit from the motion field of a temporal reference picture. Bidirectional optical flow involves deriving motion refinements for each pixel using an optical flow model. Regardless of the unit size, once weight values ​​are specified at the block level, a given video codec can perform generalized bi-prediction on a per-unit basis by using those derived motions and the given weight values.

[0059] An exemplary encoder and decoder using generalized bi-prediction are described in more detail below.

[0060] Example encoder for generalized bi-prediction

[0061] Figure 5 A block diagram of an exemplary video encoder adapted to perform generalized bi-prediction. Figure 1 Similar to the video encoder shown in FIG, spatial prediction and temporal prediction are two basic pixel domain prediction models in the exemplary video encoder. The spatial prediction model can be used with Figure 1 The spatial prediction models shown are the same. Figure 1 The temporal prediction model labeled “motion prediction” in FIG5 may be replaced by a generalized bi-prediction (GBi) model 502. The generalized bi-prediction (GBi) model is operable to combine two separate motion compensated prediction (MCP) signals in a weighted average manner. Figure 6 As shown, the GBi model can implement a process to generate the final inter-frame reference signal as follows. The GBi model can perform motion estimation within the reference picture(s) to search for the two best motion vectors (MVs) pointing to two reference blocks that minimize the weighted bi-prediction error between the current video block and the bi-prediction prediction. The GBi model can obtain the two prediction blocks by performing motion compensation using the two best MVs. The GBi model can then calculate the prediction signal for the generalized bi-prediction based on the weighted average of the two prediction blocks.

[0062] In some embodiments, all available weight values ​​may be specified in a single set. Since the weight values ​​would consume a lot of bits if they were signaled for two reference lists at the PU level (which would mean that two separate weight values ​​for each bi-predicted PU would be signaled), a unity gain constraint (the sum of the weight values ​​is equal to 1) may be applied. Under this constraint, only one single weight value for each PU is signaled, and the other weight value can be derived by subtracting the signaled weight value from 1. For ease of explanation, in this disclosure, the weight value associated with reference list L1 is signaled, and the weight value set may be obtained by W L1 In order to further save signaling overhead, the weight value can be pointed to W L1 The index value weight_idx of the item position in is encoded. L1 With appropriate assignment, both traditional uni-prediction (weight equal to 0 for one reference list and weight equal to 1 for the other) and traditional bi-prediction (weight equal to 0.5 for both reference lists) can be performed under the GBi framework. L1 ={0,0.5,1}, the GBi model can be realized Figure 1 The motion prediction model shown has the same functionality.

[0063] Except {0,0.5,1}, for W L1The additional weight values ​​may be specified at slice level, picture level or sequence level by a non-negative integer extra_number_of_weights indicating their number, so that there are extra_number_of_weights+3 separate weights within the GBi framework. More specifically, in an exemplary embodiment, when extra_number_of_weights is greater than 0, one of these additional weight values ​​may be derived block by block, depending on the control of the flag implicit_weight_flag given at slice level, picture level or sequence level. When this flag is set equal to 1, this particular weight value may not be signaled, but may be used as shown in the following example. Figure 7 As shown in the figure, it is obtained by choosing a weight value that minimizes the difference between the generalized bi-predictive signal of the directly adjacent inverse L-shape (called template) and the reconstructed signal of the template. L1 The process of constructing can be performed by the weight set construction model 504.

[0064] In order to make W L1 To adapt the additional weight values ​​within the GBi prediction model to pictures with highly dynamic illumination changes, two scaling factors (GBi_scaling_factors) can be applied and signaled at the picture level. Using these two scaling factors, the weight set construction model can scale the additional weight values ​​for GBi prediction. After inter-frame prediction (i.e., GBi prediction within the proposed video encoder) and intra-frame prediction, the original signal can be subtracted from the final prediction signal, thereby generating the final prediction residual signal for encoding.

[0065] In the exemplary proposed video encoder, block motion (motion vector and reference picture index) and weight value index are the only block-level information to be indicated for each intra-coded PU.

[0066] In an exemplary embodiment, the block motion information of the GBi prediction may be encoded in the same manner as the base video codec, except that when weight_idx is associated with a weight equal to 0 or 1 (i.e., equivalent to the case of uni-prediction), two sets of motion information for the PU are signaled.

[0067] In the exemplary video encoder, the weight index encoding module 506 is used to binarize the weight_idx of each PU. The output of the weight index encoding module may be a unique binary representation of weight_idx, binary_weight_index (binary_weight_idx). Figure 8The tree structure of an exemplary binarization scheme is shown. Similar to conventional inter prediction, the first bit of binary_weight_idx can distinguish between uni-prediction (weight index associated with a weight value equal to 0 or 1) and bi-prediction (weight index associated with W for each inter PU). L1 In the uni-predicted branch, another bit may be signaled to indicate whether the L0 reference list (weight index associated with weight value equal to 0) or the L1 reference list (weight index associated with weight value equal to 1) is referenced. In the bi-predicted branch, each leaf node may be assigned a weight index associated with the remaining weight value (i.e., W L1 A unique weight index value is associated with one of the weights (non-zero and non-one) in the bi-prediction branch. At the slice level or picture level, the exemplary video encoder can adaptively switch between several predefined assignments, or dynamically assign each weight to a unique leaf node on a per-PU basis based on the use of weight values ​​from previously encoded blocks. In general, frequently used weight indices are assigned to leaf nodes close to the root in the bi-prediction branch, while other weight indices are assigned to leaf nodes deeper away from the root. By traversing Figure 8 Within the tree, each weight_idx can be converted into a unique binary_weight_idx for entropy coding.

[0068] On the decoding framework of general bi-prediction

[0069] Figure 9 is a block diagram of a video decoder in some embodiments. Figure 9 The decoder is operable to decode Figure 5 The bitstream generated by the video encoder shown in FIG. The coding mode and prediction information can be used to derive a prediction signal using spatial prediction or generalized bi-prediction. For generalized bi-prediction, block motion information and weight values ​​can be received and decoded.

[0070] The weight index decoding module 902 can decode the weight index encoded by the weight index encoding module 506 in the proposed video encoder. The weight index decoding module 902 can reconstruct a video similar to Figure 8 The same tree structure is specified, and in the same way as the proposed video encoder, each leaf node on the tree is assigned a unique weight_idx. In this way, the tree can be synchronized between the proposed video encoder and decoder. By traversing this tree, each received binary_weight_idx can find its associated weight_idx at a leaf node on the tree. Similar to Figure 5 The exemplary video decoder includes a method for constructing a weight set W L1The weight set construction module 904. When implicit_weight_flag is equal to 1, W L1 One of the additional weight values ​​within can be derived rather than explicitly signaled, and W L1 All additional weight values ​​within can be further scaled using the scaling factors indicated by gbi_scaling_factors. L1 This is done by retrieving the weight value pointed to by weight_idx.

[0071] Depending on the weight value selected at each block, the decoder may receive one or two sets of motion information. When the reconstruction weight value is non-zero or non-1, the decoder may receive both sets of motion information; otherwise (when it is 0 or 1), the decoder may receive only one set of motion information associated with the non-zero weight. For example, if the weight value is equal to 0, only the motion information for reference list L0 will be signaled; otherwise, if the weight value is equal to 1, only the motion information for reference list L1 will be signaled.

[0072] By utilizing block motion information and weight values, Figure 10 The illustrated generalized bi-prediction module 1050 is operable to calculate a generalized bi-prediction prediction signal based on a weighted average of two motion compensated prediction blocks.

[0073] Depending on the coding mode, a spatial prediction signal or a generalized bi-prediction signal may be added to the reconstructed residual signal to obtain a reconstructed video block signal.

[0074] Construction of weight set

[0075] The following describes the weight set W using the weights sent explicitly, the weights derived at the decoder side, and the scaled weights. L1 An exemplary construction process and a method for compressing the weight set W L1 An exemplary pruning process for the size of .

[0076] Explicit weight value

[0077] Explicit weight values ​​can be signaled and managed hierarchically at each sequence level, picture level, and slice level. Weights specified at a lower level can replace those at a higher level. Assuming the number of explicit weights at a higher level is p and the number of explicit weights at a relatively lower level is q, the following substitution rules can be applied when constructing the weight value list at the lower level:

[0078] • When p>q, the last q weights at the higher level may be replaced with q weights at the lower level.

[0079] • When p ≤ q, all weights at higher levels are replaced by those specified at lower levels.

[0080] The number of explicit weight values ​​can be indicated by extra_number_of_weights at each of the sequence level, picture level, and slice level. In some embodiments, at the slice level, the base weight set always includes three default values ​​of {0, 0.5, 1} for GBi to support traditional uni-prediction and bi-prediction, so that a total of (extra_number_of_weights+3) weights can be used for each block. For example, when the value of extra_number_of_weights given at the sequence level, picture level, and slice level is 2 (e.g., w A ,w B ), 1(for example, w C ) and 3 (e.g., w D ,w E ,w F ), the available weight values ​​at the sequence level, picture level and slice level are {w A ,w B}、{w A ,w C} and {0,0.5,1}∪{w D ,w E ,w F}. In this example, the W mentioned in the "General Dual Prediction" section L1 is the slice-level weight set.

[0081] Derivation process of implicit weight value

[0082] In some embodiments, the slice-level weight set W L1 The weight values ​​within can be derived by template matching at the encoder and decoder without the need for signaling. Figure 7 As shown, the implicit weight value can be obtained by making the prediction signal (T0 and T1) of the template using the motion information of the current block and the reconstruction signal of the template (ie, T c ) is derived by minimizing the difference between them. The problem can be formulated as:

[0083] w * =argmin w ∑ x (T c [x]-(1-w)*T0[x+v0]-w*T1[x+v1]) 2 (4)

[0084] Where v0 and v1 are the motion vectors of the current block. Since formula (4) is a quadratic function, if T0 and T1 are not exactly the same, a closed-form expression for the derived weights can be obtained, namely:

[0085]

[0086] The effectiveness of this method can be seen when the weight values ​​of the current block signal are correlated with the weight values ​​of the associated template prediction signal, however this is not always guaranteed, especially when the pixels in the current block and its associated template are located in different moving objects. In order to maximize the prediction performance of GBi, when extra_number_of_weights ≥ 1, the implicit_weight_flag can be signaled at the slice level, picture level or sequence level to determine whether implicit weights are used. Once this is set to 1, W L1 The final slice-level weight value in can be derived and does not need to be sent by signal. For example, the w mentioned in the "Explicit Weight Value" section above F It does not need to be signaled, and when implicit_weight_flag is equal to 1, the weight of the block can be derived implicitly.

[0087] About the scaling process of weight values

[0088] In some embodiments, the explicit weight values ​​can be further scaled using two scaling factors gbi_scaling_factors indicated at the picture level. Due to the potentially highly dynamic illumination changes within a picture over time, the dynamic range of these weight values ​​may not be sufficient to cover all of these cases. While weighted bi-prediction can compensate for illumination differences between pictures, it is not guaranteed to always be enabled in the base video codec. Therefore, when weighted bi-prediction is not used, those scaling factors can be used to adjust for illumination differences across multiple reference pictures.

[0089] The first scaling factor can be used to magnify W L1 Each explicit weight value in . Through this, the prediction function of GBi in formula (3) can be expressed as:

[0090] P[x]=(α*(1-w1-0.5)+0.5)*P0[x+v0]+(α*(w1-0.5)+0.5)*P1[x+v1]

[0091] =(1-w′1)*P0[x+v0]+w′1*P1[x+v1] (6) Where α is the first scaling factor of the current picture, and w1′ represents the scaled weight value (i.e., α*(w1-0.5)+0.5). The first formula in formula (6) can be expressed in the same form as formula (3). The only difference lies in the weight values ​​applied to formulas (6) and (3).

[0092] The second scaling factor can be used to reduce the illumination difference between the associated reference pictures P0 and P1. By this scaling factor, formula (6) can be further reformulated as:

[0093]

[0094] Where s, s0, and s1 represent the second scaling factors signaled at the current picture and its two reference pictures, respectively. According to formula (7), an optimal assignment for the variable s is the average value of the samples within the current picture. Therefore, after applying the second scaling factor, the average values ​​of the reference pictures can be expected to be similar. Due to the commutative property, applying the scaling factors to P0 and P1 is equivalent to applying them to the weight values, and thus formula (7) can be reinterpreted as:

[0095]

[0096] Therefore, the construction process of the weight set can be expressed as a function of explicit weights, implicit weights, scaling factors and reference pictures. L1 becomes {0,0.5,1}∪{(s / s1)*w D ',(s / s1)*w E ',(s / s1)*w F '}, and the weight set for L0 becomes {1,0.5,1}∪{(s / s0)*(1-w D '),(s / s0)*(1-w E '),(s / s0)*(1-w F ')}, where s1 is the average sample value of the reference pictures in list L1 of the current block, and s0 is the average sample value of the reference pictures in list L0 of the current block.

[0097] About weight pruning

[0098] Example embodiments are operable to further reduce W L1 The following describes two exemplary methods for pruning weight values. The first method operates in response to motion compensated prediction results, while the second method operates based on weight values ​​outside the range between 0 and 1.

[0099] Prediction-based methods. Given a motion vector for a PU, not every weight can produce a bi-prediction that is substantially different from the other weights. Exemplary embodiments exploit this property, pruning redundant weight values ​​(which produce similar bi-prediction signals) and keeping only one weight among the multiple redundant values ​​so that W L1 More compact. To do this, a function can be used to calculate the similarity between two bi-prediction signals with different weight values. The function can be, but is not limited to, a cosine similarity function, which can be run as follows:

[0100]

[0101] where w (i) and w (j) W L1 Two independent weight values ​​in , v0 and v1 are given bi-predictive motion information, and P[x; w, v0, v1] represents the same prediction function specified in equations (3), (6) and (8) given w, v0 and v1. When the value of equation (9) is below a given threshold (indicated by the slice-level weight_pruning_threshold), one of the weights may be pruned depending on the slice-level syntax pruning_smaller_weight_flag. If this flag is set equal to 1, the pruning process may start from W L1 Remove w (i) and w (j) Otherwise (when the flag is set equal to 0), the larger one may be removed. In an exemplary embodiment, this pruning process may be applied to W L1 Each pair of weight values ​​within, and finally, the final W L1 No two weight values ​​will produce similar bi-prediction signals. The similarity between two weight values ​​can also be evaluated by using the sum of absolute transformed differences (SATD). To reduce computational complexity, the similarity can be evaluated by using two sub-sampled prediction blocks. For example, it can be calculated by using sub-sampled rows or sub-sampled columns of samples in the horizontal and vertical directions.

[0102] Method based on weight values. Depending on the coding performance under different coding structures (e.g., hierarchical structures or low-latency structures), weight values ​​outside the range between 0 and 1 (or simply out-of-range weights) may have different performances. In order to take advantage of this fact, exemplary embodiments use a set of sequence-level indexes (weight_control_index (weight_control_idx)) to limit the use of out-of-range weights for each temporal layer separately. In these embodiments, each weight_control_idx is associated with all pictures of a specific temporal layer. Depending on how the index is configured, out-of-range weights may be conditionally used or pruned, as follows:

[0103] For weight_control_idx=0, W L1 The associated images remain unchanged.

[0104] For weight_control_idx=1, W L1 Weights outside the range are not available for the associated image.

[0105] For weight_control_idx=2, W L1 Weights outside the range are only available for some associated pictures whose reference frames come purely from the past (for example, low-latency configurations in HEVC and JEM).

[0106] For weight_control_idx=3, W L1 Out-of-range weights within can only be used for the associated picture when the slice level flag mvd_l1_zero_flag in HEVC and JEM is enabled.

[0107] Weight Index Encoding

[0108] Exemplary systems and methods for binarization and codeword assignment for weight index encoding are described in more detail below.

[0109] Binarization process for weight index encoding

[0110] In an exemplary embodiment, each weight index (weight_idx) may be converted to a unique binary representation (binary_weight_idx) by a systematic code before entropy encoding. For illustrative purposes, Figure 8The tree structure of the proposed binarization method is shown in FIG. The first bit of binary_weight_idx is used to distinguish between uni-prediction (which is associated with a weight equal to 0 or 1) and bi-prediction. Another bit signaled within the uni-prediction branch indicates which of the two reference lists is referenced, reference list L0 (associated with a weight index pointing to a weight value equal to 0) or reference list L1 (associated with a weight index pointing to a weight value equal to 1). In the bi-prediction branch, each leaf node can be assigned to a weight that is equal to W. L1 The exemplary video codec may support various systematic codes to binarize the bi-prediction branches, such as truncated unary codes (e.g. Figure 8 ) and Exponential Golomb codes. An exemplary technique for assigning a unique weight_idx to each leaf node in a bi-predictive branch is described in more detail below. By searching the tree structure, each weight index can be mapped to a unique codeword or recovered from a unique codeword (e.g., binary_weight_idx).

[0111] Adaptive codeword assignment for weight-indexed coding

[0112] In the exemplary binary tree structure, each leaf node corresponds to a codeword. In order to reduce the signaling overhead of the weight index, various adaptive codeword assignment methods can be used to map each leaf node in the dual prediction branch to a unique weight index. Exemplary methods include predetermined codeword assignment, block adaptive codeword assignment, time layer-based codeword assignment, and time-delayed CTU-adaptive codeword assignment. These exemplary methods can update the codeword assignment within the dual prediction branch based on the presence of weight values ​​used in previously encoded blocks. Frequently used weights can be assigned to codewords with shorter lengths (e.g., shallower leaf nodes in the dual prediction branch), while other weights can be assigned to codewords with relatively longer lengths.

[0113] 1) Predetermined codeword assignment. By using predetermined codeword assignment, a constant codeword assignment can be provided for leaf nodes within a bi-predictive branch. In this method, the weight index associated with a weight of 0.5 can be assigned the shortest codeword, i.e., Figure 8Node i in. Weight values ​​other than 0.5 can be divided into two sets: Set 1 contains all values ​​greater than 0.5, which are sorted in ascending order; Set 2 contains all values ​​less than 0.5, which are sorted in descending order. Afterwards, these two sets are interleaved to form Set 3, which can start from Set 1 or Set 2. All remaining codewords of length from short to long are assigned to weight values ​​in Set 3 in sequence. For example, when the set of all possible weight values ​​in the bi-predictive branch is {0.1, 0.3, 0.5, 0.7, 0.9}. Set 1 is {0.7, 0.9}, Set 2 is {0.3, 0.1}, if interleaving starts from Set 1, Set 3 is {0.7, 0.3, 0.9, 0.1}. Codewords of length from short to long are assigned to 0.5, 0.7, 0.3, 0.9 and 0.1 in sequence.

[0114] Some codecs may discard a motion vector difference when two motion information sets are sent, in which case the assignment can be changed. For example, this behavior can be found in HEVC via the slice level flag mvd_l1_zero_flag. In this case, alternative codeword assignments can be made for weight values ​​greater than and close to 0.5 (e.g., w + ) are assigned the weight index associated with them. Then, + The weight index associated with the nth smallest (or largest) weight value among the weight values ​​of can be assigned the (2n+1)th shortest (or 2nth shortest) codeword. Based on the previous example, the codewords from short to long in length can be sequentially assigned to 0.7, 0.5, 0.9, 0.3 and 0.1. Figures 11A-11B The final assignments for these two examples are shown.

[0115] 2) Block-adaptive codeword assignment using causal-neighboring weights. The weight values ​​used in the causal-neighboring blocks can be related to the weight values ​​used in the current block. Based on this knowledge and a given codeword assignment method (e.g., constant assignment or replaceable assignment), the weight index found from the causal-neighboring blocks can be moved to the leaf node with the shorter codeword length in the dual prediction branch. Similar to the process of building the motion vector prediction list, the causal-neighboring blocks can be assigned according to Figure 12A The sorting order shown is accessed, and at most two weight indices can be moved up. As can be seen from the figure, from the lower left block to the left block, the first available weight index (if any) can be moved up to have the shortest codeword length; from the upper right block to the upper left block, the first available weight index (if any) can be moved up to have the second shortest codeword length. For other weight indices, according to their codeword lengths in the original given assignment, they can be assigned to the remaining leaf nodes from the shallowest to the deepest. Figure 12BAn example is given of how a given codeword assignment can adapt itself to the causal neighbor weights. In this example, a constant assignment can be used and weight values ​​equal to 0.3 and 0.9 can be shifted up.

[0116] 3) Temporal layer-based codeword assignment. In an exemplary method using temporal layer-based codeword assignment, the proposed video encoder can adaptively switch between constant codeword assignment and alternative codeword assignment. Based on the use of weight indices of previously coded pictures at the same temporal layer or by using the same QP value, the optimal codeword assignment method with the minimum expected codeword length using weight indices can be found by:

[0117]

[0118] Among them L m (w) represents the codeword length of w using a certain codeword assignment method m, To set the weight set only for dual prediction, and Prob k (w) denotes the cumulative probability of w over k pictures at the temporal layer.Once the optimal codeword assignment method is determined, it can be applied to encode the weight index or parse the binary codeword index for the current picture.

[0119] Several different approaches can be considered to accumulate the weight index usage over the temporal picture. An exemplary approach can be formulated as a common formula:

[0120]

[0121] where w i W L1 A certain weight within, Count j (w) indicates the presence of a certain weight value at the jth picture of the temporal layer, n determines the number of most recent pictures to be stored, and λ is a forgetting term. Since n and λ are encoder-only parameters, they can be adaptively adjusted at each picture to accommodate various coding conditions, such as n = 0 for scene changes and a smaller λ for motion video.

[0122] In some embodiments, the choice of codeword assignment method can be explicitly indicated using slice-level syntax elements. Therefore, the decoder does not need to maintain the use of weight indices over time, and the dependency of parsing on weight indices on temporal pictures can be completely avoided. This approach can also improve decoding robustness.

[0123] 4) CTU adaptive codeword assignment. Switching between different codeword assignment methods based solely on the weight usage of previously coded pictures may not always match the codeword assignment of the current picture well. This can be attributed to the lack of consideration of the weight usage of the current picture. In an exemplary embodiment using CTU adaptive codeword assignment, Prob k (w i ) can be updated based on the weight usage of the coding blocks in the current CTU row and directly above the CTU row. Assuming that the current picture is the (k+1)th picture in the temporal layer, then Prob k (w i ) can be updated on a CTU-by-CTU basis as follows:

[0124]

[0125] Where B represents the set of coded CTUs in the current CTU row and directly above the CTU row, and Count' j (w) represents the existence of a certain weight value at the j-th CTU collected in set B. Once Prob k (w i ) is updated, it can be applied to formula (10), and the optimal codeword assignment method can be determined accordingly.

[0126] Extensions to Advanced Time Prediction Techniques

[0127] The embodiments discussed below are used to extend the application of generalized bi-prediction with other coding techniques, including local illumination compensation, weighted bi-prediction, merge mode, bidirectional optical flow, affine motion prediction, overlapped block motion compensation, and frame rate up-conversion bi-prediction.

[0128] Local illumination compensation and weighted dual prediction

[0129] An exemplary generalized bi-prediction technique may be performed based on local illumination compensation (IC) and / or weighted bi-prediction or other techniques. Both IC and weighted bi-prediction are operable to compensate for illumination changes on a reference block. One difference between them is that when IC is used, the weights (c0 and c1) and offsets (o0 and o1) are derived by performing template matching on a block-by-block basis; when weighted bi-prediction is used, these parameters are explicitly signaled on a slice-by-slice basis. Using these parameters (c0, c1, o0, o1), the prediction signal for GBi can be calculated as:

[0130]

[0131] The weight value scaling process described in the above “About the Scaling Process of Weight Values” section may be applied. When the scaling process is not applied, the prediction signal of GBi may be calculated as:

[0132] P[x]=(1-w1)*(c0*P0[x+v0]+o0)+w1*(c1*P1[x+v1]+o1). (14)

[0133] The use of a combined prediction process, such as those given in formula (13) or (14), can be signaled at the sequence level, the picture level, or the slice level. Signaling can be performed separately for the combination of GBi and IC and for the combination of GBi and weighted bi-prediction. In some embodiments, the combined prediction process of formula (13) or (14) can be applied only when the weight value (w1) is not equal to 0, 0.5, or 1. Specifically, when the use of the combined prediction process is active, the value of the block-level IC flag (which can be used to indicate the use of IC) determines whether GBi (w1≠0, 0.5, 1) is combined with IC. Otherwise, when the combined prediction process is not used, GBi (w1≠0, 0.5, 1) and IC are performed as two independent prediction modes, and for each block, the block-level IC flag does not need to be signaled and can therefore be inferred to be 0.

[0134] In some embodiments, whether GBi can be combined with IC or weighted bi-prediction can be signaled using a sequence parameter set (SPS), a picture parameter set (PPS), or a high-level syntax at the slice header using flags such as GBi_IC_comb_flag and GBi_weighted bi-prediction_comb_flag (gbi_wb_comb_flag). In some embodiments, if gbi_ic_comb_flag is equal to 0, GBi and IC are not combined, and for any bi-predicted coding unit, GBi weight values ​​(w1≠0, 0.5, 1) and IC flags will not coexist. For example, in some embodiments, if GBi weights (w1≠0, 0.5, 1) are signaled for a coding unit, then no IC flag will be signaled and the flag value can be inferred to be 0; otherwise, the IC flag can be explicitly signaled. In some embodiments, if gbi_ic_comb_flag is equal to 1, GBi and IC may be combined, and the GBi weight and IC flag may be signaled independently for one coding unit.The same syntax may be applied to gbi_wb_comb_flag.

[0135] Merge Mode

[0136] In some embodiments, the merge mode can be used to infer not only the motion information from the causal neighboring block, but also the weight index of the block. Figure 13The access order of (as shown) can be the same as specified in HEVC, where spatial blocks are accessed in the order of left block, top block, top right block, bottom left block, and top right block, and temporal blocks are accessed in the order of bottom right block and center block. In some embodiments, up to five merge candidates can be constructed by utilizing up to four blocks from the spatial blocks and up to one block from the temporal blocks. Given a merge candidate, the GBi prediction process specified in equations (3), (8), (13), or (14) can be applied. It should be noted that the weight index does not need to be signaled as it can be inferred from the weight information of the selected merge candidate.

[0137] On the JEM platform, an additional merge mode called Advanced Temporal Motion Vector Prediction (ATMVP) is available. In some embodiments of the present disclosure, ATMVP can be combined with GBi prediction. In ATMVP, the motion information of each 4x4 unit within a CU can be derived from the motion field of the temporal reference picture. In exemplary embodiments using ATMVP, when GBi prediction mode is enabled (e.g., when extra_number_of_weights is greater than 0), the weight index of each 4x4 unit can also be inferred from the weight index of the corresponding temporal block within the temporal reference picture.

[0138] Bidirectional optical flow

[0139] In some embodiments, the weight values ​​of GBi can be applied to a bidirectional optical flow (BIO) model. Based on the motion-compensated prediction signals (P0[x+v0] and P1[x+v1]), BIO can estimate the offset value o BIO [x], to reduce the difference between two corresponding samples in L0 and L1 (according to their spatial vertical and horizontal gradient values). In order to combine this offset value with the GBi prediction, formula (3) can be reformulated as:

[0140] P[x]=(1-w1)*P0[x+v0]+w1*P1[x+v1]+o BIO [x], (15)

[0141] where w1 is the weight value used to perform GBi prediction. This offset value can also be applied as an additional offset to other GBi deformations after the prediction signals in P0 and P1 are scaled, similar to formulas (8), (13) or (14).

[0142] Affine prediction

[0143] In an exemplary embodiment, GBi prediction can be combined with affine prediction in a manner similar to extending traditional bi-prediction. However, there are differences in the underlying processing units used to perform motion compensation. Affine prediction is a model-based motion field derivation technique for forming a fine-grained motion field representation for a PU, where the motion field representation for each 4x4 unit can be derived based on unidirectional or bidirectional conversion motion vectors and given model parameters. Because all motion vectors point to the same reference picture, there is no need to adjust the weight values ​​to accommodate each 4x4 unit. Therefore, the weight values ​​can be shared between each unit and only one weight index for the PU can be signaled. By utilizing the motion vectors and weight values ​​at the 4x4 unit, GBi is performed on a unit-by-unit basis, so that the same formulas (3), (8), (13) and (14) can be directly applied without any changes.

[0144] Overlapped Block Motion Compensation

[0145] Overlapped Block Motion Compensation (OBMC) is a method that provides a prediction of the intensity value of a sample based on a motion compensation signal derived from the sample's own motion vector and those within its causal neighborhood. In exemplary embodiments of GBi, weight values ​​may also be considered in motion compensation for OBMC. Figure 14 An example is shown in which a sub-block B1 within the current block has three motion compensated prediction blocks, each of which is formed by using motion information and a weight value from N1, N5 or B1 itself, and the final prediction signal for B1 can be a weighted average of the three motion compensated prediction blocks.

[0146] Frame rate upconversion

[0147] In some embodiments, GBi can be run in conjunction with frame rate up conversion (FRUC). Two different modes can be used for FRUC. If the current picture falls between the first reference picture in L0 and the first reference picture in L1, the dual prediction mode can be used. If the first reference picture in L0 and the first reference picture in L1 are both forward reference pictures or backward reference pictures, the single prediction mode can be used. The dual prediction case within FRUC will be discussed in detail below. In JEM, equal weights (i.e., 0.5) can be used for FRUC dual prediction. Although there may be differences in the quality of the two predictors within FRUC dual prediction, combining two predictors with unequal prediction qualities by using equal weights may be suboptimal. Due to the use of unequal weights, the use of GBi can improve the final dual prediction quality. In an exemplary embodiment, for blocks encoded using FRUC dual prediction, the weight values ​​of GBi can be derived and do not need to be signaled. For each 4x4 sub-block within the PU, WL1 Each weight value within can be evaluated separately by the MV derivation process of FRUC bi-prediction. The weight value that results in the minimum bilateral matching error (ie, the absolute difference between the two unidirectional motion compensated predictors associated with the two reference lists) for the 4x4 block can be selected.

[0148] In an exemplary embodiment, FRUC bi-prediction is a decoder-side MV derivation technique that derives MVs using bilateral matching. For each PU, a list of candidate MVs collected from causal neighboring blocks may be formed. Under the assumption of constant motion compensation, each candidate MV may be linearly projected onto the first reference picture in the other reference lists, where the scaling factor of the projection is set to the value that is proportional to the distance between the reference picture (e.g., at time t0 or t1) and the current picture (t c ) is proportional to the time distance between them. Figure 15 As an example, where v0 is the candidate MV associated with the reference list L0, v1 is calculated as v0*(t1-t c ) / (t0-t c ). Therefore, the bilateral matching error can still be calculated for each candidate MV, and the initial MV that achieves the minimum bilateral matching error can be selected from the candidate list. The initial MV can be represented as v0 INIT From this initial MVv0 INIT Starting from the pointed position, decoder-side motion estimation can be performed to find MVs within a predefined search range, and the MV that can achieve the minimum bilateral matching error can be selected as the PU-level MV. Assuming v1 is the projected MV, the optimization process can be formulated as:

[0149] Among them, FRUC dual prediction can be combined with GBi, and the search process in formula (16) can use W L1 The weight value w is reformulated as follows:

[0150]

[0151] The PU level v0 can be further refined individually by using the same bilateral matching in equation (17) for each 4x4 sub-block within the PU, as shown in equation (18):

[0152]

[0153] For W L1For each available weight value w within the PU, formula (18) can be evaluated, and the weight value that minimizes the bilateral matching error can be selected as the optimal weight. At the end of the evaluation process, each 4x4 sub-block within the PU has its own bi-prediction MV and weight value for performing general bi-prediction. The complexity of this exhaustive search method may be high because the weights and motion vectors are searched in a joint manner. In another embodiment, the search for the optimal motion vector and the optimal weight can be performed in two steps. In the first step, the motion vector of each 4x4 block can be obtained by using formula (18) and by setting w to an initial value (e.g., w=0.5). In the second step, the optimal weight can be searched given the optimal motion vector.

[0154] In yet another embodiment, to improve motion search accuracy, three steps may be applied. In the first step, the initial weights may be calculated by using the initial motion vector v0 INIT The initial optimal weight can be expressed as w INIT In the second step, the motion vector of each 4x4 block can be obtained by using formula (18) and setting w to w INIT In the third step, the final optimal weight can be searched given the optimal motion vector.

[0155] By formulas (17) and (18), the goal is to minimize the difference between the two weighted predictors associated with the two reference lists, respectively. Negative weights may be inappropriate for this purpose. In one embodiment, the FRUC-based GBi mode will only evaluate weight values ​​greater than zero. To reduce complexity, the calculation of the sum of absolute differences can be performed by using a portion of the samples within each sub-block. For example, the sum of absolute differences can be calculated by using samples only at even-numbered rows and columns (or, alternatively, odd-numbered rows and columns).

[0156] GBi prediction search strategy

[0157] Initial reference list for bipredictive search

[0158] The following describes a method for improving the prediction performance of GBi by determining which of the two reference lists should be searched first in the motion estimation (ME) stage of bi-prediction. As with conventional bi-prediction, there are two motion vectors associated with reference list L0 and reference list L1, respectively, that need to be determined to minimize the ME stage cost, namely:

[0159] Cost(t i ,u j )=∑ x |I[x]-P[x]|+λ*Bits(t i ,u j,weight index) (19)

[0160] Where I[x] is the original signal of sample x at position x in the current picture, P[x] is the predicted signal of GBi, and t i and u j are motion vectors pointing to the i-th reference picture in L0 and the j-th reference picture in L1, respectively. λ is the Lagrangian parameter used in the ME stage, and the Bits(·) function estimates the number of bits used to encode the input variable. Each of equations (3), (8), (13), and (14) can be applied to replace P[x] in equation (19). For the purpose of simplicity of explanation, equation (3) can be considered as an example in the following process. Therefore, the cost function in equation (19) can be rewritten as:

[0161] Cost(t i ,u j )=∑ x |I[x]-(1-w1)*P0[x+t i ]-w1*P1[x+u j ]|+λ*Bits(t i ,u j ,weightindex) (20)

[0162] Since there are two parameters (t i and u j ) can be determined using an iterative process. The first such process can follow the following rules:

[0163] 1. Utilize Optimize the best movement within

[0164] 2. Utilize Optimize the best movement within

[0165] 3. Repeat steps 1 and 2 until t i and u j No more changes or the maximum number of iterations is reached. A second exemplary iterative process may be performed as follows:

[0166] 1. Utilize Optimize the best movement within

[0167] 2. Utilize Optimize the best movement within

[0168] 3. Repeat steps 1 and 2 until u j and t iNo longer changes or the maximum number of iterations is reached.

[0169] The choice of which iteration process to use depends solely on t i and u j The ME phase cost is:

[0170] The cost function of the ME stage can be as follows:

[0171] Cost(t i )=∑ x |I[x]-P0[x+t i ]|+λ*Bits(t i ). (twenty two)

[0172] Cost(u j )=∑ x |I[x]-P1[x+u j ]|+λ*Bits(u j ). (twenty three) However, this initialization process may not be optimal when 1-w1 is not equal to w1. A typical example is when one of the weights is extremely close to 0, such as w1=lim w→0 w, and the ME phase cost of its associated motion happens to be lower than the other one. In this case, formula (20) degenerates to: Cost(t i ,u j )=∑ x |I[x]-P0[x+t i ]|+λ*Bits(t i ,u j ,weight index). (24) Targeting u j The overhead does not contribute to the prediction signal, which ultimately leads to poor search results for GBi. In the present disclosure, the magnitude of the weight value can be used to replace formula (21), specifically:

[0173]

[0174] Binary search on weighted index

[0175] Since the number of weight values ​​to be evaluated may introduce additional complexity to the encoder, the exemplary embodiment adopts a binary search method to prune the less likely weight values ​​early in the encoding. In this search method, traditional uni-prediction (associated with 0 and 1 weights) and bi-prediction (associated with 0.5 weight) can be performed initially, and W L1The weight values ​​in A can be divided into 4 groups, that is, A=[w min , 0], B = [0, 0.5], C = [0.5, 1] ​​and D = [1, w max ]. min and w max Represents W L1 The minimum and maximum weight values ​​within , and without loss of generality, we can assume that w min <0 and w max > 1. The following rules can be applied to determine the range of possible weight values.

[0176] If w = 0 gives a better ME phase cost than w = 1, the following rules apply:

[0177] o If w = 0.5 gives a better ME phase cost than w = 0 and w = 1, then a weight set W can be formed based on the weight values ​​in B (0) .

[0178] Otherwise, W can be formed based on the weight values ​​in A (0) .

[0179] Otherwise (if w = 1 gives a better ME phase cost than w = 0), the following rules apply:

[0180] o If w = 0.5 gives a better ME phase cost than w = 0 and w = 1, then a weight set W can be formed based on the weight values ​​in C (0) .

[0181] Otherwise, W can be formed based on the weight values ​​in D (0) .

[0182] In the formation of W (0) Afterwards, w min and w max The values ​​of W (0) If W (0) Associated with A and D, we can calculate w in A separately min The ME phase cost and w in D max ME phase cost.

[0183] The iterative process is operable to keep updating W (k) , until there are more than 2 weight values ​​left in the set in the kth iteration. Assuming that the process is k iterations, the iterative process can be specified as follows:

[0184] 1. Use the closest (w min +w max ) / 2 weight value w middle Execute GBi.

[0185] 2. If w middle than w min and w max Given a better ME stage cost, we can target W (k+1) Call the recursive procedure to test [w min ,w middle ] and [w middle ,w max ], and the iterative process jumps to step 6.

[0186] 3. Otherwise if w middle than w min and w max The iterative process terminates if a worse ME stage cost is given.

[0187] 4. Otherwise if w min than w max Given a better ME stage cost, based on [w min ,w middle The weight values ​​within ] form W (k+1) , and the iterative process jumps to step 6.

[0188] 5. Otherwise (if w min than w max Given a worse ME stage cost), then based on [w middle ,w max The weight values ​​within ] form W (k+1) , and the iterative process jumps to step 6.

[0189] 6. If W (k+1) If the number of remaining weight values ​​is greater than 2, then w min and w max According to W (k+1) The maximum and minimum values ​​in are reset, and the iteration process jumps to step 1; otherwise, the iteration process terminates.

[0190] After the iterative process stops, the weight value that achieves the lowest ME stage cost among all tested values ​​can be selected to perform generalized bi-prediction.

[0191] Weight estimation for non-2Nx2N partitions

[0192] In some embodiments, after testing each weight value for the 2Nx2N partition, the best performing weight values ​​other than 0, 0.5, and 1 may be used as estimates of the optimal weight value for the non-2Nx2N partition. In some embodiments, assuming there are n unique estimates, the n unique estimates and only weight values ​​equal to 0, 0.5, and 1 may be evaluated for the non-2Nx2N partition.

[0193] Partition size estimation for non-2Nx2N partitions

[0194] In some embodiments, not all non-2Nx2N partitions are tested by the exemplary video encoder. Non-2Nx2N partitions can be divided into two subcategories: Symmetric Motion Partitions (SMPs) with 2NxN and Nx2N partition types, and Asymmetric Motion Partitions (AMPs) with 2NxnU, 2NxnD, nLx2N, and nRx2N partition types. If the rate-distortion (RD) cost of a partition within an SMP is less than the distortion cost of 2Nx2N, some partition types within the AMP may be evaluated at the encoder. The decision on which partition types to test within the AMP may depend on which of 2NxN and Nx2N exhibits better performance in terms of RD cost. If the rate-distortion cost of 2NxN is less, partition types 2NxnU and 2NxnD may be further examined; otherwise (if the cost of Nx2N is less), partition types nLx2N and nRx2N may be further examined.

[0195] Fast parameter estimation for multi-channel coding

[0196] In an exemplary embodiment using a multi-channel encoder, the prediction parameters (e.g., block motion and weight values) obtained from the optimization of the early coding channel can be used as initial parameter estimates at subsequent coding channels. In this encoder, the coding blocks partitioned from the picture can be predicted and encoded twice or more, which will eventually lead to a significant increase in coding complexity. One technique to reduce this complexity is to cache the optimized prediction parameters from the initial coding channel and use them as initial parameter estimates for further refinement in subsequent coding channels. For example, if the inter prediction mode happens to be the best mode at the initial channel, the encoder can only evaluate the inter prediction mode at the remaining coding channels. In some embodiments, caching can be performed for the prediction parameters related to GBi, such as for W L1 The weight values ​​within the buffer are selected, the bi-predictive MVs associated with the selected weight values, the IC flag, the OBMC flag, the integer motion vector (IMV) flag, and the coded block flag (CBF). In such embodiments, the values ​​of these cached parameters can be reused or refined in subsequent coding passes. More specifically, when the bi-predictive MVs described above are employed, these MVs can be used as initial search positions for the bi-predictive search. They can then be refined in the motion estimation stage and subsequently used as initial search positions for the next coding pass.

[0197] Exemplary Bitstream Communication Architecture

[0198] Figure 161000 is a schematic diagram illustrating an example of a coded bitstream structure. A coded bitstream 1000 includes multiple Network Abstraction Layer (NAL) units 1001. NAL units may contain coded sample data (e.g., coded slices 1006) or high-level syntax metadata (e.g., parameter set data, slice header data 1005, or supplemental enhancement information data 1007 (which may be referred to as SEI messages)). A parameter set is a high-level syntax structure containing basic syntax elements, which can be applied to multiple bitstream layers (e.g., video parameter set 1002 (VPS)), to a coded video sequence within a layer (e.g., sequence parameter set 1003 (SPS)), or to multiple coded pictures within a coded video sequence (e.g., picture parameter set 1004 (PPS)). The parameter set can be transmitted along with the coded pictures in the video bitstream or via other means (including out-of-band transmission using a reliable channel, hard coding, etc.). The slice header 1005 is also a high-level syntax structure that can contain relatively small amounts of picture-related information or information that is only relevant to certain slices or picture types. The SEI message 1007 carries information that is not necessarily required for the decoding process but may be used for various other purposes, such as picture output timing or display and loss detection and concealment.

[0199] Figure 17 13 is a diagram illustrating an example of a communication system. The communication system 1300 may include an encoder 1302, a communication network 1304, and a decoder 1306. The encoder 1302 may communicate with the network 1304 via a connection 1308, which may be a wired connection or a wireless connection. Figure 1 The encoder 1302 may include a single-layer codec (e.g., Figure 1 ) or multi-layer codec. For example, encoder 1302 can be a multi-layer (e.g., two-layer) scalable coding system that supports picture-level ILP. Decoder 1306 can communicate with network 1304 via connection 1310, which can be a wired connection or a wireless connection. Decoder 1306 can communicate with Figure 2 The decoder 1306 may include a single-layer codec (e.g. Figure 2 ) or a multi-layer codec. As an example, the decoder 1306 may be a multi-layer (eg, two-layer) scalable decoding system supporting picture-level ILP.

[0200] The encoder 1302 and / or decoder 1306 may be incorporated into various wired communication devices and / or wireless transmit / receive units (WTRUs), such as, but not limited to, digital televisions, wireless broadcast systems, network components / terminals, servers (e.g., content or network servers (e.g., Hypertext Transfer Protocol (HTTP) servers)), personal digital assistants (PDAs), laptop or desktop computers, tablet computers, digital cameras, digital recording devices, video gaming devices, video game consoles, cellular or satellite radiotelephones, and / or digital media players.

[0201] The communication network 1304 may be a communication network of any appropriate type. For example, the communication network 1304 may be a multiple access system that provides content (e.g., voice, data, video, messaging, broadcast, etc.) to multiple wireless users. The communication network 1304 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. By way of example, the communication network 1304 may utilize one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), and / or single carrier FDMA (SC-FDMA), etc. The communication network 1304 may include multiple connected communication networks. The communication network 1304 may include the Internet and / or one or more private commercial networks, such as a cellular network, a WiFi hotspot, and / or an Internet service provider (ISP) network, etc.

[0202] Figure 18 is a system diagram of an example WTRU that may implement an encoder or decoder as described herein. As shown, the example WTRU 1202 may include a processor 1218, a transceiver 1220, a transmit / receive element 1222, a speaker / microphone 1224, a keyboard or numeric keypad 1226, a display / touchpad 1228, non-removable memory 1230, removable memory 1232, a power supply 1234, a global positioning system (GPS) chipset 1236, and / or other peripherals 1238. It should be understood that the WTRU 1202 may also include any sub-combination of the foregoing components while remaining consistent with the embodiment. Further, a terminal incorporating an encoder (e.g., encoder 100) and / or a decoder (e.g., decoder 200) may be included in Figure 18 The WTRU 1202 is depicted and referenced herein. Figure 18 Some or all of the components described for WTRU 1202.

[0203] The processor 1218 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, and the like. The processor 1218 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 1500 to operate in a wired and / or wireless environment. The processor 1218 may be coupled to the transceiver 1220, which in turn may be coupled to the transmit / receive element 1222. Although Figure 18 The processor 1218 and the transceiver 1220 are depicted as separate components, however, it should be understood that the processor 118 and the transceiver 1220 may also be integrated together in an electronic package and / or chip.

[0204] The transmit / receive component 1222 can be configured to transmit and / or receive signals to and from another terminal via the air interface 1215. For example, in one or more embodiments, the transmit / receive component 1222 can be an antenna configured to transmit and / or receive RF signals. As an example, in one or more embodiments, the transmit / receive component 1222 can be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In one or more embodiments, the transmit / receive component 1222 can be configured to transmit and / or receive RF and light signals. It should be understood that the transmit / receive component 1222 can be configured to transmit and / or receive any combination of wireless signals.

[0205] In addition, although Figure 18 1215, the WTRU 1202 may include any number of transmit / receive components 1222. More specifically, the WTRU 1202 may employ MIMO technology. Thus, in one embodiment, the WTRU 1202 may include two or more transmit / receive components 1222 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 1215.

[0206] The transceiver 1220 may be configured to modulate signals to be transmitted by the transmit / receive element 1222 and / or demodulate signals received by the transmit / receive element 1222. As described above, the WTRU 1202 may have multi-mode capabilities. Thus, the transceiver 1220 may include multiple transceivers that allow the WTRU 1202 to communicate via multiple RATs, such as UTRA and IEEE 802.11.

[0207] The processor 1218 of the WTRU 1202 may be coupled to a speaker / microphone 1224, a keypad 1226, and / or a display / touchpad 1228 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit) and may receive user input data from these components. The processor 1218 may also output user data to the speaker / microphone 1224, the keypad 1226, and / or the display / touchpad 1228. Furthermore, the processor 1218 may access information from, and store information in, any suitable memory, such as the non-removable memory 1230 and / or the removable memory 1232. The non-removable memory 1230 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 1232 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In one or more embodiments, the processor 1218 may access information from, and store data in, memory that is not physically located on the WTRU 1202, such as on a server or a home computer (not shown).

[0208] The processor 1218 may receive power from the power source 1234 and may be configured to distribute and / or control power to the other components in the WTRU 1202. The power source 1234 may be any suitable device for powering the WTRU 1202. For example, the power source 1234 may include one or more dry cell batteries (e.g., nickel-cadmium (Ni-Cd), nickel-zinc (Ni-Zn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0209] The processor 1218 may also be coupled to a GPS chipset 1236, which may be configured to provide location information (e.g., longitude and latitude) related to the current location of the WTRU 1202. In addition to or in lieu of the information from the GPS chipset 1236, the WTRU 1202 may receive location information from a terminal (e.g., a base station) via the air interface 1215 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 1202 may acquire location information using any suitable positioning method while remaining consistent with the embodiments.

[0210] The processor 1218 may be further coupled to other peripherals 1238, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 1238 may include an accelerometer, an orientation sensor, a motion sensor, a proximity sensor, an electronic compass, a satellite transceiver, a digital camera and / or video recorder (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, modules, FM radio units, and software modules (such as digital music players, media players, video game console modules, and Internet browsers, etc.).

[0211] By way of example, the WTRU 1202 may be configured to transmit and / or receive wireless signals and may comprise a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a tablet, a personal computer, a wireless sensor, a consumer electronic product, or any other terminal capable of receiving and processing compressed video communications.

[0212] The WTRU 1202 and / or the communication network (e.g., the communication network 804) may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 1215. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink Packet Access (HSDPA) and / or High Speed ​​Uplink Packet Access (HSUPA). The WTRU 1202 and / or the communication network (e.g., the communication network 804) may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may use Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) to establish the air interface 1215.

[0213] The WTRU 1202 and / or the communication network (e.g., the communication network 804) may implement a radio technology such as IEEE 802.16 (e.g., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 IX, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), and GSM EDGE (GERAN), etc. The WTRU 1202 and / or the communication network (e.g., the communication network 804) may implement a radio technology such as IEEE 802.11 or IEEE 802.15.

[0214] It should be noted that the different hardware components of one or more of the described embodiments are referred to as “modules”, which refer to “modules” for performing (i.e., implementing, running, etc.) the different functions described herein in conjunction with the corresponding modules. The modules used herein include hardware that is deemed suitable for use with the specified implementation by a person skilled in the relevant art (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more memory devices). Each module described may also include instructions that can be executed to implement one or more functions described as being performed by the corresponding module, and it should be noted that these instructions may take the form of or include hardware (i.e., hard-wired) instructions, firmware instructions, and / or software instructions, and may be stored in any appropriate non-transitory computer-readable medium or media, such as those commonly referred to as RAM, ROM, and the like.

[0215] Although features and elements are described above in specific combinations, one skilled in the art will recognize that each feature or element can be used alone or in any combination with other features and elements. In addition, the methods described herein can be implemented in a computer program, software, or firmware that is incorporated into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electrical signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, buffer memory, semiconductor storage devices, magnetic media such as internal hard disk enclosures and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital versatile discs (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any computer host.

Claims

1. A method for encoding a video, comprising: For at least a current block in a current picture of a video, obtaining at least a first weight from a predetermined set of no more than five bi-prediction weights; predicting the current block as a weighted sum of a first reference block in a first reference picture and a second reference block in a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by the second weight; as well as Block-level information identifying at least the first weight in the set of bi-predictive weights is encoded in a bitstream using a truncated unary code, wherein the truncated unary code assigns a codeword to each of the weights in the predetermined set of bi-predictive weights, and wherein the predetermined set of bi-predictive weights, arranged in order of increasing codeword length, includes ascending sets of first weights interleaved with descending sets of second weights. The method of claim 1 , wherein predicting the current block is performed by using affine prediction.

3. The method of claim 1, wherein the encoding comprises entropy encoding of a truncated unary code binarization of the first weight.

4. The method of claim 1, wherein the predetermined set of bi-prediction weights is a set of five bi-prediction weights.

5. A device for decoding a video, comprising a processor configured to at least perform: decoding, for at least a current block in a current picture of a video, block-level information from a bitstream using a truncated unary code, the block-level information identifying at least a first weight from among a predetermined set of no more than five bi-predictive weights, wherein the truncated unary code assigns a codeword to each of the weights in the predetermined set of bi-predictive weights, and wherein the predetermined set of bi-predictive weights, arranged in order of increasing codeword length, comprises ascending first sets of weights interleaved with descending second sets of weights; and The current block is predicted as a weighted sum of a first reference block in a first reference picture and a second reference block in a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by a second weight. The apparatus of claim 5 , wherein the second weight is derived from the first weight. The apparatus of claim 5 , wherein predicting the current block is performed using affine prediction.

8. The apparatus of claim 5, wherein the decoding comprises entropy decoding of a truncated unary code binarization of the first weight.

9. The apparatus according to claim 5, wherein The processor is further configured to perform: For at least a subsequently coded block in the current picture, decoding from the bitstream information identifying the current block as a selected merge candidate; and The subsequently coded block is predicted by weighted bi-prediction using at least the first weight.

10. The apparatus of claim 5, wherein the predetermined set of bi-prediction weights is a set of five bi-prediction weights.

11. A method for decoding a video, comprising: decoding, for at least a current block in a current picture of a video, block-level information from a bitstream using a truncated unary code, the block-level information identifying at least a first weight from among a predetermined set of no more than five bi-predictive weights, wherein the truncated unary code assigns a codeword to each of the weights in the predetermined set of bi-predictive weights, and wherein the predetermined set of bi-predictive weights, arranged in order of increasing codeword length, comprises ascending sets of first weights interleaved with descending sets of second weights; as well as The current block is predicted as a weighted sum of a first reference block in a first reference picture and a second reference block in a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by a second weight.

12. The method according to claim 11, wherein The second weight is derived from the first weight. The method of claim 11 , wherein predicting the current block is performed by using affine prediction.

14. The method of claim 11, wherein the decoding comprises entropy decoding of a truncated unary code binarization of the first weight.

15. The method according to claim 11, further comprising: decoding, for at least a subsequently coded block in the current picture, information identifying the current block as a selected merge candidate from the bitstream; as well as The subsequently coded block is predicted by weighted bi-prediction using at least the first weight.

16. The method of claim 11, wherein the predetermined set of bi-prediction weights is a set of five bi-prediction weights.

17. An apparatus for encoding a video, comprising a processor configured to at least perform: For at least a current block in a current picture of a video, obtaining at least a first weight from a predetermined set of no more than five bi-prediction weights; predicting the current block as a weighted sum of a first reference block in a first reference picture and a second reference block in a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by the second weight; as well as Block-level information identifying at least the first weight in the set of bi-predictive weights is encoded in a bitstream using a truncated unary code, wherein the truncated unary code assigns a codeword to each of the weights in the predetermined set of bi-predictive weights, and wherein the predetermined set of bi-predictive weights, arranged in order of increasing codeword length, includes ascending sets of first weights interleaved with descending sets of second weights. The apparatus of claim 17 , wherein predicting the current block is performed by using affine prediction.

19. The apparatus of claim 17, wherein the encoding comprises entropy encoding of a truncated unary binarization of the first weight.

20. The apparatus of claim 17, wherein the predetermined set of bi-prediction weights is a set of five bi-prediction weights.

21. A computer-readable medium comprising instructions for causing one or more processors to perform the method of any one of claims 1-4 or 11-16.

Images