A system and method for generalized multi-hypothessis prediction for video coding.

Abstract

To provide systems and methods for video coding using generalized bi-prediction.SOLUTION: A video encoding method comprises: for at least a current block in a current image, encoding block-level information identifying a first weight and a second weight, where at least one of the first and second weights has a value not equal to 0, 0.5 or 1; and predicting the current block as a weighted sum of a first reference block in a first reference image and a second reference block in a second reference image, where the first reference block is weighted by the first weight and the second reference block is weighted by the second weight. Encoding the block-level information includes mapping a weight index to a codeword using a truncated unary code, and entropy-encoding the codeword in a bitstream.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a system and method for generalized multiple hypothesis prediction for video coding. [Background technology]

[0002] Cross-reference of related applications This application is a non-provisional application of the following U.S. provisional patent applications, namely, No. 62 / 336227 entitled "Systems and Methods for Generalized Multi-Hypothesis Prediction for Video Coding," filed on 13 May 2016, No. 62 / 342772 entitled "Systems and Methods for Generalized Multi-Hypothesis Prediction for Video Coding," filed on 27 May 2016, No. 62 / 399234 entitled "Systems and Methods for Generalized Multi-Hypothesis Prediction for Video Coding," filed on 23 September 2016, and No. 62 / 415187 entitled "Systems and Methods for Generalized Multi-Hypothesis Prediction for Video Coding," filed on 31 October 2016, and claims benefits from them under Section 119(c) of the U.S. Patent Act. All of these applications, in their entirety, are incorporated herein by reference.

[0003] Video coding systems are widely used to compress digital video signals to reduce the storage requirements and / or transmission bandwidth of such signals. Among the various types of video coding systems, including block-based, wavelet-based, and object-based systems, block-based hybrid video coding systems are the most widely used and deployed today. Examples of block-based video coding systems include international video coding standards such as MPEG-1 / 2 / 4 part 2, H.264 / MPEG-4 part 10 AVC, VC-1, and the latest video coding standard called High Efficiency Video Coding (HEVC), developed by the ITU-T / SG16 / Q.6 / VCEG and ISO / IEC / MPEG's JCT-VC (Joint Collaborative Team on Video Coding).

[0004] Video encoded using block-based coding accounts for a significant portion of electronically transmitted data, for example, over the internet. It is desirable to increase the efficiency of video compression so that high-quality video content can be stored and transmitted using fewer bits. [Overview of the project]

[0005] In exemplary embodiments, a system and method for performing generalized bi-prediction (GBi) are described. The exemplary method includes the step of encoding and decoding (collectively “encoded”) a video including a plurality of images, including a current image, a first reference image, and a second reference image, each image including a plurality of blocks. In the exemplary method, for at least the current block in the current image, a block-level index is encoded that identifies the first weight and the second weight from a set of weights, wherein at least one of the weights in the set of weights has a value not equal to 0, 0.5, or 1. The current block is predicted as a weighted sum of the first reference block in the first reference image and the second reference block in the second reference image, where the first reference block is weighted by the first weight and the second block is weighted by the second weight.

[0006] In some embodiments (or for some blocks), block-level information identifying a first weight and a second weight for a given block can be encoded by means other than encoding an index for that block. For example, a block can be encoded in merge mode. In such a case, the block-level information can be information identifying a candidate block from a plurality of merge candidate blocks. In that case, the first weight and the second weight can be identified based on the weights used to encode the identified candidate block.

[0007] In some embodiments, the first and second reference blocks are further scaled by at least one scaling factor that is signaled within the bitstream for the current image.

[0008] In some embodiments, the set of weights is encoded within the bitstream, allowing different weight sets to be adapted for use in different slices, images, or sequences. In other embodiments, the set of weights is predetermined. In some embodiments, only one of two weights is signaled within the bitstream, and the other weight is derived by subtracting the signaled weight from 1.

[0009] In some embodiments, codewords are assigned to each weight, and the weights are identified using the corresponding codewords. The assignment of codewords to weights can be a predetermined assignment, or the assignment can be adapted based on weights used in previously encoded blocks.

[0010] Exemplary encoders and decoders for performing generalized biprediction are also described herein.

[0011] The systems and methods described herein provide novel techniques for predicting blocks of sampled values. Such techniques can be used by both encoders and decoders. Block prediction results in blocks of sampled values ​​that can be subtracted from the original input blocks to determine the residuals to be encoded in the bitstream in the encoding method. In the decoding method, the residuals can be added to the predicted blocks to decode from the bitstream and obtain reconstructed blocks that are the same as, or approximately the same as, the original input blocks. Thus, prediction methods such as those described herein improve the operation of video encoders and decoders by reducing the number of bits required to encode and decode video, at least in some implementations. Further exemplary benefits of prediction methods to the operation of video encoders and decoders are provided in the detailed description. [Brief explanation of the drawing]

[0012] A more detailed understanding can be obtained from the following explanation, which is presented as an example, along with the attached diagram, which is first briefly explained below.

[0013] [Figure 1] This is a functional block diagram illustrating an example of a block-based video encoder. [Figure 2] This is a functional block diagram illustrating an example of a block-based video decoder. [Figure 3] This is a schematic diagram of a prediction using template TC and associated prediction blocks T0 and T1. [Figure 4] This graph provides a schematic diagram of changes in illuminance over time. [Figure 5] This is a functional block diagram showing a video encoder configured to use generalized biprediction according to several embodiments. [Figure 6] This is a functional block diagram of an exemplary generalized dual-prediction module for use in a video encoder. [Figure 7] This is a schematic diagram illustrating an exemplary decoder-side derivation of implicit weight values ​​for use in generalized biprediction. [Figure 8] This is a schematic diagram of a tree structure for binary evolution of weight_idx, where each circle represents a bit that is signaled. [Figure 9] This is a functional block diagram showing a video decoder configured to use generalized biprediction according to several embodiments. [Figure 10] This is a functional block diagram of an exemplary generalized dual-prediction module for use in a video decoder. [Figure 11A] This is a schematic diagram of a fixed assignment method among codeword assignment methods. [Figure 11B] This is a schematic diagram of an alternative codeword assignment method. [Figure 12A]This is a schematic diagram showing the weight value field in an example of block-adaptive codeword assignment. [Figure 12B] This is a schematic diagram showing an updated codeword assignment from a fixed assignment, which is an example of block-adaptive codeword assignment. [Figure 13] This is a schematic diagram of potential merge locations. [Figure 14] This is a schematic diagram of an example of superimposed block motion compensation (OBMC), where m is the basic processing unit for executing OBMC, N1 to N8 are causally related neighboring subblocks, and B1 to B7 are subblocks within the current block. [Figure 15] This figure illustrates an example of frame rate upconversion (FRUC), where v0 is a given motion vector corresponding to a reference list L0, and v1 is the scaled MV based on v0 and time distance. [Figure 16] This figure shows an example of an encoded bitstream structure. [Figure 17] This is a diagram illustrating an exemplary communication system. [Figure 18] This figure shows an exemplary wireless transceiver unit (WTRU) that can be used as an encoder or decoder in some embodiments. [Modes for carrying out the invention]

[0014] Block-based encoding Figure 1 is a block diagram of a general-purpose block-based hybrid video encoding system 100. The input video signal 102 is processed block by block. In HEVC, an extended block size (called an "encoded unit" or CU) is used to efficiently compress high-resolution (1080p or higher) video signals. In HEVC, a CU can be up to 64x64 pixels. A CU can be further divided into prediction units or PUs, to which a different prediction method is applied. For each input video block (MB or CU), spatial prediction (160) and / or temporal prediction (162) can be performed. Spatial prediction (or "intra-prediction") predicts the current video block using pixels from already encoded neighboring blocks within the same video image / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also called "inter-prediction" or "motion-compensated prediction") predicts the current video block using pixels from already encoded video images. Temporal prediction reduces the temporal redundancy inherent in the video signal. A temporal prediction signal for a given video block can be signaled by one or more motion vectors indicating the amount and direction of movement between the current block and its reference block. Also, if multiple reference images are supported (as in the case of recent video encoding standards such as H.264 / AVC or HEVC), a reference index of the reference image can also be transmitted for each video block. The reference index is used to identify which reference image in the reference image store (164) the temporal prediction signal originates from. After spatial and / or temporal prediction, a mode determination block (180) in the encoder selects the best prediction mode, for example, based on a rate-distortion optimization method. The prediction block is then subtracted from the current video block (116), and the prediction residual is decorrelated using a transform (104) and quantized (106) to achieve the target bitrate.The quantized residual coefficients are inversely quantized (110) and inversely transformed (112) to form reconstructed residuals, which are then added back to the prediction blocks (126) to form reconstructed video blocks. Furthermore, in-loop filtering such as deblocking filters and adaptive loop filters can be applied to the reconstructed video blocks (166), which are then placed into a reference image store (164) and used to encode further video blocks. To form the output video bitstream 120, the encoding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to an entropy encoding unit (108), where they are further compressed and packed to form the bitstream.

[0015] Block-based decoding Figure 2 gives an overall block diagram of the block-based video decoder 200. The video bitstream 202 is unpacked and entropi-decoded in the entropi-decoding unit 208. The encoding mode and prediction information are sent to the spatial prediction unit 260 (in the case of intra-coding) or to the temporal prediction unit 262 (in the case of inter-coding) to form a prediction block. The residual transformation coefficients are sent to the inverse quantization unit 210 and the inverse transformation unit 212 to reconstruct the residual block. The prediction block and residual block are then summed in 226. The reconstructed block can further pass through in-loop filtering and is then stored in the reference image store 264. The reconstructed video in the reference image store is then sent out to drive a display device and is also used to predict further video blocks.

[0016] In modern video codecs, bidirectional motion-compensated prediction (MCP) is widely adopted in most state-of-the-art video codecs due to its high efficiency in eliminating temporal redundancy by utilizing the temporal correlation between images. However, a bi-prediction signal is formed simply by combining two uni-prediction signals using weight values ​​equal to 0.5. This is not always optimal for combining uni-prediction signals, especially under certain conditions where illuminance changes rapidly when switching from one reference image to another. Therefore, several prediction techniques have been developed that aim to compensate for illuminance fluctuations over time by applying several global or local weights and offset values ​​to each of the sample values ​​in the reference images.

[0017] Weighted biprediction Weighted biprediction is an encoding tool primarily used to compensate for changes in illumination over time, such as fading transitions, when performing motion compensation. For each slice, two sets of values—a multiplicative weight value and an additive offset value—are explicitly specified and applied separately to the motion compensation prediction, one at a time, for each reference list. This technique is most effective when illumination changes linearly and uniformly across all images.

[0018] Local illumination compensation In local illuminance compensation, parameters (two pairs of multiplicative weight values ​​and additive offset values) are fitted block by block. Unlike weighted biprediction, which shows these parameters at the slice level, this technique uses a template (T C The approach involves fitting the optimal parameters to the illuminance changes between the reconstructed signal and the template's predicted signals (T0 and T1) (see Figure 3). The resulting parameters are (for the first pair of weight and offset values) T C The illuminance difference between and T0, and (for the second pair of weight and offset values) T CThe illuminance difference between and T1 is optimized by minimizing them separately. Then, the same motion compensation process as for the weighted biprediction is performed using the derived parameters.

[0019] Effects of changes in illuminance Changes in illuminance over space and time can seriously affect the performance of motion-compensated prediction. As can be seen in Figure 4, when illuminance decreases along the time direction, motion-compensated prediction does not provide good performance. For example, if a sample of an object moves over a time period from t-3 to t, and the intensity value of this sample decreases along its motion trajectory, v t-3 from v t It changes to. Assuming that this sample will be predicted in the t-th image, its predicted value is v t-3 from v t-1 This can lead to limitations and, consequently, poor motion-compensated predictions. The aforementioned techniques of weighted biprediction and local illuminance compensation may not adequately address this problem. Weighted biprediction may fail because illuminance can fluctuate wildly within an image. Local illuminance compensation sometimes produces poor estimates of weights and offset values ​​due to low illuminance correlations between blocks and their associated template blocks. These examples demonstrate that global and template-based local descriptions are insufficient to represent illuminance fluctuations across space and time.

[0020] Exemplary Embodiments The exemplary embodiments described herein can improve the prediction efficiency for weighted motion-compensated predictions. In some embodiments, systems and methods for generalized multiple-hypothesis prediction are proposed that use block-level weight values ​​to linearly combine motion-compensated predictions and multiple-hypothesis prediction signals. In some embodiments, a generalized dual-prediction framework using weight values ​​is described. In some embodiments, a finite set of weights is used at the sequence, image, or slice level, and a construction process for the set of weights is described. In some embodiments, the weight values ​​are determined based on a given set of weights and optimized considering the signals of the current block and its reference block. An exemplary encoding method for signaling weight values ​​is described. An exemplary encoder search criterion for a motion estimation process for proposed predictions is described, and the proposed prediction process, combined with the disclosed temporal prediction techniques, is described.

[0021] This disclosure describes systems and methods for temporal forecasting using generalized multiple hypothesis forecasting. Exemplary encoders and decoders using generalized biprediction are described with reference to Figures 5 and 9. The systems and methods disclosed herein are organized into sections as follows: Section “Generalized Multiple Hypothesis Forecasting” describes exemplary embodiments using generalized multiple hypothesis forecasting. Section “Generalized Biprediction” discloses an exemplary framework and forecasting process for generalized biprediction. Sections “Weight Set Construction” and “Weight Index Coding” describe an exemplary construction process for a weight set and an exemplary technique for signaling weight selection within this set, respectively. Section “Extensions to Advanced Temporal Forecasting Techniques” describes systems and methods for combining the exemplary proposed forecasting methods with advanced interprediction techniques, including local illuminance compensation and weighted biprediction, merge mode, superimposed block motion compensation, affine forecasting, biprediction optical flow, and a decoder-side motion vector derivation technique called frame rate upconversion biprediction. The section "GBi Predictive Search Strategies" describes an exemplary encoder-limited method for improving the efficiency of the exemplary prediction method.

[0022] Generalized multiple hypothesis prediction The exemplary systems and methods described herein utilize generalized multiple hypothesis prediction. Generalized multiple hypothesis prediction can be described as a generalized form of multiple hypothesis prediction that provides an estimated value of a pixel's intensity value based on a linear combination of multiple motion compensated prediction signals. Generalized multiple hypothesis prediction can take advantage of their benefits by combining multiple predictions with different qualities together. To reach an accurate estimated value, the motion compensated prediction signals can be processed (e.g., gamma correction, local illumination correction, dynamic range correction) through a predefined function f(●) and then linearly combined. Generalized multiple hypothesis prediction can be described with reference to Equation (1),

[0023] [Number]

[0024] where P[x] represents the resulting prediction signal of sample x placed at image position x, w i represents the weight value applied to the i-th motion hypothesis from the i-th reference image, P i [x + v i is the motion compensated prediction signal of x using the motion vector (MV) v i and n is the total number of motion hypotheses.

[0025] One factor to consider with respect to motion compensation prediction is how the accuracy of the motion field is balanced with the required motion overhead to achieve maximum rate-distortion performance. A more accurate motion field implies better prediction, however, the required motion overhead can sometimes outweigh the benefits of improved prediction accuracy. Therefore, in exemplary embodiments, the proposed video encoder can adaptively switch between different numbers n of motion hypotheses, and the n value that provides optimal rate-distortion performance is found for each respective PU. Since most modern video coding standards commonly use two motion hypotheses, the value n=2 is chosen as an example in the following sections to facilitate the explanation of how generalized multiple-hypothesis prediction works, although other values ​​of n can be used alternatively. For ease of understanding of exemplary embodiments, equation f(●) is treated as an identity function and is therefore not explicitly explained. The application of the systems and methods disclosed herein to cases where f(●) is not an identity function will be apparent to those skilled in the art in light of this disclosure.

[0026] Generalized biprediction The term Generalized Bi-Prediction (GBi) is used herein to refer to a special case of generalized multiple-hypothesis prediction in which the number of motion hypotheses is limited to two, i.e., n=2. In this case, the predicted signal at sample x, given by equation (1), is: P[x]=w0*P0[x+v0]+w1*P1[x+v1] (2) This can be simplified to (w0,w1)=(1,0), where w0 and w1 are two weight values ​​shared across all samples in a block. Based on this equation, a wide variety of prediction signals can be generated by adjusting the weight values ​​w0 and w1. Several configurations for w0 and w1 can yield the same predictions as conventional single and double predictions, such as (w0,w1)=(1,0) for single prediction using reference list L0, (w0,w1)=(0,1) for single prediction using reference list L1, and (w0,w1)=(0.5,0.5) for double prediction using two reference lists. In the cases of (1,0) and (0,1), only one set of motion information is signaled because the other set, associated with a weight value equal to zero, has no effect on the prediction signal P[x].

[0027] In particular, flexibility in the values ​​of w0 and w1 at high levels of precision can lead to a cost in the form of high signaling overhead. To reduce signaling overhead, in some embodiments, a unit gain constraint w0 + w1 = 1 is applied, and therefore, for GBi-encoded PUs, only one weight value per block is explicitly shown. To further reduce the overhead of weight signaling, the weight values ​​may be signaled at the CU level instead of the PU level. For the sake of clarity, in the description of this disclosure, w1 is signaled, and therefore equation (2) is: P[x]=(1-w1)*P0[x+v0]+w1*P1[x+v1] (3) As shown above, it can be further simplified. In an exemplary embodiment, in order to further limit the signaling overhead, the frequently used weight values ​​are (hereafter, W L1 It can be placed in a set called (called W), so each weight value is W L1 This can be indicated by a limited range of index values, weight_idx, which point to which entry within the system it occupies.

[0028] In exemplary embodiments, generalized biprediction does not introduce additional decoding overhead to support the generation of a weighted average of two reference blocks. Since most modern video standards (e.g., AVC, HEVC) support weighted biprediction, the same prediction module can be adapted for use in GBi prediction. In exemplary embodiments, generalized biprediction can be applied not only to conventional uniprediction and biprediction but also to other advanced temporal prediction techniques such as affine prediction, advanced temporal motion vector derivation, and biprediction optical flow. These techniques aim to derive motion field representations in finer units (e.g., 4x4) with very low motion overhead. Affine prediction is a model-based motion field coding method in which the motion of each unit within a single PU can be derived based on model parameters. Advanced temporal motion vector derivation involves deriving the motion of each unit from the motion field of a temporal reference image. Biprediction optical flow involves deriving motion fine-tuning for each pixel using an optical flow model. Regardless of the unit size, at the block level, once weight values ​​are specified, the proposed video codec can use these derived motions and given weight values ​​to perform generalized biprediction per unit.

[0029] An exemplary encoder and decoder that utilize generalized biprediction is described in more detail below.

[0030] Exemplary encoder for generalized biprediction Figure 5 is a block diagram of an exemplary video encoder adapted to perform generalized biprediction. Similar to the video encoder shown in Figure 1, in the exemplary video encoder, spatial and temporal predictions are two basic pixel-region prediction modules. The spatial prediction module can be the same as that shown in Figure 1. The temporal prediction module, named “motion prediction” in Figure 1, can be replaced by a generalized biprediction (GBi) module 502. The generalized biprediction (GBi) module can be operated to combine two separate motion-compensated prediction (MCP) signals in a weighted-average manner. As shown in Figure 6, the GBi module can implement a process for generating the final interprediction signal as follows: The GBi module can perform motion estimation on a reference image to search for two optimal motion vectors (MVs) that point to two reference blocks that minimize the weighted biprediction error between the current video block and the biprediction. The GBi module can fetch these two prediction blocks through motion compensation using their two optimal MVs. The GBi module can then calculate the generalized biprediction signal as a weighted average of the two prediction blocks.

[0031] In some embodiments, all available weight values ​​are specified within a single set. If the weight values ​​are signaled for both reference lists at the PU level, it means that it signals two separate weight values ​​per dual prediction PU, which can cost a large number of bits, so a unit gain constraint (the sum of weight values ​​equals 1) can be applied. Under this constraint, only one single weight value per PU is signaled, while the other can be derived by subtracting the signaled weight value from 1. For ease of explanation, in this disclosure, the weight values ​​associated with reference list L1 are signaled, and the set of weight values ​​is W L1It is represented by . To further reduce signaling overhead, the weight values ​​are W L1 It is encoded by the index value weight_idx, which points to the entry position within it. L1 With appropriate assignments to W, both traditional single-predictions (with weights equal to 0 for one criterion list and 1 for the other) and traditional double-predictions (with weight values ​​equal to 0.5 for both criterion lists) can be represented under the GBi framework. L1 In the special case of ={0,0.5,1}, the GBi module can achieve the same functionality as the motion prediction module shown in Figure 1.

[0032] In addition to {0,0.5,1}, W L1 Additional weight values ​​can be specified at the slice, image, or sequence level, and the non-negative integer extra_number_of_weights indicates the number of such weights; therefore, within the GBi framework, there are extra_number_of_weights + 3 distinct weights. In particular, in exemplary embodiments, when extra_number_of_weights is greater than zero, one of these additional weight values ​​can be derived on a block-by-block basis, depending on the control of the flag implicit_weight_flag, which exists at the slice, image, or sequence level. When this flag is set to equal 1, this particular weight value is not signaled but can be derived by finding the one that minimizes the difference between the generalized biprediction signal of the direct inverted L-shaped neighborhood (called the template) and the reconstructed signal of the template, as shown in Figure 7. L1 The above-mentioned process related to the construction can be performed by the weight set construction module 504.

[0033] W L1To adapt the additional weight values ​​within to images with high illuminance changes in dynamics, two scaling factors (gbi_scaling_factors) can be applied and signaled at the image level. Using these, the weight set construction module can scale the values ​​of the additional weights for GBi prediction. After inter-prediction (which is GBi prediction in the proposed video encoder) and intra-prediction, the original signal can be subtracted from this final predicted signal, thus generating the resulting predicted residual signal for encoding.

[0034] In the exemplary proposed video encoder, block motion (motion vectors and reference image indices), as well as weight value indices, are the only block-level information provided for each intercoded PU.

[0035] In an exemplary embodiment, the block motion information for GBi predictions is encoded in the same manner as that of the underlying video codec. Two sets of motion information per PU are signaled, except when weight_idx is associated with a weight equal to 0 or 1, i.e., the case equivalent to a single prediction.

[0036] In the exemplary video encoder, a weight index coding module 506 is used to binary-code the weight_idx of each PU. The output of the weight index coding module can be binary_weight_idx, a unique binary representation of weight_idx. The tree structure of the exemplary binary-coded scheme is shown in Figure 8. As in conventional interprediction, the first bit of binary_weight_idx is used for each interPU, as a single predictor (weight index associated with a weight value equal to 0 or 1) and a double predictor (W L1It is possible to distinguish between weight indices associated with weight values ​​other than 0 and 1 within the same branch. In a single-prediction branch, another bit is signaled to indicate whether the L0 criterion list (weight indices associated with weight values ​​equal to 0) or the L1 criterion list (weight indices associated with weight values ​​equal to 1) is being referenced. In a dual-prediction branch, each leaf node is W L1 A unique weight index value is assigned, associated with one of the remaining weight values ​​within the tree, i.e., a weight value that is neither 0 nor 1. At the slice or image level, an exemplary video encoder can adaptively switch between several predetermined assignment schemes, or it can dynamically assign each weight to a unique leaf node on a per-PU basis, based on the use of weight values ​​from preceding coding blocks. Generally, frequently used weight indices are assigned to leaf nodes closer to the root in the biprediction branch, while others, in contrast, are assigned to deeper leaf nodes further from the root. While traversing this tree in Figure 8, every weight_idx can be converted to a unique binary_weight_idx for entropy coding.

[0037] Generalized biprediction decoding framework Figure 9 is a block diagram of a video decoder in several embodiments. The decoder in Figure 9 can be operated to decode a bitstream generated by the video encoder shown in Figure 5. The encoding mode and prediction information can be used to derive a prediction signal using spatial prediction or generalized biprediction. For generalized biprediction, block motion information and weight values ​​are received and decoded.

[0038] The weight index decoding module 902 decodes the weight index encoded by the weight index encoding module 506 in the proposed video encoder. The weight index decoding module 902 reconstructs the same tree structure as specified in Figure 8, and each leaf node on the tree is assigned a unique weight_idx in the same way as in the proposed video encoder. In this way, the tree is synchronized between the proposed video encoder and decoder. By traversing this tree, any received binary_weight_idx can be found at a leaf node on the tree with its associated weight_idx. The exemplary video decoder, like the video encoder in Figure 5, uses weight set W L1 Includes the weight set construction module 904 for constructing the weight set. When implicit_weight_flag is equal to 1, W L1 One of the additional weight values ​​within can be derived instead of being explicitly signaled, W L1 All additional weight values ​​within can be further scaled using the scaling factors indicated by gbi_scaling_factors. Then the weight values ​​are reconstructed, W L1 This can be done by fetching what is pointed to by weight_idx.

[0039] The decoder can receive one or two sets of motion information depending on the weight values ​​selected in each block. If the reconstructed weight value is neither 0 nor 1, two sets of motion information may be received; otherwise (when it is 0 or 1), only one set of motion information associated with the non-zero weight is received. For example, if the weight value is equal to 0, only motion information for reference list L0 is signaled; otherwise, if the weight value is equal to 1, only motion information for reference list L1 is signaled.

[0040] Using block motion information and weight values, the generalized dual-prediction module 1050 shown in Figure 10 can operate to calculate the generalized dual-prediction signal as a weighted average of two motion-compensated prediction blocks.

[0041] To obtain the reconstructed video block signal, depending on the encoding mode, a spatial prediction signal or a generalized biprediction signal can be summed with the reconstructed residual signal.

[0042] Building the weight set A weight set W using explicitly signaled weights, decoder-derived weights, and scaled weights. L1 An exemplary construction process is the weight set W L1 This is described below, along with an exemplary reduction process for making the size more compact.

[0043] Explicit weight values Explicit weight values ​​can be signaled and managed hierarchically at the sequence, image, and slice levels, respectively. Weights specified at lower levels can substitute for those at higher levels. Assuming that the number of explicit weights at higher levels is p, and that at relatively lower levels is q, the following rules for substitution can be applied when constructing the list of weight values ​​at lower levels. ● When p > q, the last q weights at a higher level are replaced by q weights at a lower level. ● When p ≤ q, all weights at higher levels are replaced by those specified at lower levels.

[0044] The number of explicit weight values ​​can be indicated by extra_number_of_weights at each of the sequence, image, and slice levels. In some embodiments, at the slice level, the base weight set always includes three default values ​​that form {0, 0, 5, 1} so that GBi supports conventional single-prediction and bi-prediction, and therefore, for each block, a total of (extra_number_of_weights + 3) weights can be used. For example, if the values ​​of extra_number_of_weights present at the sequence, image, and slice levels are 2 (e.g., w) A , w B ), 1 (for example, w C ), and 3 (for example, w D , w E , w F When this is the case, the available weight values ​​at the sequence, image, and slice levels are, respectively, {w A ,w B}, {w A ,w C}, and {0,0.5,1}∪{w D ,w E ,w F}. In this example, W, which was mentioned in the section "Generalized Biprediction", is L1 This is the weight set for each slice level.

[0045] Implicit weight value derivation process In some embodiments, the weight set W of the slice levels L1 The weight values ​​are derived through template matching in both the encoder and decoder, without signaling. As shown in Figure 7, these implicit weight values ​​are derived from the prediction signals (T0 and T1) of the template, which currently have block motion information, and the reconstruction signal (i.e., T) of the template. C This can be derived by minimizing the difference between ). This problem is, w * = argmin w Σ X(T C [x]-(1-w)*T0[x+v0]-w*T1[x+v1]) 2 (4) It can be formulated as follows, where v0 and v1 are the motion vectors of the current block. Since equation (4) is a quadratic function, if T0 and T1 are not exactly the same, a closed-form representation of the derived weights can be obtained, i.e.,

[0046]

number

[0047] This is the result.

[0048] The effectiveness of this method can be seen when the weight values ​​of the current block's signal are correlated with those of the associated template prediction signal, however this is not always guaranteed, especially when the pixels in the current block and their associated templates are located in different motion objects. To maximize GBi's prediction performance, the flag implicit_weight_flag can be signaled at the slice, image, or sequence level to determine whether implicit weights are used when extra_number_of_weights≧1. Once this is set to equal 1, W L1 The weight values ​​for the last slice level within are derived and therefore do not need to be signaled. For example, as mentioned above in the section "Explicit Weight Values" above. F If signaling is not required and implicit_weight_flag is equal to 1, the weights for the block can be implicitly derived.

[0049] Weight scaling process In some embodiments, explicit weight values ​​can be further scaled by using two scaling factors gbi_scaling_factors, which are shown at the image level. Due to the high potential for illuminance changes in the temporal dynamics within the image, the dynamic range of these weight values ​​may not be sufficient to cover all these cases. While weighted biprediction can compensate for illuminance differences between images, this is not always guaranteed to be possible in the underlying video codec. Therefore, these scaling factors can be used to adjust for illuminance differences across multiple reference images when weighted biprediction is not used.

[0050] The first scaling factor is W L1 Each explicit weight value within can be increased. Using this, the prediction function for GBi in equation (3) is:

[0051]

number

[0052] It can be expressed as follows, where α is the first scaling factor of the current image and w1' represents the scaled weight value (i.e., α*(w1-0.5)+0.5). The first equation of equation (6) can be expressed in the same form as equation (3). The only difference is the weight value applied to equations (6) and (3).

[0053] A second scaling factor can be used to reduce the difference in illuminance between the associated reference images of P0 and P1. Using this scaling factor, equation (6) becomes:

[0054]

number

[0055] It can be further reformulated as follows, where s, s0, and s1 represent the signaled second scaling coefficients in the current image and its two reference images, respectively. According to equation (7), one optimal assignment for the variable s can be the sample mean in the current image. Thus, the mean of the reference images can be expected to be similar after the second scaling coefficient is applied. Due to commutativity, applying the scaling coefficients to P0 and P1 is equivalent to applying them to the weight values, and therefore equation (7) is,

[0056]

number

[0057] It can be reinterpreted in this way.

[0058] Therefore, the weight set construction process can be expressed as a function of explicit weights, implicit weights, scaling factors, and a reference image. For example, the slice-level weight set W described above. L1 is {0,0.5,1}∪{(s / s1)*w D ',(s / s1)*w E ',(s / s1)*w F '} and the weight set for L0 is {1,0.5,1}∪{(s / s0)*(1-w D '),(s / s0)*(1-w E '),(s / s0)*(1-w F ')}, where s1 is the average sample value of the reference image in list L1 for the current block, and s0 is the average sample value of the reference image in list L0 for the current block.

[0059] Weight reduction Exemplary embodiments include W L1It works to further reduce the number of weight values. Two exemplary techniques for reducing weight values ​​are described in detail below. The first technique operates in response to the motion compensation prediction results, and the second technique operates based on weight values ​​outside the range of 0 to 1.

[0060] A prediction-based method. Given PU motion information, not all weights will generate bipredictions that are substantially different from each other. An exemplary embodiment reduces redundant weight values ​​(which generate similar biprediction signals) and retains only one weight from among the redundant values, W L1 This property is utilized by making it more compact. To do so, a function can be used to calculate the similarity between two predictive signals with two different weight values. This function can be, but is not limited to, a cosine similarity function, and operates as follows:

[0061]

number

[0062] Here, w (i) and w (j) is, W L1 The two independent weight values ​​within are v0 and v1, where v0 and v1 are given bipredictive motion information, and P[x;w,v0,v1] represents the same prediction function shown in equations (3), (6), and (8), using the given w, v0, and v1. When the value of equation (9) falls below a given threshold (indicated by weight_pruning_threshold at the slice level), one of the weights may be reduced according to the syntax pruning_smaller_weight_flag at this slice level. If this flag is set to equal 1, the reduction process is performed W L1 From lol (i) and w (j) The smaller of the two weights is removed. Otherwise (when this flag is set to equal 0), the larger one is removed. In an exemplary embodiment, this reduction process is WL1 This is applied to each pair of weight values ​​within, and as a result, the resulting W L1 No two weight values ​​within the set will produce similar biprediction signals. The similarity between two weight values ​​can also be assessed using the sum of absolute transformed differences (SATD). To reduce computational complexity, this similarity can be assessed using two subsampled prediction blocks. For example, it can be calculated using subsampled rows or subsampled columns of the sample in both the horizontal and vertical directions.

[0063] A weight-based approach. Weight values ​​outside the range of 0 to 1 (or, simply, out-of-range weights) can exhibit different behavior with respect to encoding performance under different encoding structures (e.g., hierarchical or low-latency structures). Taking advantage of this fact, an exemplary embodiment utilizes a set of sequence-level indices, weight_control_idx, to restrict the use of out-of-range weights separately for each temporal layer. In such an embodiment, each weight_control_idx is associated with all images in a particular temporal layer. Depending on how this index is constructed, out-of-range weights can be made available for use or reduced, conditionally as follows: ● If weight_control_idx=0, W L1 The associated image remains unchanged. ● If weight_control_idx=1, W L1 Weights outside the range are not available for the associated image. ● If weight_control_idx=2, W L1 Out-of-range weights within the range are available for some of the associated images whose reference frame is purely historical (e.g., low-latency configurations in HEVC and JEM). ● If weight_control_idx=3, W L1 Out-of-range weights are available for the associated image only when the slice-level flag mvd_ll_zero_flag is enabled in HEVC and JEM.

[0064] Weight Index Coding Exemplary systems and methods for binary-coded and codeword assignment for weight-index coding are described in more detail below.

[0065] Binary evolution process for weight index coding In an exemplary embodiment, each weight index (weight_idx) is converted to a unique binary representation (binary_weight_idx) through a systematic code before entropy coding. For illustrative purposes, the tree structure of the proposed binary evolution method is shown in Figure 8. The first bit of binary_weight_idx is used to distinguish between single-predictions (associated with weights equal to 0 or 1) and double-predictions. In the single-prediction branch, another bit is signaled to indicate which of two reference lists is being referenced: reference list L0 (associated with weight indices pointing to weight values ​​equal to 0) or reference list L1 (associated with weight indices pointing to weight values ​​equal to 1). In the double-prediction branch, each leaf node is W L1Each leaf node in the biprediction branch is assigned a unique weight index value associated with one of the remaining weight values, i.e., weight values ​​that are neither 0 nor 1. The exemplary video codec supports a variety of systematic codes for binary evolution of the biprediction branch, including truncated unary code (e.g., Figure 8) and exponential-Golomb code. An exemplary technique in which each leaf node in the biprediction branch is assigned a unique weight_idx is described in more detail below. By examining this tree structure, each weight index can be mapped to, or recovered from, a unique codeword (e.g., binary_weight_idx).

[0066] Adaptive codeword assignment for weight index coding In the exemplary binary tree structure, each leaf node corresponds to one codeword. To reduce the signaling overhead of weight indices, various adaptive codeword assignment methods can be used to map each leaf node in a biprediction branch to a unique weight index. Exemplary methods include predetermined codeword assignment, block adaptive codeword assignment, temporal layer-based codeword assignment, and time-delayed CTU adaptive codeword assignment. These exemplary methods update the codeword assignment in the biprediction branch based on the occurrence of weight values ​​used in previously encoded blocks. Frequently used weights can be assigned to shorter codewords (i.e., shallower leaf nodes in the biprediction branch), while others can be assigned to relatively longer codewords.

[0067] 1) Predefined codeword assignment. Using a predetermined codeword assignment, a fixed codeword assignment can be provided for leaf nodes in a biprediction branch. In this method, the weight index associated with the weight of 0.5 is assigned the shortest codeword, i.e., node i in Figure 8. Weight values ​​other than 0.5 can be divided into two sets: Set 1 contains all values ​​greater than 0.5 and is ordered in ascending order, and Set 2 contains all values ​​less than 0.5 and is ordered in descending order. These two sets are then interleaved to form Set 3, starting from either Set 1 or Set 2. All remaining codewords, ordered from shortest to longest in length, are then assigned sequentially to the weight values ​​in Set 3. For example, if the set of all possible weight values ​​in the biprediction branch is {0.1, 0.3, 0.5, 0.7, 0.9}, then set 1 is {0.7, 0.9}, set 2 is {0.3, 0.1}, and set 3 is {0.7, 0.3, 0.9, 0.1} if interleaving starts from set 1. The codewords, arranged from shortest to longest in length, are assigned to 0.5, 0.7, 0.3, 0.9, and 0.1 in order.

[0068] This assignment can change in environments where a codec can drop one motion vector difference when two sets of motion information are transmitted. For example, in HEVC, this behavior can be found from the slice-level flag mvd_ll_zero_flag. In this case, an alternative codeword assignment is the closest weight value greater than 0.5 (e.g., w + Assign it to the weight index associated with ) then w +A weight index associated with a weight value that is the nth smallest (or largest) among those greater than (or smaller than) a given value is assigned the (2n+1)th (or 2nth) shortest codeword. Based on the previous example, the codewords, ordered from shortest to longest, are assigned to 0.7, 0.5, 0.9, 0.3, and 0.1, respectively. The resulting assignments for both examples are shown in Figures 11A and 11B.

[0069] 2) Block-adaptive codeword assignment using causal-neighboring weights. The weight values ​​used in causal-neighboring blocks can be correlated with those currently used for the block. Based on this knowledge and a given codeword assignment method (e.g., constant assignment or alternative assignment), weight indices that can be found in causal-neighboring blocks are encouraged to leaf nodes in the biprediction branch that have shorter codeword lengths. Similar to the process of building a motion vector prediction list, causal-neighboring blocks can be accessed in an ordered sequence as shown in Figure 12A, and at most two weight indices can be encouraged. As can be seen from the figure, moving from the bottom left block to the left block, the first available weight indices (if any) can be encouraged to have the shortest codeword length, and moving from the top right block to the top left block, the first available weight indices (if any) can be encouraged to have the second shortest codeword length. For other weight indices, they can be assigned to the remaining leaf nodes, from shallowest to deepest, according to their codeword lengths in the initial assignment. Figure 12B provides an example showing how a given codeword assignment can fit itself to causally related neighbor weights. In this example, constant assignments are used, and weight values ​​equal to 0.3 and 0.9 are encouraged.

[0070] 3) Temporal layer-based codeword assignment. In an exemplary method using temporal layer-based codeword assignment, the proposed video encoder adaptively switches between constant codeword assignment and alternative codeword assignment. An optimal codeword assignment method using the weight index of the minimum expected codeword length, based on the use of weight indices from previously encoded images, either in the same temporal layer or using the same QP value, can be found as follows:

[0071]

number

[0072] Here, L m (w) represents the codeword length of w using a certain codeword assignment method m, and W L1 Bi This is a set of weight values ​​used only for biprediction, and Prob k (w) represents the cumulative probability of w across k images in the temporal layer. Once the best codeword assignment method is determined, it can be applied to the encoding of the weight index for the current image, or to the analysis of the binary weight index.

[0073] Several different methods are considered for accumulating the use of weight indices over a temporal image. Illustrative methods can be formulated with a common formula:

[0074]

number

[0075] Here, w i is, W L1 It is a certain weight within the Count j(w) represents the occurrence of a certain weight value in the j-th image of the temporal layer, n determines the number of recent images to be stored, and λ is the forgetting term. n and λ are encoder-specific parameters, and they can adapt themselves to various encoding conditions for each image, such as n=0 for scene changes and a smaller λ for motion videos.

[0076] In some embodiments, the choice of codeword assignment method can be explicitly indicated using slice-level syntax elements. Thus, the decoder does not need to maintain the use of weight indices over time, and therefore, analysis dependence on weight indices across temporal images can be completely avoided. Such a method also improves decoding robustness.

[0077] 4) CTU Adaptive Codeword Assignment. Switching between different methods for codeword assignment based solely on the weight usage of previously encoded images may not always match well with that of the current image. This can be due to a lack of consideration for the weight usage of the current image. In exemplary embodiments using CTU Adaptive Codeword Assignment, Prob k (w i ) can be updated based on the weight usage of the encoded blocks in the current CTU row and the CTU row immediately above it. Assuming the current image is the (k+1)th image in the temporal layer, Prob k (w i ) can be updated per CTU as follows:

[0078]

number

[0079] Here, B represents the set of encoded CTUs in the current CTU row and the CTU row immediately above it, and Count' j(w) represents the occurrence of a certain weight value in the j-th CTU collected in set B. Prob k (w i ), once updated, is applied to Equation (10), and thus, the best codeword assignment method can be determined.

[0080] Extension to Advanced Temporal Prediction Techniques Embodiments for extending the application of generalized bi-prediction to be used together with other coding techniques, including local illumination compensation, weighted bi-prediction, merge mode, bi-predicted optical flow, affine motion prediction, overlapped block motion compensation, and frame rate up-conversion bi-prediction, are described below.

[0081] Local Illumination Compensation and Weighted Bi-Prediction Exemplary generalized bi-prediction techniques can be performed, among other techniques, especially on local illumination compensation (IC) and / or weighted bi-prediction. Both IC and weighted bi-prediction operate to compensate for illumination changes on the reference block. One difference between them is that in the use of IC, the weights (c0 and c1) and the offset values (o0 and o1) are derived through block-by-block template matching, while in the use of weighted bi-prediction, these parameters are explicitly signaled slice-by-slice. Using these parameters (c0, c1, o0, o1), the prediction signal of GBi can be

[0082]

Equation

[0083] calculated as such, where the weight value scaling process described in the above section "Weight Value Scaling Process" is applied. When this scaling process is not applied, the prediction signal of GBi is P[x]=(1-w1)*(c0*P0[x+v0]+o0)+w1*(c1*P1[x+v1]+o1) (14) It can be calculated as follows.

[0084] For example, the use of these combined prediction processes, as described in equation (13) or equation (14), can be signaled at the sequence level, image level, or slice level. Signaling can be performed separately for the GBi and IC combination and for the GBi and weighted dual prediction combination. In some embodiments, the combined prediction process of equation (13) or equation (14) is applied only when the weight value (w1) is not equal to 0, 0.5, or 1. In particular, when the use of the combined prediction process is active, the value of the block-level IC flag (used to indicate the use of IC) determines whether GBi is combined with IC (using w1 ≠ 0, 0.5, 1). If not, and the combined prediction process is not used, GBi and IC perform two independent prediction modes (using w1 ≠ 0, 0.5, 1), and for each block, this block-level IC flag does not need to be signaled and is therefore presumed to be zero.

[0085] In some embodiments, whether GBi can be combined with IC or with weighted biprediction is signaled separately in the sequence parameter set (SPS), image parameter set (PPS), or slice header using a high-level syntax that uses flags such as gbi_ic_comb_flag and gbi_wb_comb_flag. In some embodiments, if gbi_ic_comb_flag is equal to 0, GBi and IC are not combined, and therefore the GBi weight values ​​(w1≠0,0.5,1) and IC flag do not coexist for any biprediction coding unit. For example, in some embodiments, if GBi weights w1≠0,0.5,1 are signaled for a coding unit, there is no IC flag to signal, and the value of this flag is inferred to be zero; otherwise, the IC flag is explicitly signaled. In some embodiments, if gbi_ic_comb_flag is equal to 1, GBi and IC are combined, and both the GBi weight and IC flag are signaled independently for a single coding unit. The same semantics can be applied to gbi_wb_comb_flag.

[0086] Merge mode In some embodiments, the merge mode is used to infer not only movement information from causally related neighboring blocks, but also the weight index of that block at the same time. The access order to causally related neighboring blocks (as shown in Figure 13) can be the same as that specified in HEVC, with spatial blocks accessed in the order of left, top, top right, bottom left, and top right blocks, while temporal blocks are accessed in the order of bottom right and center blocks. In some embodiments, up to four merge candidates are constructed from spatial blocks and up to one from temporal blocks, for a maximum of five merge candidates. Given the merge candidates, the GBi prediction process specified in equation (3), equation (8), equation (13), or equation (14) can be applied. Note that the weight index does not need to be signaled, as it is inferred from the weight information of the selected merge candidates.

[0087] The JEM platform offers an additional merge mode called advanced temporal motion vector prediction (ATMVP). In some embodiments of this disclosure, ATMVP is combined with GBi prediction. In ATMVP, motion information for each 4x4 unit within a single CU is derived from the motion field of a temporal reference image. In exemplary embodiments using ATMVP, when the GBi prediction mode is enabled (e.g., when extra_number_of_weights is greater than 0), the weight indices for each 4x4 unit can also be inferred from those of the corresponding temporal blocks in the temporal reference image.

[0088] Dual-predictive optical flow In some embodiments, the GBi weights can be applied to a bipredictive optical flow (BIO) model. Based on motion-compensated prediction signals (P0[x+v0] and P1[x+v1]), BIO uses an offset value O to reduce the difference between two corresponding samples in L0 and L1 with respect to their spatial vertical and horizontal gradient values. BIO [x] can be estimated. To combine this offset value with the GBi prediction, equation (3) is given. P[x]=(1-w1)*P0[x+v0]+w1*P1[x+v1]+o BIO [x] (15) It can be reformulated as follows, where w1 is the weight value used to perform the GBi prediction. This offset value can also be applied to other GBi variations as an additive offset after the prediction signals in P0 and P1 have been scaled, as in equation (8), equation (13), or equation (14).

[0089] Affine prediction In exemplary embodiments, GBi prediction can be combined with affine prediction in a manner similar to an extension of conventional biprediction. However, there are differences in the underlying processing units used to perform motion compensation. Affine prediction is a model-based motion field derivation technique for forming a fine-grained motion field representation of PUs, where the motion field representation for each 4x4 unit is derived based on a single-prediction or biprediction transformed motion vector and given model parameters. Since all motion vectors point to the same reference image, it is not necessary to fit weight values ​​to each of the 4x4 units. Therefore, weight values ​​can be shared across each unit, and only one weight index per PU can be signaled. With the motion vectors and weight values ​​in the 4x4 units, GBi can be performed on a unit-by-unit basis, and thus the same equations (3), (8), (13), and (14) can be applied directly without modification.

[0090] Overlapping block motion compensation Superimposed block motion compensation (OBMC) is a method for providing a prediction of the intensity value of a sample based on the sample's own motion vector and motion compensation signals derived from those in its causally related neighbors. In exemplary embodiments of GBi, weight values ​​may also be considered in motion compensation for OBMC. An example is shown in Figure 14, where subblock B1 in the current block has three motion compensation prediction blocks, each of which is formed by using motion information and weight values ​​from blocks N1, N5, or B1 itself, and the resulting prediction signal of B1 can be a weighted average of the three.

[0091] Frame rate upconversion biprediction In some embodiments, GBi can work in conjunction with frame rate upconversion (FRUC). For FRUC, two different modes can be used. A biprediction mode can be used when the current image falls between a first reference image in L0 and a first reference image in L1. A uniprediction mode can be used when both the first reference image in L0 and the first reference image in L1 are either forward or backward reference images. The biprediction case in FRUC is described in detail below. In JEM, equal weights (i.e., 0.5) are used for FRUC biprediction. The quality of the two predictors in FRUC biprediction can be different, but combining two predictors with unequal predictive quality using equal weights can be suboptimal. The use of GBi can improve the final biprediction quality due to the use of unequal weights. In exemplary embodiments, the GBi weight values ​​are derived for the block encoded using FRUC biprediction and therefore do not need to be signaled. L1Each weight value within is independently evaluated for each of the 4×4 sub-blocks within the PU using the MV derivation process of FRUC dual prediction. For a 4×4 block, the weight value that results in the minimal bilateral matching error (the sum of the absolute differences between two one-sided prediction motion compensation predictors associated with two reference lists) is selected.

[0092] In an exemplary embodiment, FRUC dual prediction is a decoder-side MV derivation technique that derives an MV by using dual prediction matching. For each PU, a list of candidate MVs collected from causally neighboring blocks is formed. Under the assumption of a certain motion, each candidate MV is linearly projected onto the first reference image within the other reference list, and the scaling factor for the projection is set to be proportional to the temporal distance between the reference image (e.g., at time t0 or t1) and the current image (t c ). Taking FIG. 15 as an example where v0 is the candidate MV associated with the reference list L0, v1 is calculated as v0*(t1 - t c ) / (t0 - t c ). Thus, the bilateral matching error can still be calculated for each candidate MV, and the first MV that reaches the minimal bilateral matching error is selected from the candidate list. Denote this first MV as v0 INIT . Starting from the first MV indicated by v INIT 0, decoder-side motion estimation is performed to find an MV within a predetermined search range, and the MV that reaches the minimal bilateral matching error is selected as the MV at the PU level. Assuming that v1 is the projected MV, the optimization process can be formulated as

[0093]

Number

[0094] as shown. When FRUC dual prediction is combined with GBi, the search process in Equation (16) is WL1 Using the internal weight values ​​w, it is reformulated, i.e.,

[0095]

number

[0096] This PU level v0 can be further refined independently for each 4x4 subblock within the PU, as shown in equation (18), using the same bilateral matching as in equation (17).

[0097]

number

[0098] W L1 For each available weight value within the PU, equation (18) can be evaluated, and the weight value that minimizes the bilateral matching error is selected as the optimal weight. At the end of the evaluation process, each 4x4 subblock in the PU has its own bilateral prediction MV and weight values ​​for performing generalized bilateral prediction. The complexity of such an exhaustive search method can be high because the weights and motion vectors are searched collaboratively. In another embodiment, the search for the optimal motion vectors and optimal weights can be performed in two steps. In the first step, the motion vector for each 4x4 block can be obtained using equation (18) by setting w to an initial value, for example, w = 0.5. In the second step, given the optimal motion vectors, the optimal weights can be searched.

[0099] In another embodiment, three steps can be applied to improve the accuracy of motion search. In the first step, the initial weight is the initial motion vector v0 INIT This is used for the search. The initial optimal weights are w INIT This is expressed as follows. In the second step, the motion vector for each 4x4 block is w INITBy setting this, it can be obtained using equation (18). In the third step, given the optimal motion vector, the final optimal weight can be searched for.

[0100] From equations (17) and (18), the goal is to minimize the difference between two criterion lists and the two weighted predictors associated with each. Negative weights may not be suitable for this purpose. In one embodiment, the FRUC-based GBi mode only evaluates weight values ​​greater than zero. To reduce complexity, the calculation of the sum of absolute differences can be performed using partial samples within each subblock. For example, the sum of absolute differences can be calculated using only samples located in even-numbered rows and columns (or, or odd-numbered rows and columns).

[0101] GBi Predictive Search Strategy Initial criteria list for bipredictive search A method for improving GBi's predictive performance by determining which of the two reference lists should be explored first in the motion estimation (ME) stage of biprediction is described below. Similar to conventional biprediction, the ME stage cost, i.e., Cost(t i ,U j )=Σ x |I[x]-P[x]|+λ*Bits(t i ,u j (weight index) (19) To minimize t, there exist two motion vectors associated with reference list L0 and reference list L1, respectively, where I[x] is the original signal of sample x currently placed at x in the image, and P[x] is the predicted signal of GBi, and t i and u jλ are motion vectors pointing to the i-th reference image in L0 and the j-th reference image in L1, respectively, λ is a Lagrangian parameter used in the ME stage, and the Bits(●) function estimates the number of bits to encode the input variable. Equations (3), (8), (13), and (14) can each be applied to substitute P[x] in equation (19). For brevity of explanation, we consider equation (3) as an example for the following process. Thus, the cost function in equation (19) is, Cost(t i ,u j )=Σ x |I[x]-(1-w1)*P0[x+t i ]-w1*P1[x+u j ]|+λ*Bits(t i ,u j (weight index) (20) It can be rewritten as follows.

[0102] Two parameters (t) are determined i and u j Since such a procedure exists, iterative procedures can be used. The first such procedure can proceed as follows: 1.{u j Using the best move in |∀j}, t i Optimize ∀i. 2.{t i Using the best move in |∀i}, u j Optimize ∀j. 3.t i and u j Repeat steps 1 and 2 until the variable no longer changes or until the maximum number of iterations is reached.

[0103] The second exemplary iteration procedure can proceed as follows: 1.{t i Using the best move in |∀i}, u j Optimize ∀j. 2.{u jUsing the best move in |∀j}, t i Optimize ∀i. 3.u j and t i Repeat steps 1 and 2 until the variable no longer changes or until the maximum number of iterations is reached.

[0104] Which iteration procedure is selected depends on t i and u j It can only depend on the ME stage cost, i.e.,

[0105]

number

[0106] And here, the ME stage cost function can be as follows: Cost(t i )=Σ x |I[x]-P0[x+t i ]|+λ*Bits(t i ) (twenty two) Cost(u j )=Σ x |I[x]-P1[x+u j ]|+λ*Bits(u j ) (twenty three) However, this initialization process may not be optimal in cases where 1-w1 and w1 are not equal. For example, when one of the weight values ​​is very close to 0, w1 = lim w→0 A typical example where w is such that the ME stage cost of its associated movement happens to be lower than the other. In this case, equation (20) is: Cost(t i ,u j )=Σ x |I[x]-P0[x+t i ]|+λ*Bits(t i ,u j (weight index) (24) It degenerates. jThe overhead incurred for this makes no contribution to the prediction signal and results in poor search results for GBi. In this disclosure, instead of equation (21), the magnitude of the weight values ​​is used, i.e.,

[0107]

number

[0108] That is the case.

[0109] Binary search for weight index The number of weight values ​​to be evaluated can introduce additional complexity to the encoder, so an exemplary embodiment utilizes a binary search method to reduce less likely weight values ​​in the initial stages of encoding. In one such search method, a conventional single prediction (associated with weights 0 and 1), as well as a biprediction (associated with weight 0.5), are performed right at the start, W L1 The weight values ​​within are for four groups, i.e., A=[w min B=[0,0.5], C=[0.5,1], and D=[1,w max It can be classified as ]. min and w max These are, W L1 Represents the minimum and maximum weight values ​​within, without loss of generality, w min <0, and w max It is assumed that > ​​1. The following rules can be applied to determine the range of possible weight values. ● If w=0 provides a better ME stage cost than w=1, the following rule applies: ○ If w=0.5 gives a better ME stage cost than w=0 and w=1, then based on the weight values ​​in B, the weight set W (0) However, it is formed. ○ Otherwise, based on the weight values ​​in A, W (0) However, it is formed. ● In all other cases (where w=1 provides a better ME stage cost than w=0), the following rules apply: ○ If w=0.5 gives a better ME stage cost than w=0 and w=1, then based on the weight values ​​in C, W (0) However, it is formed. ○ Otherwise, based on the weight values ​​in D, W (0) However, it is formed.

[0110] W (0) However, after it is formed, W (0) According to the minimum and maximum values ​​within, w min and w max The value of W can be reset. (0) However, when associated with A and D, the w within A is, respectively min and w in D max The ME stage cost can be calculated.

[0111] In the kth iteration, the iterative process continues until three or more weight values ​​remain in the set, W (k) It is possible to keep updating it. Assuming the process is in its kth iteration, the iterative process can be specified as follows: 1. (w min + w max w is the weight value closest to ) / 2 middle Use this to run GBi. 2.w middle But, lol min and w max If you want to give a better ME stage cost than [w min ,w middle ] and [w middle ,w max To test both of ] independently, W (k+1) The recursive process for is called, and the iterative process jumps to step 6. 3. No, w middle But, lol min and w maxIf the ME stage cost is worse than that, the iterative process terminates. 4. No, w min But, lol max If you want to give a better ME stage cost than [w min ,w middle Based on the weight values ​​in ], W (k+1) However, once formed, the iterative process jumps to step 6. 5. Otherwise (w min But, lol max (If you give a worse ME stage cost than that), [w middle ,w max Based on the weight values ​​in ], W (k+1) However, once formed, the iterative process jumps to step 6. 6.W (k+1) If the number of weight values ​​remaining within is 3 or more, W (k+1) According to the minimum and maximum values ​​within, w min and w max However, if it is reset, the iterative process returns to step 1; otherwise, the iterative process terminates.

[0112] After the iterative process has stopped, the weight value that achieved the lowest ME stage cost among all test values ​​can be selected to perform generalized biprediction.

[0113] Weight estimation for non-2N×2N partitions In some embodiments, after testing each weight value for a 2N×2N partition, the best-running weight values ​​other than 0, 0.5, and 1 can serve as estimates for the optimal weight values ​​for non-2N×2N partitions. In some embodiments, assuming n unique estimates exist, only weight values ​​equal to 0, 0.5, 1, and n estimates are evaluated for non-2N×2N partitions.

[0114] Partition size estimation for non-2N×2N partitions In some embodiments, not all non-2N×2N partitions are tested by the exemplary video encoder. Non-2N×2N partitions can be classified into two subcategories: symmetric motion partitions (SMPs) having 2N×N and N×2N partition types, and asymmetric motion partitions (AMPs) having 2N×nU, 2N×nD, nL×2N, and nR×2N partition types. Some partition types within AMPs are evaluated by the encoder if the rate-distortion (RD) cost for a partition in an SMP is smaller than that of a 2N×2N partition. The decision of which partition types within an AMP to test may depend on which of 2N×N and N×2N exhibits better performance in terms of RD cost. If the rate-distortion cost of 2N×N is smaller, partition types 2N×nU and 2N×nD can be further checked; otherwise (if the cost of N×2N is smaller), partition types nL×2N and nR×2N can be further checked.

[0115] Fast parameter estimation for multipath encoding In exemplary embodiments using a multi-pass encoder, optimized prediction parameters (such as block motion and weight values) from earlier encoding passes can be adopted as initial parameter estimates in subsequent encoding passes. In such encoders, the encoding blocks separated from the image can be predicted and encoded two or more times, resulting in a considerable increase in encoding complexity. One technique to reduce this complexity is to buffer the optimized prediction parameters from the initial encoding pass and take them as initial parameter estimates for further refinement in subsequent encoding passes. For example, if the inter-prediction mode happens to be the best mode in the initial pass, the encoder evaluates only the inter-prediction mode for the rest of the encoding passes. In some embodiments, buffering is performed W L1 The process involves selecting weight values ​​within the GBi, and setting prediction parameters associated with the selected weight values, such as the bipredictive MV, IC flag, OBMC flag, integer motion vector (IMV) flag, and coded block flag (CBF). In such embodiments, the values ​​of these buffered parameters can be reused or refined in subsequent encoding passes. In particular, when the aforementioned bipredictive MVs are taken, these MVs can serve as initial search positions for the bipredictive search. They are then refined in the motion estimation stage and subsequently taken as initial search positions for the next encoding pass.

[0116] An example bitstream communication framework Figure 16 is a schematic diagram showing an example of an encoded bitstream structure. The encoded bitstream 1000 consists of a number of NAL (Network Abstraction Layer) units 1001. The NAL units may contain high-level syntax metadata such as encoded sample data, or parameter set data, such as encoded slice 1006, or slice header data 1005, or supplemental enhancement information data 1007 (sometimes called SEI messages). The parameter set is a high-level syntax structure containing basic syntax elements that can be applied to multiple bitstream layers (e.g., video parameter set 1002 (VPS)), or to an encoded video sequence within a single layer (e.g., sequence parameter set 1003 (SPS)), or to a number of encoded images within a single encoded video sequence (e.g., image parameter set 1004 (PPS)). The parameter set can be transmitted together with the encoded images of the video bitstream, or through other means (including out-of-band transmission using a trusted channel, hardcoding, etc.). The slice header 1005 is also a high-level syntax structure that can contain some image-related information, which may be relatively small or relevant only to a particular slice or image type. The SEI message 1007 carries information that may not be required by the decoding process but can be used for various other purposes, such as image output timing or display, as well as loss detection and concealment.

[0117] Figure 17 is a schematic diagram showing an example of a communication system. The communication system 1300 may comprise an encoder 1302, a communication network 1304, and a decoder 1306. The encoder 1302 can communicate with the network 1304 via connection 1308, which may be a wired or wireless connection. The encoder 1302 may be analogous to the block-based video encoder in Figure 1. The encoder 1302 may include a single-layer codec (e.g., Figure 1) or a multi-layer codec. For example, the encoder 1302 may be a multi-layer (e.g., 2-layer) scalable coding system with image-level ILP support. The decoder 1306 can communicate with the network 1304 via connection 1310, which may be a wired or wireless connection. The decoder 1306 may be analogous to the block-based video decoder in Figure 2. The decoder 1306 may include a single-layer codec (e.g., Figure 2) or a multi-layer codec. For example, the decoder 1306 can be a multi-layer (e.g., 2-layer) scalable decoding system having image-level ILP support.

[0118] The encoder 1302 and / or decoder 1306 can be incorporated into a wide variety of wired communication devices and / or wireless transceiver units (WTRUs), including, but not limited to, digital televisions, wireless broadcasting systems, network elements / terminals, servers such as content or web servers (e.g., Hypertext Transfer Protocol (HTTP) servers), personal digital assistants (PDAs), laptop or desktop computers, tablet computers, digital cameras, digital recording devices, video game devices, video game consoles, cellular or satellite radio phones, and / or digital media players.

[0119] The communication network 1304 can be any type of communication network appropriate to the situation. For example, the communication network 1304 can be a multiple access system that provides content such as voice, data, video, messaging, and broadcasting to multiple wireless users. The communication network 1304 can enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communication network 1304 can utilize one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), quadrature FDMA (OFDMA), and / or single-carrier FDMA (SC-FDMA). The communication network 1304 can include multiple connected communication networks. The communication network 1304 can include the Internet, and / or one or more private commercial networks, such as cellular networks, WiFi hotspots, and / or Internet service provider (ISP) networks.

[0120] Figure 18 is a system diagram of an exemplary WTRU in which an encoder or decoder described herein may be implemented. As shown, the exemplary WTRU 1202 may include a processor 1218, a transceiver 1220, a transmit / receive element 1222, a speaker / microphone 1224, a keypad or keyboard 1226, a display / touchpad 1228, a non-removable memory 1230, a removable memory 1232, a power supply 1234, a Global Positioning System (GPS) chipset 1236, and / or other peripherals 1238. It will be understood that the WTRU 1202 may include any subcombination of the above elements while maintaining consistency with the embodiment. Furthermore, a terminal into which an encoder (e.g., encoder 100) and / or decoder (e.g., decoder 200) is incorporated may include some or all of the elements shown in the WTRU 1202 of Figure 18 and described herein with reference thereto.

[0121] The processor 1218 can be a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), multiple microprocessors, one or more microprocessors working with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and a state machine. The processor 1218 can perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 1202 to operate in wired and / or wireless environments. The processor 1218 can be coupled to the transceiver 1220, and the transceiver 1220 can be coupled to the transmit / receive element 1222. Although Figure 18 shows the processor 1218 and the transceiver 1220 as separate components, it will be understood that the processor 1218 and the transceiver 1220 can be integrated together in an electronic package and / or chip.

[0122] The transmit / receive element 1222 can be configured to transmit signals to and / or receive signals from another base station over the air interface 1215. For example, in one or more embodiments, the transmit / receive element 1222 may be an antenna configured to transmit and / or receive RF signals. In one or more embodiments, the transmit / receive element 1222 may be a radiator / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In one or more embodiments, the transmit / receive element 1222 may be configured to transmit and / or receive both RF signals and optical signals. It will be understood that the transmit / receive element 1222 can be configured to transmit and / or receive any combination of radio signals.

[0123] In addition, although the transmit / receive element 1222 is shown as a single element in Figure 18, the WTRU 1202 can include any number of transmit / receive elements 1222. More specifically, the WTRU 1202 can utilize MIMO technology. Therefore, in one embodiment, the WTRU 1202 can include two or more transmit / receive elements 1222 (e.g., multiple antennas) for transmitting and receiving radio signals over the air interface 1215.

[0124] The transceiver 1220 can be configured to modulate the signal to be transmitted by the transmit / receive element 1222 and / or demodulate the signal received by the transmit / receive element 1222. As mentioned above, the WTRU 1202 can have multimode capabilities. Therefore, the transceiver 1220 may include multiple transceivers to enable the WTRU 1202 to communicate via multiple RATs, such as UTRA and IEEE 802.11.

[0125] The processor 1218 of the WTRU1202 can be coupled to a speaker / microphone 1224, a keypad 1226, and / or a display / touchpad 1228 (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit) and can receive user input data from them. The processor 1218 can also output user data to the speaker / microphone 1224, the keypad 1226, and / or the display / touchpad 1228. In addition, the processor 1218 can retrieve information from any type of suitable memory, such as non-removable memory 1230 and / or removable memory 1232, and store data in them. Non-removable memory 1230 can include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 1232 can include subscriber identification module (SIM) cards, memory sticks, and secure digital (SD) memory cards, etc. In one or more embodiments, the processor 1218 can obtain information from memory not physically located on the WTRU 1202, such as on a server or home computer (not shown), and can store data therein.

[0126] The processor 1218 can receive power from the power supply 1234 and can be configured to distribute and / or control power to other components within the WTRU 1202. The power supply 1234 can be any suitable device for supplying power to the WTRU 1202. For example, the power supply 1234 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), a solar cell, and a fuel cell.

[0127] The processor 1218 can be coupled to a GPS chipset 1236, which can be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 1202. In addition to, or instead of, the information from the GPS chipset 1236, the WTRU 1202 can receive location information from a terminal (e.g., a base station) over the air interface 1215 and / or determine its own location based on the timing of signals received from two or more nearby base stations. It will be understood that the WTRU 1202 can acquire location information using any suitable location determination method while maintaining consistency with the embodiments.

[0128] The processor 1218 can be further coupled to other peripherals 1238, which may include one or more software modules and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 1238 may include an accelerometer, orientation sensor, motion sensor, proximity sensor, e-compass, satellite transceiver, digital camera and / or video recorder (for photos and / or videos), Universal Serial Bus (USB) port, vibration device, TV transceiver, hands-free headset, Bluetooth® module, frequency modulation (FM) radio unit, as well as software modules such as digital music player, media player, video game player module, and internet browser.

[0129] For example, the WTRU1202 can be configured to transmit and / or receive radio signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, tablet computers, personal computers, radio sensors, consumer electronics, or any other terminal capable of receiving and processing compressed video communications.

[0130] WTRU1202 and / or a communication network (e.g., communication network 1304) can implement radio technologies such as Universal Mobile Communications System (UMTS) Terrestrial Radio Access (UTRA) that can establish the air interface 1215 using broadband CDMA (WCDMA®). WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink Packet Access (HSDPA) and / or High Speed ​​Uplink Packet Access (HSUPA). WTRU1202 and / or a communication network (e.g., communication network 1304) may implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA) that can establish the air interface 1215 using Long-Term Evolution (LTE) and / or LTE Advanced (LTE-A).

[0131] WTRU1202 and / or communication networks (e.g., communication network 1304) can implement radio technologies such as IEEE 802.16 (e.g., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM®), Enhanced Data rates for GSM Evolution (EDGE), and GSM EDGE (GERAN). WTRU1202 and / or communication networks (e.g., communication network 804) can implement radio technologies such as IEEE 802.11 or IEEE 802.15.

[0132] It should be noted that various hardware elements in one or more of the embodiments described are referred to as “modules” and perform (i.e., execute, perform, etc.) various functions associated with each module as described herein. Where used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more memory devices) that is considered suitable by those skilled in the art for a given implementation. Each described module may also include executable instructions for performing one or more functions described as being performed by each module, and it should be noted that these instructions may take the form of hardware (i.e., wiring) instructions, firmware instructions, and / or software instructions, or may include them, and may be stored in one or more of any suitable non-temporary computer-readable media, generally referred to as RAM, ROM, etc.

[0133] While features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. In addition, the methods described herein can be implemented in computer programs, software, or firmware embedded in computer-readable media for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital multi-purpose disks (DVDs). A processor working in conjunction with software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A video encoding method, The decision is made to use merge mode to predict at least the current block in the current image, The process involves determining the current merge candidate from among multiple merge candidates, wherein the current merge candidate is used to predict the current block. The movement information of the current block is derived from the movement information of the current merge candidate, The method involves deriving the weight index of the current block from the weight index of the current merge candidate, wherein the weight index of the current block represents the first and second weights used in the weighted biprediction of the current block, and at least one of the first and second weights has a value not equal to 0, 0.5, or 1. The method involves obtaining a prediction of the current block from the motion information of the current block as a weighted sum of the first reference block of the first reference image and the second reference block of the second reference image, wherein the first reference block is weighted by the first weight, and the second reference block is weighted by the second weight. Encoding an indication that the merge mode is being used to predict the current block, Encoding an indication that the current merge candidate among the multiple merge candidates is being used, In response to the prediction of the current block, encode the current block in the merge mode, A method that includes this.

2. The method of claim 1, further comprising using an affine motion model to predict the current block, and determining that the first and second reference blocks used in the weighted sum are reference subblocks derived from the affine motion model.

3. The method of claim 1, wherein at least one of the first weight and the second weight of the weighted biprediction of the current block is identified from a predetermined set of weights using the weight index.

4. The method of claim 1, wherein the plurality of merge candidates include at least one spatially adjacent block or a temporally adjacent block.

5. A video decoding method, Decode an indication that merge mode is being used to predict at least the current block in the current image, Obtain an indication that the current merge candidate among multiple merge candidates is being used to predict the current block, The movement information of the current block is derived from the movement information of the current merge candidate, The method involves deriving the weight index of the current block from the weight index of the current merge candidate, wherein the weight index of the current block represents the first and second weights used in the weighted biprediction of the current block, and at least one of the first and second weights has a value not equal to 0, 0.5, or 1. Obtaining a prediction of the current block from a weighted sum of a first reference block of a first reference picture and a second reference block of a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by the second weight, In response to the prediction of the current block, the current block is decoded in the merge mode, A method that includes this.

6. The method of claim 5, further comprising decoding an indication that an affine motion model is used to predict the current block, wherein the first and second reference blocks used in the weighted sum are reference subblocks derived from the affine motion model.

7. The method of claim 5, wherein at least one of the first weight and the second weight of the weighted biprediction of the current block is identified from a predetermined set of weights using the weight index.

8. The method of claim 5, wherein the plurality of merge candidates include at least one spatially adjacent block or a temporally adjacent block.

9. A video encoding device, The decision is made to use merge mode to predict at least the current block in the current image, The process involves determining the current merge candidate from among multiple merge candidates, wherein the current merge candidate is used to predict the current block. The movement information of the current block is derived from the movement information of the current merge candidate, The method involves deriving the weight index of the current block from the weight index of the current merge candidate, wherein the weight index of the current block represents the first and second weights used in the weighted biprediction of the current block, and at least one of the first and second weights has a value not equal to 0, 0.5, or 1. The method involves obtaining a prediction of the current block from the motion information of the current block as a weighted sum of the first reference block of the first reference image and the second reference block of the second reference image, wherein the first reference block is weighted by the first weight, and the second reference block is weighted by the second weight. Encoding an indication that the merge mode is being used to predict the current block, Encoding an indication that the current merge candidate among the multiple merge candidates is being used, In response to the prediction of the current block, encode the current block in the merge mode, A device equipped with a processor configured to perform at least the following.

10. The apparatus of claim 9, wherein the processor is further configured to at least perform the following: use an affine motion model to predict the current block; and determine that the first and second reference blocks used in the weighted sum are reference subblocks derived from the affine motion model.

11. The apparatus of claim 9, wherein at least one of the first and second weights of the weighted biprediction of the current block is identified from a predetermined set of weights using the weight index.

12. The apparatus of claim 9, wherein the plurality of merge candidates include at least one spatially adjacent block or a temporally adjacent block.

13. A video decoding device, Decode an indication that merge mode is being used to predict at least the current block in the current image, Obtain an indication that the current merge candidate among multiple merge candidates is being used to predict the current block, The movement information of the current block is derived from the movement information of the current merge candidate, The method involves deriving the weight index of the current block from the weight index of the current merge candidate, wherein the weight index of the current block represents the first and second weights used in the weighted biprediction of the current block, and at least one of the first and second weights has a value not equal to 0, 0.5, or 1. Obtaining a prediction of the current block from a weighted sum of a first reference block of a first reference picture and a second reference block of a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by the second weight, In response to the prediction of the current block, the current block is decoded in the merge mode, A device equipped with a processor configured to perform at least the following.

14. The apparatus of claim 13, wherein the processor is further configured to at least perform an indication that an affine motion model is being used to predict the current block, and the first and second reference blocks used in the weighted sum are reference subblocks derived from the affine motion model.

15. The apparatus of claim 13, wherein at least one of the first and second weights of the weighted biprediction of the current block is identified from a predetermined set of weights using the weight index.

16. The apparatus of claim 13, wherein the plurality of merge candidates include at least one spatially adjacent block or a temporally adjacent block.

17. A non-temporary computer-readable medium containing instructions, wherein, when executed, the instructions are transmitted to one or more processors. Decode an indication that merge mode is being used to predict at least the current block in the current image, Obtain an indication that the current merge candidate among multiple merge candidates is being used to predict the current block, The movement information of the current block is derived from the movement information of the current merge candidate, The method involves deriving the weight index of the current block from the weight index of the current merge candidate, wherein the weight index of the current block represents the first and second weights used in the weighted biprediction of the current block, and at least one of the first and second weights has a value not equal to 0, 0.5, or 1. Obtaining a prediction of the current block from a weighted sum of a first reference block of a first reference picture and a second reference block of a second reference picture, wherein the first reference block is weighted by the first weight and the second reference block is weighted by the second weight, In response to the prediction of the current block, the current block is decoded in the merge mode, A non-temporary computer-readable medium that enables the execution of a method including [a specific method].

18. A non-temporary computer-readable medium according to claim 17, comprising instructions causing one or more processors to perform a method further comprising decoding an indication that an affine motion model is being used to predict the current block, wherein the first base block and the second base block used in the weighted sum are base subblocks derived from the affine motion model.

19. The non-temporary computer-readable medium of claim 17, wherein at least one of the first and second weights of the weighted biprediction of the current block is identified from a predetermined set of weights using the weight index.

20. The non-temporary computer-readable medium of claim 17, wherein the plurality of merge candidates include at least one of spatially adjacent blocks or temporally adjacent blocks.

Images