Method and device for image encoding and decoding

Abstract

To provide a method for obtaining a temporal motion vector predictor.SOLUTION: A method for obtaining a temporal motion vector predictor is disclosed. A new temporal motion vector predictor is obtained by using the motion vectors of blocks in a collocated image. The block is displaced (with respect to the current block's spatial location) by a motion vector derived from a new temporal motion vector predictor (TMVP) process.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] At least one of the present embodiments relates generally to methods and devices for image encoding and decoding, and more particularly to methods and devices for encoding and decoding information representing motion in an image. [Background technology]

[0002] To achieve high compression efficiency, video coding schemes usually utilize prediction and transformation to exploit spatial and temporal redundancy in the video content. During encoding, an image of the video content is divided into blocks of pixels, which are then partitioned into one or more sub-blocks, hereafter referred to as original sub-blocks. Then, intra- or inter-prediction is applied to each sub-block to exploit intra- or inter-image correlation. Whatever the prediction method used (intra or inter), a predictor sub-block is determined for each original sub-block. Then, a sub-block representing the difference between the original sub-block and the predictor sub-block, often referred to as prediction error sub-block, prediction residual sub-block, or simply residual block, is transformed, quantized, and entropy coded to generate an encoded video stream. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to the transform, quantization, and entropy coding.

[0003] Essentially, a sub-block encoded using inter prediction, i.e., a block encoded using inter mode, is represented by a residual block and motion information indicating where to find a predictor sub-block. In previous generations of video compression standards (e.g., in the international standard named Versatile Video Coding (VVC), which is being developed by a joint collaboration team of ITU-T and ISO / IEC experts known as the Joint Video Experts Team (JVET), or in the standard HEVC (ISO / IEC 23008-2-MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265)), compression gain was obtained by predicting not only the texture of the sub-block, but also the motion information. Summary of the Invention [Problem to be solved by the invention]

[0004] Motion information prediction is mainly based on the assumption that the motion of a subblock is generally correlated with the motion of other subblocks located in its neighborhood. Therefore, the definition of the neighborhood of a subblock is the key point of motion information prediction. In fact, this neighborhood should be large enough to ensure that the best possible motion information predictor is within this neighborhood, but should not be too large to limit the cost of signaling the motion information predictor.

[0005] It is desirable to propose a solution that allows to improve the motion information prediction, i.e. to ensure that the neighborhood used contains the best candidates for the motion information prediction. [Means for solving the problem]

[0006] In a first aspect, one or more of the embodiments provide a method for decoding, the method comprising the steps of identifying an area, called a collocated region, of a first reference image that is collocated with a current block for decoding a current image; obtaining a first motion vector from a position within the collocated region or in a vicinity of the collocated region, pointing on a second reference image; obtaining a position, called a shift position, in the second reference image by applying a displacement to a position in the current block based on the first motion vector; determining whether a second motion vector is available at the shift position, and if so, obtaining a third motion vector based on the second motion vector that is intended to be inserted in a list of motion vector predictor candidates used to reconstruct the motion vector of the current block.

[0007] In a second aspect, one or more of the present embodiments provide a method for encoding, the method comprising the steps of: identifying an area, called a collocated area, of a first reference image that is collocated with a current block for decoding of a current image; obtaining a first motion vector pointing on a second reference image from a position within the collocated area or within a vicinity of the collocated area; obtaining a position, called a shift position, in the second reference image by applying a displacement to a position in the current block based on the first motion vector; determining whether a second motion vector is available at the shift position, and if so, obtaining a third motion vector based on the second motion vector that is intended to be inserted in a list of motion vector predictor candidates used to reconstruct the motion vector of the current block.

[0008] In an embodiment of the method according to the first or second aspect, the first motion vector is rescaled before being used to obtain the shift position to ensure that the first motion vector points on the second reference image.

[0009] In an embodiment of the method according to the first or second aspect, when the shift position is outside the constraint area, the shift position is clipped to lie within the shift area.

[0010] In an embodiment of the method according to the first or second aspect, when the shift position is outside the constraint area, the third motion vector is based on the first motion vector.

[0011] In an embodiment of the method according to the first or second aspect, when a first motion vector pointing to a second reference image cannot be obtained from a position within the collocated area or in a vicinity of the collocated area, the method comprises the steps of determining whether a spatial motion vector is available in a list of motion vector predictor candidates used for decoding the motion vector of the current block, and if a spatial motion vector is available, modifying the first motion vector to a value dependent on the available spatial motion vector.

[0012] In an embodiment of the method according to the first or second aspect, when a first motion vector pointing on a second reference image cannot be obtained from a position within the collocated area or within a vicinity of the collocated area, the method comprises the step of modifying the first motion vector to be a spatial motion vector.

[0013] In an embodiment of the method according to the first or second aspect, a first motion vector is obtained from a position in a vicinity of the collocated area, and the method further comprises the steps of obtaining a fourth motion vector from a position in the collocated area, pointing on a second reference image; obtaining a position, referred to as the second shift position, in the second reference image by applying a displacement to a position in the current block based on the fourth motion vector; determining whether a fifth motion vector is available at the second shift position, and if so, obtaining a sixth motion vector based on the fifth motion vector, intended to be inserted in a list of motion vector predictor candidates used to reconstruct the motion vectors of the current block; and selecting the first available motion vector in an ordered set of motion vectors to be inserted in the list, wherein the ordered set includes at least the third motion vector, the sixth motion vector, the first motion vector and the fourth motion vector, when these motion vectors are available.

[0014] In an embodiment of the method according to the first or second aspect, the ordered set further comprises, when available, motion vectors derived from positions within said collocated region.

[0015] In an embodiment of the method according to the first or second aspect, the current block is divided into sub-blocks, and the step of obtaining the shift position comprises a step of obtaining a shift position for each sub-block by applying a displacement to a position within the sub-block based on the first motion vector, and a third motion vector is obtained for each sub-block if a second motion vector is available at the shift position corresponding to said sub-block.

[0016] In a third aspect, one or more of the embodiments provide a device for decoding, the device comprising an electronic circuit adapted to: identify an area, called a collocated area, of a first reference image that is collocated with a current block in order to decode a current image; obtain a first motion vector pointing on a second reference image from a position within the collocated area or within a vicinity of the collocated area; obtain a position, called a shift position, in the second reference image by applying a displacement to a position in the current block based on the first motion vector; determine whether a second motion vector is available at the shift position, and if so, obtain a third motion vector based on the second motion vector that is intended to be inserted in a list of motion vector predictor candidates used to reconstruct the motion vector of the current block.

[0017] In a fourth aspect, one or more of the embodiments provide a device for encoding, the device comprising an electronic circuit adapted to: identify an area, called a collocated area, of a first reference image that is collocated with a current block to decode a current image; obtain a first motion vector pointing on a second reference image from a position within the collocated area or within a vicinity of the collocated area; obtain a position, called a shift position, in the second reference image by applying a displacement to a position in the current block based on the first motion vector; determine whether a second motion vector is available at the shift position, and if so, obtain a third motion vector based on the second motion vector that is intended to be inserted in a list of motion vector predictor candidates used to reconstruct the motion vector of the current block.

[0018] In an embodiment of the device according to the third or fourth aspect, the first motion vector is rescaled before being used to obtain the shift position to ensure that the first motion vector points on the second reference image.

[0019] In an embodiment of the device according to the third or fourth aspect, the shift position is outside the constraint area and the shift position is clipped to fit within the shift area.

[0020] In an embodiment of the device according to the third or fourth aspect, when the shift position is outside the constrained area, the third motion vector is based on the first motion vector.

[0021] In an embodiment of the device according to the third or fourth aspect, when a first motion vector pointing to a second reference image cannot be obtained from a position within the collocated area or in a vicinity of the collocated area, the device comprises an electronic circuit adapted to determine whether a spatial motion vector is available in a list of motion vector predictor candidates used for decoding a motion vector of a current block, and if a spatial motion vector is available, modify the first motion vector to a value dependent on the available spatial motion vector.

[0022] In an embodiment of the device according to the third or fourth aspect, when a first motion vector pointing on a second reference image cannot be obtained from a position within the collocated area or in the vicinity of the collocated area, the device comprises electronic circuitry adapted to modify the first motion vector to be a spatial motion vector.

[0023] In an embodiment of the device according to the third or fourth aspect, a first motion vector is obtained from a position in a vicinity of the collocated area, and the device further comprises an electronic circuit adapted to: obtain a fourth motion vector from a position in the collocated area, pointing on a second reference image; obtain a position, referred to as the second shift position, in the second reference image by applying a displacement to a position in the current block based on the fourth motion vector; determine whether a fifth motion vector is available at the second shift position and, if available, obtain a sixth motion vector based on the fifth motion vector intended to be inserted in a list of motion vector predictor candidates used to reconstruct the motion vector of the current block; and select the first available motion vector in an ordered set of motion vectors to be inserted in the list, the ordered set including at least the third motion vector, the sixth motion vector, the first motion vector and the fourth motion vector, when these motion vectors are available.

[0024] In an embodiment of the device according to the third or fourth aspect, the ordered set further comprises, when available, motion vectors derived from positions within said collocated region.

[0025] In an embodiment of the device according to the third or fourth aspect, the current block is divided into sub-blocks, and obtaining the shift position includes obtaining a shift position for each sub-block by applying a displacement to a position within the sub-block based on the first motion vector, and a third motion vector is obtained for each sub-block if a second motion vector is available at a shift position corresponding to the sub-block.

[0026] In a fifth aspect, one or more of the embodiments provide a signal comprising data generated in accordance with a method for encoding in accordance with the second aspect or by a device for encoding in accordance with the fourth aspect.

[0027] In a sixth aspect, one or more of the present embodiments provide a computer program comprising program code instructions for carrying out a method according to the first or second aspect.

[0028] In a seventh aspect, one or more of the present embodiments provide information storage means storing program code instructions for carrying out a method according to the first or second aspect. Effect of the Invention

[0029] A novel method and device for image encoding and decoding is provided. [Brief description of the drawings]

[0030] [Figure 1] FIG. 2 shows an example of partitioning undergone by an image made of pixels of an original video. [Diagram 2] 2 illustrates a schematic diagram of a method for encoding a video stream, performed in an encoding module; [Diagram 3] FIG. 1 illustrates a schematic diagram of a method for decoding an encoded video stream (bitstream). [Figure 4A] FIG. 2 illustrates a schematic diagram of a processing module hardware architecture in which various embodiments can be implemented and an encoding or decoding module can be implemented. [Figure 4B] 1 is a block diagram of an example system in which various embodiments may be implemented. [Diagram 5] FIG. 13 illustrates the position of a temporal motion vector predictor in a list of candidates for normal merge mode. [Figure 6]FIG. 13 illustrates motion vector scaling for a temporal motion vector predictor in a list of candidates for normal merge mode. [Figure 7] FIG. 2 is a diagram illustrating spatially neighboring blocks considered in the sub-block temporal motion vector prediction process. [Figure 8] FIG. 2 illustrates an example process that enables derivation of a sub-block temporal motion vector predictor. [Figure 9] FIG. 1 shows a schematic diagram of the overall improved TMVP derivation process. [Figure 10] FIG. 1 shows a schematic diagram of a first variant of the overall improved TMVP derivation process. [Figure 11] FIG. 13 shows a schematic diagram of a second variant of the overall improved TMVP derivation process. [Figure 12] FIG. 2 illustrates a schematic diagram of an example clipping process; [Figure 13] FIG. 13 shows a schematic diagram of a third variant of the overall improved TMVP derivation process. [Figure 14] FIG. 13 shows an outline of a fourth variant of the overall improved TMVP derivation process. [Figure 15] FIG. 13 shows a schematic diagram of a fifth variant of the overall improved TMVP derivation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] In the following description, some embodiments use tools developed in the context of VVC or in the context of HEVC, but these embodiments are not limited to video encoding / decoding methods corresponding to VVC or HEVC, but apply to other video encoding / decoding methods, or even to any method in which an image is predicted from another image.

[0032] 1 illustrates an example of partitioning undergone by an image of pixels 11 of an original video 10. Here, a pixel is considered to be composed of three components, namely a luminance component and two chrominance components. However, the following embodiments are adapted to images composed of pixels containing another number of components, for example gray level images in which the pixels contain one component, or images composed of pixels containing three color components and a transparency component and / or a depth component.

[0033] An image is divided into multiple coding entities. First, as represented by reference 13 in FIG. 1, an image is divided into a grid of blocks called coding tree units (CTUs). A CTU is composed of an N×N block of luminance samples together with two corresponding blocks of chrominance samples. N is generally a power of 2, e.g., has a maximum value of "128". Second, an image is divided into one or more groups of CTUs. For example, it can be divided into one or more tile rows and tile columns, where a tile is a series of CTUs that covers a rectangular area of ​​the image. In some cases, a tile can be divided into one or more bricks, each of which consists of at least one row of CTUs in the tile. On top of the concept of tiles and bricks, there is another encoding entity called a slice, which can include at least one tile of an image, or at least one brick of a tile.

[0034] In the example in FIG. 1, an image 11 is divided into three slices S1, S2, S3, as represented by reference 12, each including a number of tiles (not represented).

[0035] As represented by reference 14 in FIG. 1, a CTU can be partitioned into a hierarchical tree of one or more sub-blocks called coding units (CUs). A CTU is the root (i.e., parent node) of the hierarchical tree and can be partitioned into multiple CUs (i.e., child nodes). Each CU becomes a leaf of the hierarchical tree if it is not further partitioned into smaller CUs, or becomes a parent node of smaller CUs (i.e., child nodes) if it is further partitioned. Several types of hierarchical trees can be applied, including, for example, quad trees, binary trees, and ternary trees. In a quad tree, a CTU (or CU) can be partitioned (i.e., can be the parent node of) "4" square CUs of equal size. In a binary tree, a CTU (or CU) can be partitioned horizontally or vertically into "2" rectangular CUs of equal size. In a ternary tree, a CTU (or CU) can be partitioned horizontally or vertically into "3" rectangular CUs. For example, a CU of height N and width M is divided vertically (or horizontally) into a first CU of height N (or N / 4) and width M / 4 (or M), a second CU of height N (or N / 2) and width M / 2 (or M), and a third CU of height N (or N / 4) and width M / 4 (or M).

[0036] In the example of FIG. 1, CTU 14 is first partitioned into "4" square CUs using quad-tree type partitioning. The top-left CU is a leaf of the hierarchical tree since it is not further partitioned, i.e., it is not a parent node of any other CU. The top-right CU is further partitioned into "4" smaller square CUs, again using quad-tree type partitioning. The bottom-right CU is partitioned vertically into "2" rectangular CUs using binary tree type partitioning. The bottom-left CU is partitioned vertically into "3" rectangular CUs using ternary tree type partitioning.

[0037] During image encoding, the partitioning is adaptive, and each CTU is partitioned to optimize the compression efficiency on a CTU basis.

[0038] In some compression methods, the concept of prediction unit (PU) and transform unit (TU) has emerged. In that case, the coding entity used for prediction (i.e., PU) and the coding entity used for transformation (i.e., TU) can be a subdivision of CU. For example, as shown in FIG. 1, a CU of size 2N×2N can be divided into PUs 1411 of size N×2N or size 2N×N. In addition, a CU can be divided into "4" TUs 1412 of size N×N, or "16" TUs of size (N / 2)×(N / 2).

[0039] In this application, the terms "block" or "image block" or "sub-block" may be used to refer to any one of a CTU, a CU, a PU, and a TU. In addition, the terms "block" or "image block" may be used to refer to macroblocks, partitions, and sub-blocks, as defined in H.264 / AVC or other video coding standards, and more generally to refer to arrays of samples of various sizes.

[0040] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture", "subpicture", "slice" and "frame" may be used interchangeably.

[0041] Fig. 2 shows a schematic diagram of a method for encoding a video stream, which is executed by an encoding module. Variations of this method for encoding are contemplated, but for the sake of clarity, in the following, the method for encoding of Fig. 2 is described without describing all possible variations.

[0042] The encoding of the current original image 201 starts with the partitioning of the current original image 201 during step 202, as described in relation to Fig. 1. Thus, the current image 201 is partitioned into CTUs, CUs, PUs, TUs, etc. For each block, the encoding module decides the coding mode between intra-prediction and inter-prediction.

[0043] The intra prediction, represented by step 203, consists of predicting the pixels of the current block from a prediction block derived from the pixels of a reconstructed block located within a causal neighborhood of the current block to be coded, according to an intra prediction method. The result of the intra prediction is a prediction direction indicating which pixels of the blocks within the neighborhood to use, and a residual block resulting from the calculation of the difference between the current block and the prediction block.

[0044] Inter prediction consists of predicting the pixels of the current block from a block of pixels, called a reference block, of an image preceding or succeeding the current image (this image is called the reference image). According to the inter prediction method, during the coding of the current block, the block of the reference image that is closest to the current block according to a similarity criterion is determined by a motion estimation step 204. During step 204, a motion vector is determined that indicates the position of the reference block in the reference image. This motion vector is used during a motion compensation step 205, during which a residual block is calculated in the form of the difference between the current block and the reference block.

[0045] In the original video compression standards, the unidirectional inter prediction mode described above was the only inter mode available. As video compression standards have evolved, the family of inter modes has grown significantly and now includes many different inter modes.

[0046] During a selection step 206, among the tested prediction modes (intra prediction modes, inter prediction modes), the prediction mode that optimizes the compression performance according to a rate / distortion criterion (i.e., the RDO criterion) is selected by the encoding module.

[0047] When a prediction mode is selected, the residual block is transformed during step 207 and quantized during step 209. Note that the encoding module can skip the transform and apply quantization directly to the untransformed residual signal.

[0048] When the current block is coded according to an intra prediction mode, during step 210, the prediction direction and the transformed and quantized residual block are encoded by an entropy encoder.

[0049] When the current block is encoded according to an inter prediction mode, in step 208, the motion data associated with this inter prediction mode is coded.

[0050] In general, two modes can be used to encode motion data, called AMVP (Adaptive Motion Vector Prediction) and merging, respectively.

[0051] AMVP essentially signals the reference pictures used to predict the current block, the motion vector predictor index, and the motion vector difference (also called the motion vector residual).

[0052] The merge mode is to signal the index of some motion data collected in a list of motion data predictors. The list is made of "5" or "7" candidates and is constructed in the same way at the decoder side and the encoder side. Thus, the merge mode aims to derive some motion data obtained from the merge list. The merge list generally contains motion data associated with some spatial and temporal neighboring blocks that are available in a reconstructed state when the current block is being processed. The merge mode can take several forms, including a normal merge mode and a sub-block merge mode. The list of candidates for each of these two merge modes includes a temporal motion vector predictor (TMVP).

[0053] In the following, we use the terms motion data, motion information, or motion vector interchangeably. Thus, the term motion vector covers all information representing the motion of a block, including at least one index representing one reference image, a motion vector represented by an index representing a motion vector predictor, and a difference between the motion vector predictor and a predicted motion vector, or only the motion vector.

[0054] Figure 5 represents the location of a temporal motion vector predictor, hereafter called normal temporal motion vector predictor (RTMVP), in the list of candidates for the normal merge mode. The RTMVP is derived from a motion vector corresponding to a position H, which is located at the lower right corner of a block 51 collocated with the current block 50. If no motion data is available at position H, the RTMVP is derived from motion data at a center position C of the collocated block 51. The block 51 belongs to a particular reference picture signaled in the slice header, called the collocated image. The RTMVP is then obtained by rescaling the obtained motion vector so that the rescaled motion vector points onto a reference picture at a first position in a reference picture buffer (hereafter also called the decoded picture buffer with reference 219).

[0055] FIG. 6 illustrates motion vector scaling for the temporal motion vector predictor among a list of candidates for the normal merge mode.

[0056] In Fig. 6, a current picture 62 includes a current block 64 to encode. The motion data of the current block 64 is encoded in normal merge mode using the motion data of a collocated block 65 in a collocated picture 63. The motion data of the collocated block 65 includes a motion vector 650 pointing to an area in a reference picture 60. The RTMVP corresponding to the motion vector 640 is obtained by rescaling the motion vector 650. As can be noted, the RTMVP 640 has the same direction as the motion vector 650, but points to a reference area in picture 61. Picture 61 is the first picture in the decoded picture buffer 219.

[0057] The sub-block merging mode uses sub-block temporal motion vector prediction to generate a sub-block temporal motion vector predictor (SbTMVP). SbTMVP differs from RTMVP in two main aspects:

[0058] · RTMVP predicts motion at the block level, while SbTMVP predicts motion at the sub-block level.

[0059] · RTMVP is derived from the co-located blocks in the co-located picture, but before deriving SbTMVP from the shifted position of the current block and its co-located blocks in the co-located picture, the position of the current block is first shifted. The shift, hereafter called motion shift, is obtained from the motion vectors of blocks spatially neighboring the current block.

[0060] FIG. 8 illustrates an example of a process that allows for deriving a sub-block temporal motion vector predictor.

[0061] Sub-block motion vector prediction predicts the motion vector of a sub-block within the current block 810 of the current image 81 in two steps.

[0062] In the first step, the spatially neighboring blocks of the current block 810 are examined. Figure 7 represents the spatially neighboring blocks considered in the sub-block temporal motion vector prediction process. As can be seen in Figure 7, four blocks are considered: two blocks A1, A0 located at the bottom left corner of the block 810, and two blocks B1, B0 located at the top right corner of the block 810. The spatially neighboring blocks are examined in the order A1, B1, B0, A0. As soon as a spatially neighboring block with a motion vector pointing to the collocated image 80 in this order is identified, this motion vector is selected to be the motion shift to be applied. If no such motion vector is identified from the spatially neighboring blocks A1, B1, B0, A0, the motion shift is set to be (0, 0), i.e. no motion.

[0063] In the second step, the motion shift identified in the first step is applied to the position of the current block 810 (i.e., added to the coordinates of the current block 810'). Sub-block level motion data (motion vectors and reference indices) are then derived from block 800 of the co-located image 80 that is co-located with the shifted position of the current block 810. In the example of FIG. 8, it is assumed that the motion shift is set to be the motion of block A1'. For each sub-block of the current block 810, the motion data of its corresponding sub-block in block 800 (the smallest motion grid covering the center sample) is used to derive the motion data for the sub-block of the current block 810. The SbTMVP derivation is then completed by applying temporal motion vector scaling to the derived motion vectors for each sub-block and aligning the reference images of these derived motion vectors to that of the current block 810.

[0064] The sub-block size used in SbTMVP is generally 8 x 8. In that case, the SbTMVP mode is only applicable to a block when both the width and height of the block are equal to or greater than "8".

[0065] As can be seen from the above, when the normal merge mode is applied to the current block of the current picture, the RTMVP is the rescaled motion vector of the block of the co-located picture that is co-located with the current block. Therefore, the derivation of RTMVP does not take into account the displacement of the block between the current picture and the co-located picture.

[0066] On the other hand, SbTMVP is derived from a block of a collocated image that is collocated with the shifted block obtained by shifting the current block. The shift of the current block is determined from blocks that are spatially neighboring to the current block. The difficulty with this approach is that the blocks that are spatially neighboring to the current block may not contain motion data (e.g., because they were encoded in intra mode) or may not be available (e.g., when the current block is located at the edge of the current image).

[0067] The embodiments described below aim to overcome the above-mentioned drawbacks.

[0068] In particular, at least one embodiment described below aims to create a new temporal motion vector predictor.

[0069] In addition, at least one embodiment described below utilizes features of both TMVP and SbTMVP.

[0070] At least one embodiment, in normal merge mode, uses a process similar to the sub-block merge mode, but uses a displaced motion vector predictor that uses temporal motion vector prediction to displace motion vectors.

[0071] At least one embodiment uses a fallback solution as the default TMVP when the above mentioned motion vectors cannot be derived.

[0072] At least one embodiment uses non-sub-block motion vector derivation in sub-block merging mode to limit the complexity of the motion vector derivation process.

[0073] At least one embodiment varies the condition of the motion vector displacement derivation constraints to use a fallback solution.

[0074] At least one embodiment uses more initial vectors for the center displacement of the current block using spatial merging candidates in addition to those of the temporal merging candidates.

[0075] At least one embodiment uses the temporal merge candidates as alternative initialization vectors for standard SbTMVP sub-block processing.

[0076] Once predicted, the motion information is then encoded by an entropy encoder along with the transformed and quantized residual block during step 210. Note that the encoding module can bypass both the transform and the quantization, i.e., entropy encoding is applied to the residual without application of the transform or quantization process. The result of the entropy encoding is inserted into the encoded video stream (i.e., bitstream) 211.

[0077] It should be noted that the entropy encoder can be implemented in the form of a context-adaptive binary arithmetic coder (CABAC), which encodes binary symbols, which keeps the complexity low and allows probability modeling for the more frequently used bits of any symbol.

[0078] After the quantization step 209, the current block is reconstructed so that the pixels corresponding to it can be used for future prediction. This reconstruction phase is also called the prediction loop. Thus, during step 212, inverse quantization is applied to the transformed and quantized residual block, and during step 213, an inverse transform is applied. According to the prediction mode used for the current block, obtained during step 214, the prediction block of the current block is reconstructed. If the current block is encoded according to an inter prediction mode, the encoding module applies motion compensation to the reference block, when appropriate, during step 216, using the motion information of the current block. If the current block is encoded according to an intra prediction mode, during step 215, the prediction direction corresponding to the current block is used to reconstruct the reference block of the current block. The reference block and the reconstructed residual block are added to obtain the reconstructed current block.

[0079] Following the reconstruction, during step 217, in-loop post-filtering is applied to the reconstructed blocks, intended to reduce encoding artifacts. This post-filtering is called in-loop post-filtering because it is performed within the prediction loop in order to obtain the same reference image in the encoder as in the decoder, thus avoiding drift between the encoding and decoding processes. For example, in-loop post-filtering includes deblocking filtering and SAO (Sample Adaptive Offset) filtering. Parameters describing the activation or deactivation of the in-loop deblocking filter, and the characteristics of the in-loop deblocking filter when activated, are introduced into the encoded video stream 211 during the entropy encoding step 210.

[0080] When a block is reconstructed, it is inserted into a reconstructed image stored in a decoded picture buffer (DPB) 219 during step 218. The reconstructed image stored in this way can then serve as a reference image for other images to be coded.

[0081] Fig. 3 shows a schematic diagram of a method for decoding an encoded video stream (i.e., a bitstream) 211, encoded according to the method described in relation to Fig. 2. The method for decoding is executed by a decoding module. Variations of this method for decoding are contemplated, but for the sake of clarity, in the following, the method for decoding of Fig. 3 is described without describing all possible variations.

[0082] The decoding is performed block by block: for the current block, it starts with an entropy decoding of the current block, during step 310. The entropy decoding makes it possible to obtain the prediction mode of the current block.

[0083] If the current block has been encoded according to an intra prediction mode, entropy decoding makes it possible to obtain information representative of the intra prediction direction and the residual block.

[0084] If the current block is encoded according to an inter prediction mode, entropy decoding allows obtaining information representing motion data and the residual block. When appropriate, during step 308, the motion data is reconstructed for the current block according to AMVP or merge mode. In merge mode, the motion data obtained by entropy decoding includes an index in a list of motion vector predictor candidates. The decoding module applies the same process as the encoding module to build the candidate lists for normal merge mode and sub-block merge mode. Using the reconstructed list and index, the decoding module can retrieve the motion vectors used to predict the motion vector of the block.

[0085] The method for decoding includes steps 312, 313, 315, 316, 317, which are identical in all respects to steps 212, 213, 215, 216, 217, respectively, of the method for encoding. At the encoding module level, step 214 includes a mode selection process that evaluates each mode and selects the best mode according to a rate-distortion criterion, but step 314 only reads information representing the selected mode in the bitstream 211. The decoded blocks are saved in a decoded picture, which is stored in a DPB 319 in step 318. When the decoding module decodes a given picture, the picture stored in the DPB 319 is identical to the picture stored in the DPB 219 by the encoding module during the encoding of the given picture. The decoded picture can also be output by the decoding module, for example for display.

[0086] FIG. 4A illustrates, in a schematic manner, an example of a hardware architecture of a processing module 40 capable of implementing an encoding module or a decoding module capable of implementing the method for encoding of FIG. 2 and the method for decoding of FIG. 3, respectively, as modified according to different aspects and embodiments. The processing module 40 comprises a processor or CPU (Central Processing Unit) 400, including, by way of non-limiting example, one or more microprocessors, general purpose computers, special purpose computers, and processors based on multi-core architectures; a random access memory (RAM) 401; a read-only memory (ROM) 402; a storage unit 403, which may include non-volatile and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drive, and / or optical disk drive, or storage media reader such as a SD (Secure Digital) card reader, and / or hard disk drive (HDD), and / or a network-accessible storage device; and at least one communication interface 404, connected by a communication bus 405, for exchanging data with other modules, devices, or equipment. The communication interface 404 may include, but is not limited to, a transceiver configured to transmit and receive data over a communication channel. The communication interface 404 may include, but is not limited to, a modem or a network card.

[0087] If processing module 40 implements a decoding module, communication interface 404 may, for example, enable processing module 40 to receive an encoded video stream and provide a decoded video stream. If processing module 40 implements an encoding module, communication interface 404 may, for example, enable processing module 40 to receive original image data to encode and provide an encoded video stream.

[0088] The processor 400 is capable of executing instructions loaded into the RAM 401 from the ROM 402, from an external memory (not shown), from a storage medium, or from a communication network. When the processing module 40 is powered on, the processor 400 is capable of reading instructions from the RAM 401 and executing them. These instructions form a computer program that causes the processor 400 to perform, for example, a decoding method as described in relation to Fig. 3 or an encoding method described in relation to Fig. 2, the decoding and encoding methods including various aspects and embodiments described below in this document.

[0089] All or some of the algorithms and steps of the encoding or decoding methods may be implemented in software form by execution of a set of instructions by a programmable machine such as a DSP (Digital Signal Processor) or a microcontroller, or may be implemented in hardware form by a machine or dedicated components such as an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).

[0090] FIG. 4B illustrates a block diagram of an example of a system 4 in which various aspects and embodiments are implemented. The system 4 may be embodied as a device including various components described below and configured to execute one or more of the aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of the system 4, alone or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one embodiment, the system 4 includes one processing module 40 that implements a decoding module or an encoding module. However, in another embodiment, the system 4 may include a first processing module 40 that implements a decoding module and a second processing module 40 that implements an encoding module, or one processing module 40 that implements a decoding module and an encoding module. In various embodiments, system 40 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communications bus or through dedicated input and / or output ports. In various embodiments, system 4 is configured to implement one or more of the aspects described in this document.

[0091] The system 4 includes at least one processing module 40 capable of implementing either an encoding module or a decoding module, or both.

[0092] Input to processing module 40 may be provided through various input modules, as shown in block 42. Such input modules include, but are not limited to, (i) a radio frequency (RF) module that receives an RF signal, e.g., transmitted wirelessly by a broadcast station, (ii) a component (COMP) input module (or set of COMP input modules), (iii) a universal serial bus (USB) input module, and / or (iv) a high-definition multimedia interface (HDMI) input module. Other examples not shown in FIG. 4B include composite video.

[0093] In various embodiments, the input modules of block 42 have associated respective input processing elements as known in the art. For example, the RF module may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select a signal frequency band (for example), which in some embodiments may be referred to as a channel, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF module of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner to perform various of these functions, including down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to baseband) or to baseband. In one embodiment of a set-top box, the RF module and its associated input processing elements perform frequency selection by receiving RF signals transmitted over a wired (e.g., cable) medium and filtering, downconverting, and filtering again to a desired frequency band. Various embodiments shuffle the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF module includes an antenna.

[0094] Additionally, the USB and / or HDMI modules may include respective interface processors for connecting system 4 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or, if desired, in processing module 40. Similarly, aspects of the USB or HDMI interface processing may be implemented in a separate interface IC or, if desired, in processing module 40. The demodulated, error corrected, multiplexed stream is provided to processing module 40.

[0095] The various elements of system 4 may be provided within a unified housing in which the various elements may be interconnected and transmit data between them using suitable connection configurations, such as internal buses, wiring, and printed circuit boards as are known in the art, including an inter-IC (I2C) bus. For example, in system 4, processing module 40 is interconnected to the other elements of system 4 by bus 405.

[0096] The communication interface 404 of the processing module 40 enables the system 4 to communicate over a communication channel 41. The communication channel 41 can be implemented, for example, in a wired medium and / or wirelessly.

[0097] In various embodiments, data is streamed or otherwise provided to system 4 using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received over a communication channel 41 and communication interface 404 adapted for Wi-Fi communication. The communication channel 41 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 4 using a set-top box that delivers data over an HDMI connection of input block 42. Still other embodiments provide streamed data to system 4 using an RF connection of input block 42. As indicated above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0098] System 4 can provide output signals to various output devices, including a display 46, speakers 47, and other peripheral devices 48. The display 46 in various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 46 can be for a television, a tablet, a laptop, a cell phone (mobile phone), or other device. The display 46 can be integrated with other components (e.g., as in a smartphone) or can be separate (e.g., an external monitor for a laptop). In various example embodiments, the other peripheral devices 46 include one or more of a standalone digital video disc (or digital versatile disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 48 that provide functionality based on the output of system 4. For example, a disc player performs the function of playing the output of system 4.

[0099] In various embodiments, control signals are communicated between system 4 and display 46, speaker 47, or other peripheral device 48 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow inter-device control with or without user intervention. The output devices may be communicatively coupled to system 4 via dedicated connections through respective interfaces 43, 44, 45. Alternatively, the output devices may be connected to system 4 using communication channel 41 via communication interface 404. Display 46 and speaker 47 may be integrated into a single unit with other components of system 4, for example, in an electronic device such as a television. In various embodiments, display interface 43 includes a display driver, for example, a timing controller (T Con) chip.

[0100] Alternatively, display 46 and speakers 47 can be separate from one or more of the other components, for example, if the RF module of input 42 is part of a separate set-top box. In various embodiments in which display 46 and speakers 47 are external components, the output signal can be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0101] Various implementations include decoding. As used in this application, "decoding" can encompass all or part of the processes performed, for example, on a received encoded sequence to generate a final output suitable for display. In various embodiments, such processes include processes typically performed by a decoder, such as one or more of entropy decoding, inverse quantization, inverse transform, and prediction. In various embodiments, such processes also include, or alternatively include, processes performed by a decoder of various implementations or embodiments described in this application, such as a process for determining a temporal motion vector predictor for a coding unit that encodes according to a merge mode.

[0102] As a further example, in one embodiment, "decoding" refers only to entropy decoding (step 310 in FIG. 3). Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process generally will be clear based on the context of the specific description and is believed to be well understood by one of ordinary skill in the art.

[0103] Various implementations include encoding. Similar to the above description of "decoding," as used in this application, "encoding" can encompass all or part of the processes performed, for example, on an input video sequence to generate an encoded bitstream. In various embodiments, such processes include processes typically performed by an encoder, such as one or more of partitioning, prediction, transform, quantization, and entropy encoding. In various embodiments, such processes also include, or alternatively include, processes performed by an encoder of various implementations or embodiments described in this application, such as a process for determining a temporal motion vector predictor for a coding unit that encodes according to a merge mode.

[0104] As a further example, in one embodiment, "encoding" refers to encoding mode selection (step 206 in FIG. 2) and entropy encoding (step 210 in FIG. 2). Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process generally will be clear based on the context of the specific description and is believed to be well understood by one of ordinary skill in the art.

[0105] It should be noted that syntax element names, prediction mode names, and tool names as used herein are descriptive terms, and as such, they do not preclude the use of other syntax element names, prediction mode names, or tool names.

[0106] It should be understood that when a diagram is presented as a flow diagram, it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0107] Various embodiments refer to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered. Rate-distortion optimization is usually formulated as minimizing the rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist for solving the rate-distortion optimization problem. For example, the approach can be based on an extensive test of all encoding options, including all considered modes or coding parameter values, with a full evaluation of their coding costs and associated distortion of the reconstructed signal after encoding and decoding. To save on encoding complexity, faster approaches can also be used, in particular those that use an approximated distortion calculation based on a predicted or predicted residual signal, rather than a reconstructed one. A mixture of these two approaches can also be used, such as by using an approximated distortion for only some of the possible encoding options and a full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches utilize any of a variety of techniques to perform the optimization, but the optimization is not necessarily a full evaluation of both the coding cost and the associated distortion.

[0108] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if described only in the context of a single form of implementation (e.g., described only as a method), the implementation of the described features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. A method may be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the transfer of information between end users.

[0109] References to "one embodiment" or "embodiment" or "one implementation" or "implementation" and other variations thereof mean that particular features, structures, characteristics, etc. described in connection with an embodiment are included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places throughout this application are not necessarily all referring to the same embodiment.

[0110] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, inferring information from other information, retrieving information from memory, or obtaining information from another device, module, or user, for example.

[0111] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, inferring information, or estimating information.

[0112] In addition, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" may generally be included in various ways during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, inferring information, or deducing information.

[0113] For example, in the cases of "A / B," "A and / or B," and "at least one of A and B," it will be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first enumerated alternative (A), or the selection of only the second enumerated alternative (B), or the selection of both alternatives (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first enumerated alternative (A), or the selection of only the second enumerated alternative (B), or the selection of only the third enumerated alternative (C), or the selection of only the first and second enumerated alternatives (A and B), or the selection of only the first and third enumerated alternatives (A and C), or the selection of only the second and third enumerated alternatives (B and C), or the selection of all three alternatives (A and B and C). This can be expanded as many times as the number of items listed, as would be apparent to one of ordinary skill in this and related arts.

[0114] Also, as used herein, the term "signal" refers to, among other things, indicating something to a corresponding decoder. For example, in an embodiment, an encoder signals a syntax element or parameter associated with a selected motion vector predictor in a list of motion vectors for a coding unit encoded in merge mode. In this way, in an embodiment, the same parameters are used at both the encoder side and the decoder side. Thus, for example, an encoder can send a specific parameter to a decoder so that the decoder can use the same specific parameter (explicit signaling). Conversely, if the decoder already has a specific parameter, etc., it can use signaling without sending it to simply allow the decoder to know and select the specific parameter (implicit signaling). By avoiding the transmission of any actual function, bit savings are realized in various embodiments. It should be understood that signaling can be achieved in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. Although the above relates to the verb form of the word "signal," the word "signal" can also be used as a noun in this specification.

[0115] As will be apparent to one skilled in the art, the implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry an encoded video stream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding the encoded video stream and modulating a carrier with the encoded video stream. The information that the signal carries can be, for example, analog information or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

[0116] Figure 9 shows the overall improved TMVP derivation process in schematic form. The process of Figure 9 is performed by processing module 40 for a current block when processing module 40 implements an encoder or a decoder. In particular, the TMVP derivation process is performed during steps 208 and 308. The new TMVP derived from the process of Figure 9 can be used, for example, as the TMVP for the normal merge mode.

[0117] Before beginning the TMVP derivation process, processing module 40 identifies blocks in a collocated image that are collocated with the current block.

[0118] In step 300, the processing module 40 determines whether a motion vector mv can be obtained from the position H of a block co-located with the current block in a collocated image as defined in relation to Fig. 5. If a motion vector mv can be derived from the position H, this motion vector mv is stored by the processing module 40 as a temporal motion vector and step 300 is followed by step 302.

[0119] If not, step 300 is followed by step 301, during which the processing module 40 determines whether a motion vector mv can be derived from a center position C of a block collocated with the current block in a collocated image as defined in relation to Fig. 5. If a motion vector mv can be derived from the center position C, this motion vector mv is stored by the processing module 40 as a temporal motion vector, and step 301 is followed by step 302. Steps 300 and 301 make it possible to obtain a temporal motion vector from a position in a block collocated with the current block or in a neighborhood of the block in the collocated image.

[0120] If a temporal motion vector cannot be determined in step 300 or step 301, processing module 40 stops the TMVP derivation process in step 311. In that case, the TMVP cannot be derived by the TMVP derivation process.

[0121] In step 302, since the temporal motion vector mv can point to any image in the reference image buffer, the derived temporal motion vector mv is rescaled by the processing module 40 to ensure that this temporal motion vector mv points to the current reference image (i.e., the reference image at the first position in the reference image buffer).

[0122] In step 303, the processing module 40 uses the rescaled temporal motion vector mv to displace the center of the current block (represented by position C) to location P=mv+C.

[0123] In step 304, the processing module 40 applies clipping to the position P to avoid pointing outside a predefined area (i.e. to avoid pointing outside the CTU that contains the position C, for example). Steps 303 and 304 make it possible to obtain the position P in the current reference image by applying a displacement to the center position C of the current block based on the temporal motion vector mv.

[0124] In step 305, processing module 40 determines whether a motion vector is available at position P. This motion vector is called "mv'".

[0125] If the motion vector mv' is available, step 305 is followed by step 307. During step 307, the motion vector mv' is rescaled to point to the current reference image as motion vector mv" and is retained as the final motion vector for the TMVP derivation process (i.e., the TMVP).

[0126] If the motion vector mv' is not available, then in step 306, the motion vector mv is retained as the final motion vector (ie, the TMVP).

[0127] Steps 306 and 308 are followed by step 311. In step 311, the TMVP derived from the TMVP derivation process is inserted into a list of motion vector predictor candidates that are used to encode or decode (i.e., reconstruct) the motion vector of the current block.

[0128] As can be seen, given the availability of the vector, either mv or mv" can be used as the TMVP.

[0129] Note that the rescaling process is the same as in HEVC, as explained earlier.

[0130] In an embodiment, the availability of a motion vector at position H (tested in step 300) depends on the border of the CTU that contains the current block. When the bottom-right sub-block corresponding to position H is outside the current CTU, the motion vector is considered not available and a motion vector at position C in the co-located image is obtained.

[0131] In a variant, a sub-block of the current CTU located in the lower right corner of the current CTU can be used to avoid checking unnecessary CTU boundary constraints.

[0132] In another variation, location C is examined first.

[0133] In all cases, if the motion vector does not exist in the collocated image (e.g., the corresponding subblock is in intra mode), the next candidate (e.g., at position C when H is the first position checked) is examined.

[0134] For example, the clipping process applied in step 304 is to clip position P within a given constraint area. In an embodiment, the constraint area corresponds to the CTU containing the current block plus the column of sub-blocks to the right of this CTU (see FIG. 12). The center C displaced to position P is clipped to fit within the constraint area to give position P'. A final motion vector (i.e., TMVP) is then extracted from the sub-block at position P' in the co-located image.

[0135] Figure 10 shows a schematic diagram of a first variant of the overall improved TMVP derivation process. The process of Figure 10 is performed by processing module 40 for a current block when processing module 40 implements an encoder or a decoder. In particular, the TMVP derivation process is performed during steps 208 and 308.

[0136] The variant of Figure 10 modifies the process of Figure 9 when no time motion can be found in steps 300 and 301. Compared to the block diagram of Figure 9, two additional steps 309 and 310 are added between steps 301 and 307. Again, the new TMVP derived from the process of Figure 10 can be used, for example, as the TMVP for the normal merge mode.

[0137] In step 309, the processing module 40 obtains the center position C as the position P.

[0138] In step 310, the processing module 40 checks the availability of a motion vector mv' at a position P in the collocated image.

[0139] If it is available, then in step 307, the motion vector mv' is scaled to become the motion vector mv". Otherwise, no motion vector is obtained from the TMVP derivation process and the process ends in step 311.

[0140] 11 shows a schematic diagram of a second variant of the overall improved TMVP derivation process. The process of FIG. 11 is performed by processing module 40 for a current block when processing module 40 implements an encoder or a decoder. In particular, the TMVP derivation process of FIG. 11 is performed during steps 208 and 308.

[0141] The variant of Figure 11 also modifies the process of Figure 9 when a temporal motion vector cannot be found in steps 300 and 301. In that case, in this variant, the first available spatial motion vector of the merging process (or the first candidate of the merging process, if one exists) is obtained to displace the center of the current block. If no motion vector can be found after all, no temporal motion vector is obtained from the TMVP derivation process. Again, the new TMVP derived from the process of Figure 11 can be used, for example, as the TMVP for the normal merge mode.

[0142] The process of FIG. 11 differs from the process of FIG. 9 as explained below.

[0143] In steps 300 and 301, when no temporal motion vector is found, step 301 is followed by step 4000.

[0144] In step 4000, processing module 40 determines whether a first merge motion vector candidate is available in the list of motion vector candidates for the normal merge mode. The first merge motion vector in the list of candidates for the normal merge mode is a spatial motion vector. In an embodiment, instead of determining whether a first merge motion vector candidate is available, processing module 40 determines whether at least one spatial candidate is available in the normal merge list, and if at least one motion vector candidate is available, module 40 selects the motion vector candidate in the first position in that list.

[0145] If the first merge motion vector is not available, then in step 4005 the processing module 40 uses the center C of the current block for the position P.

[0146] In step 4006, the processing module 40 determines whether a motion vector mv' at the position P is available.

[0147] If mv' is available, then in 4007 the processing module 40 uses mv' to displace the center of the current block.

[0148] Then, in step 307, the processing module 40 rescales the motion vector mv' to obtain the motion vector mv".

[0149] If, in step 4006, mv' is not available, then no motion vector is obtained from the TMVP derivation process and the process stops in step 311.

[0150] If, in step 4000 , a first merged motion vector is available, the processing module 40 rescales this motion vector during a step 4001 .

[0151] In step 4002, the processing module 40 displaces the center position of the current block to position P using the rescaled motion vector.

[0152] In step 4003, the processing module 40 clips the displaced position P.

[0153] During step 4004, processing module 40 checks the availability of a motion vector mv' at position P in the co-located image. If the motion vector mv' is available, it is rescaled to become the motion vector mv" in step 307. If the motion vector mv' is not available, no motion vector is obtained from the TMVP derivation process and the process ends in step 311.

[0154] 13 shows a schematic diagram of a third variant of the overall improved TMVP derivation process. The process of FIG. 13 is performed by processing module 40 for a current block when processing module 40 implements an encoder or a decoder. In particular, the TMVP derivation process of FIG. 13 is performed during steps 208 and 308.

[0155] Again, the new TMVP derived from the process of FIG. 13 can be used, for example, as the TMVP for normal merge mode.

[0156] An alternative to the clipping process is to use a fallback motion vector predictor if the displaced sub-block is outside the constraint area.

[0157] FIG. 13 shows a variant of the TMVP derivation process that avoids the clipping process. In this process of FIG. 13, if the displaced center is outside the constrained area, a fallback to the original motion vector mv (when it was selected) is used. Compared to FIG. 11, two steps (500 and 501) are inserted, which check whether the position P is inside the constrained area. Step 500 is inserted between step 303 and step 305 / 306. In step 500, if the position P is inside the constrained area, the processing module executes step 305. If not, the processing module executes step 306.

[0158] Step 501 is inserted between step 4002 and step 4004. In step 501, if the position P is inside the constraint area, the processing module executes step 4004. Otherwise, step 501 is followed by step 311 and no motion vector is obtained from the derivation process.

[0159] 14 shows a schematic diagram of a fourth variant of the overall improved TMVP derivation process. The process of FIG. 14 is performed by processing module 40 for a current block when processing module 40 implements an encoder or a decoder. In particular, the TMVP derivation process of FIG. 14 is performed during steps 208 and 308.

[0160] Again, the new TMVP derived from the process of FIG. 14 can be used, for example, as the TMVP for normal merge mode.

[0161] In step 1401, the processing module 40 determines whether a motion vector mv0 at a position H in the collocated image is available.

[0162] If the motion vector mv0 is available at position H, in step 1403, the processing module 40 rescales the motion vector mv0 so that the rescaled motion vector mv0 points to the current reference image (i.e., the reference image at the first position in the reference image buffer).

[0163] In step 1404, the processing module 40 displaces the center of the current block to be at position P=C+mv0.

[0164] In step 1405, if the position P is outside the constraint area, the processing module 40 clips the position P.

[0165] In step 1406, processing module 40 determines whether a motion vector mv0' at position P in the collocated image is available.

[0166] If the motion vector mv0′ is available at the position P, then in step 1408, the processing module rescales the motion vector mv0′ to obtain the motion vector mv0″. Step 1408 is followed by step 1421.

[0167] If, in step 1401, a motion vector is not available at position H, the processing module 40 determines, during a step 1402, that motion vectors mv0 and mv0″ are not available. Step 1402 is followed by step 1421.

[0168] If, in step 1406, the motion vector mv0′ is not available at the position P, the processing module 40 determines, during a step 1407, that the motion vector mv0″ is not available. Step 1407 is followed by step 1421.

[0169] In parallel with, or subsequent to, steps 1401 to 1408, the processing module performs steps 1409 to 1416.

[0170] In step 1409, the processing module 40 determines whether the motion vector mv1 at the position C in the collocated image is available.

[0171] If the motion vector mv1 is available at position C, in step 1411, the processing module 40 rescales the motion vector mv1 so that the rescaled motion vector mv1 points to the current reference image (i.e., the reference image at the first position in the reference image buffer).

[0172] In step 1412, the processing module 40 displaces the center of the current block to be at position P=C+mv1.

[0173] In step 1413, if the position P is outside the constraint area, the processing module 40 clips the position P.

[0174] In step 1414, processing module 40 determines whether a motion vector mv1' at position P in the collocated image is available.

[0175] If the motion vector mv1′ is available at the position P, then in step 1416, the processing module rescales the motion vector mv1′ to obtain the motion vector mv1″. Step 1416 is followed by step 1421.

[0176] If, in step 1409, a motion vector is not available at position C, the processing module 40 determines, during a step 1410, that the motion vectors mv1 and mv1″ are not available. Step 1410 is followed by step 1421.

[0177] If, in step 1414, the motion vector mv1′ is not available at the position P, the processing module 40 determines, during a step 1415, that the motion vector mv1″ is not available. Step 1415 is followed by step 1421.

[0178] In parallel with or subsequent to steps 1401 to 1408 and / or steps 1409 to 1416, the processing module 40 executes steps 1417 to 1420.

[0179] In step 1417, the processing module 40 determines a position P to be the center of the current block (P=C).

[0180] During step 1418, the processing module determines whether a motion vector mv3″ is available at position P in the co-located image. If a motion vector is available at position P, then during step 1420, the processing module 40 rescales the motion vector mv3″.

[0181] If this is not the case, during a step 1419, the processing module 40 determines that the motion vector mv3'' is not available. Steps 1419 and 1420 are followed by step 1421.

[0182] During step 1421, the processing module 40 selects the first available motion vector in the ordered set {mv0", mv1", mv3", mv0, mv1, none} as the TMVP. If mv0" is available, the processing module 40 selects mv0" as the TMVP. Otherwise, if mv1" is available, the processing module 40 selects mv1" as the TMVP. Otherwise, if mv3" is available, the processing module 40 selects mv3" as the TMVP. Otherwise, if mv0 is available, the processing module 40 selects MV0 as the TMVP. Otherwise, if mv1 is available, the processing module 40 selects MV1 as the TMVP. Otherwise, no TMVP is selected, i.e., no motion vector is obtained from the TMVP derivation process.

[0183] As described above in connection with FIG. 11 (step 4000), the same process can be applied when the first available spatial motion vector of the merging process (or the first candidate of the merging process, if one exists) is used as the initial motion vector.

[0184] 15 shows a schematic diagram of a fifth variant of the overall improved TMVP derivation process. The process of FIG. 15 is performed by processing module 40 for a current block when processing module 40 implements an encoder or a decoder. In particular, the TMVP derivation process of FIG. 15 is performed during steps 208 and 308.

[0185] The new TMVP derived from the process of FIG. 15 is specifically adapted for the sub-block merging mode.

[0186] In step 1501, processing module 40 determines whether a motion vector mv is available at position H in the co-located image. If the motion vector mv is available, step 1501 is followed by step 1503. Otherwise, processing module 40 executes step 1502, during which processing module 40 determines whether a motion vector mv is available at position C in the co-located image. If the motion vector mv is available, step 1502 is followed by step 1503.

[0187] In step 1502, if the motion vector mv is not available, step 1502 is followed by step 1508.

[0188] In step 1503, the motion vector mv is rescaled by the processing module 40. Step 1503 is the same as step 302.

[0189] In step 1504, the processing module 40 calculates, for each sub-block i of the current block, a position Pi=Ci+mv, where Ci is the position of the centre of the i-th sub-block of the current block.

[0190] In step 1505, the processing module 40 clips each position Pi to fit inside a constrained area, for example inside the CTU that contains the current block.

[0191] In step 1506, the processing module 40 determines, for each sub-block i, whether a motion vector mv'i is available at the position Pi.

[0192] In step 1507, each available motion vector mv'i is rescaled to obtain a motion vector mv"i. When a motion vector mv'i is not available for sub-block i, in step 1517, sub-block i is given a default motion vector, e.g., mv"i=(0,0).

[0193] In step 1508, which is the same as step 4000, the processing module 40 determines whether the first merged motion vector mv is available. If the first merged motion vector mv is available, in step 1509, the processing module 40 rescales this first merged motion vector mv.

[0194] In step 1510, the processing module 40 calculates, for each sub-block i of the current block, the position Pi=Ci+mv.

[0195] In step 1511, the processing module 40 clips each position Pi to fit inside the constraint area.

[0196] In step 1512, the processing module 40 determines, for each sub-block i, whether a motion vector mv'i is available at the position Pi.

[0197] In step 1513, each available motion vector mv'i is rescaled to obtain a motion vector mv"i. When a motion vector mv'i is not available for sub-block i, in step 1517, sub-block i is given a default motion vector.

[0198] If, in step 1508, the processing module 40 determines that the first merged motion vector mv is not available, then step 1508 is followed by step 1514.

[0199] In step 1514, the processing module 40 calculates, for each sub-block i of the current block, a position Pi=Ci.

[0200] In step 1515, processing module 40 determines, for each sub-block i, whether a motion vector mv'i is available at position Pi. In step 1516, each available motion vector mv'i is rescaled to obtain a motion vector mv"i. When a motion vector mv'i is not available for sub-block i, in step 1517, sub-block i is given a default motion vector.

[0201] This application describes various aspects, including tools, features, embodiments, models, methods, and the like. Many of these aspects are specifically described, and often described in a manner that may sound restrictive, at least to illustrate individual features. However, this is for clarity in the description and does not limit the application or scope of the aspects. In fact, all of the different aspects can be combined and interchanged to provide further aspects. Furthermore, aspects can be combined and interchanged with aspects described in prior applications as well.

[0202] The aspects described and contemplated in this application can be implemented in many different forms. Figures 2, 3, 4A, and 4B provide some embodiments, but other embodiments are contemplated, and the descriptions of Figures 2, 3, 4A, and 4B do not limit the breadth of implementations. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0203] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions can be varied or combined.

[0204] Various methods and other aspects described in the present application can be used to modify modules, for example, the motion vector encoding step 208 of a video encoder and / or the motion vector decoding step 308 of a decoder. Moreover, the aspects are not limited to VVC or HEVC, but can be applied, for example, to other standards and recommendations, whether existing or developed in the future, and to extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically precluded, the aspects described in the present application can be used individually or in combination.

[0205] In this application, various numerical values, various locations, and / or various constraint areas are used, and the specific values, locations, and constraint areas are for illustrative purposes and the described aspects are not limited to these specific values, locations, and constraint areas.

[0206] Furthermore, embodiments can include one or more of the following features, devices, or aspects, alone or in any combination, across the various claim categories and types. · Providing a new temporal motion vector predictor. Obtaining a new temporal motion vector predictor by using motion vectors of blocks in the co-located image, The block is displaced (with respect to the spatial location of the current block) by the motion vector MV0 that comes from the original TMVP process. The new temporal motion vector predictor probably undergoes some temporal rescaling. Clipping block displacements to fit inside a constraint area. In cases where the temporal motion vector predictor does not satisfy some constraints, e.g., it points outside the constraint area, apply the original TMVP process. Obtaining a new sub-block temporal motion vector predictor by using the motion vector of a block in a collocated image as the initial motion vector instead of the motion vector of a spatially neighboring block. Obtaining new sub-block temporal motion vector predictors by applying the initial motion vector to each sub-block. Inserting syntax elements into the signaling that allow the decoder to identify the motion vector prediction method to be used. A bitstream or signal including a syntax conveying information generated according to any of the described embodiments. Inserting syntax elements into the signaling that allow the decoder to adapt the motion vector prediction in a manner corresponding to that used by the encoder. · Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal that includes one or more of the described syntax elements, or variations thereof. Creating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments. A method, process, apparatus, instruction storage medium, data storage medium, or signal according to any of the described embodiments. A television, set-top box, cell phone, tablet, or other electronic device that performs motion vector prediction adaptation according to any of the described embodiments. A television, set-top box, cell phone, tablet, or other electronic device that performs the motion vector prediction adaptation in accordance with any of the described embodiments and displays the resulting image (e.g., using a monitor, screen, or other type of display). A television, set-top box, cell phone, tablet, or other electronic device that selects a channel (e.g., using a tuner) to receive a signal including an encoded image and performs motion vector prediction adaptation according to any of the described embodiments. A television, set-top box, cell phone, tablet, or other electronic device that wirelessly receives (e.g., using an antenna) a signal containing an encoded image and performs motion vector prediction adaptation according to any of the described embodiments. [Industrial Applicability]

[0207] The present invention can be used in image encoding and decoding.

Claims

1. A method for encoding, comprising: identifying a collocated region of a first reference image that is collocated with a current block for decoding the current image; obtaining a first motion vector from a location within or near the collocated region, the first motion vector pointing to a second reference image; rescaling the first motion vector to ensure that the first motion vector points to the second reference image; obtaining a shifted position in the second reference image by applying a displacement to a position in the current block based on the first motion vector; determining whether a second motion vector is available at the shift position, and if so, obtaining a third motion vector based on the second motion vector, the third motion vector being intended to be inserted into a list of motion vector predictor candidates used to reconstruct the motion vector of the current block; A method for providing the above.

2. The method of claim 1 , wherein when the shift position is outside a constraint area, the shift position is clipped to fit within the shift area.

3. The method of claim 1 , wherein the third motion vector is based on the first motion vector when the shift position is outside a constrained area.

4. When a first motion vector pointing to the second reference image cannot be obtained from a location within the collocated region or within a vicinity of the collocated region, the method further comprises: determining whether a spatial motion vector is available in the list of motion vector predictor candidates used to decode the motion vector of the current block; and if a spatial motion vector is available, modifying the first motion vector to a value that is dependent on the available spatial motion vector. The method of claim 1 , comprising:

5. When a first motion vector pointing to the second reference image cannot be obtained from a location within the collocated region or within a vicinity of the collocated region, the method further comprises: modifying the first motion vector into a spatial motion vector. The method of claim 1 , comprising:

6. The first motion vector is obtained from a position within a neighborhood of the collocated region, the method comprising: obtaining a fourth motion vector from a position within the collocated region, the fourth motion vector pointing to the second reference image; - obtaining a position in the second reference image, referred to as a second shift position, by applying a displacement to a position in the current block based on the fourth motion vector; determining whether a fifth motion vector is available at the second shift position, and if so, obtaining a sixth motion vector based on the fifth motion vector, the sixth motion vector being intended to be inserted into a list of motion vector predictor candidates used to reconstruct a motion vector of the current block; selecting a first available motion vector in an ordered set of motion vectors to be inserted into the list, the ordered set including at least the third motion vector, the sixth motion vector, the first motion vector, and the fourth motion vector, when these motion vectors are available; The method of claim 1 further comprising:

7. The method of claim 6 , wherein the ordered set further comprises a motion vector, when available, derived from the position within the collocated region.

8. The current block is divided into sub-blocks; the step of obtaining a shift position includes obtaining a shift position for each sub-block by applying a displacement to a position within the sub-block based on the first motion vector; For each sub-block, if a second motion vector is available at the shift position corresponding to the sub-block, a third motion vector is obtained. The method of claim 1.

9. 1. A device for encoding, comprising: Identifying a collocated region of a first reference image that is collocated with the current block for decoding the current image; obtaining a first motion vector from a location within or near the collocated region, the first motion vector pointing to a second reference image; rescaling the first motion vector to ensure that the first motion vector points to the second reference image; obtaining a shifted position in the second reference image by applying a displacement to a position in the current block based on the first motion vector; determining whether a second motion vector is available at the shift position, and if so, obtaining a third motion vector based on the second motion vector, the third motion vector being intended to be inserted into a list of motion vector predictor candidates used to reconstruct a motion vector of the current block; An electronic circuit configured as follows: A device with.

10. The device of claim 9 , wherein when the shift position is outside a constraint area, the shift position is clipped to fit within the shift area.

11. The device of claim 9 , wherein the third motion vector is based on the first motion vector when the shift position is outside a constrained area.

12. When a first motion vector pointing to the second reference image cannot be obtained from a location within or near the collocated region, the device: an electronic circuit adapted to determine whether a spatial motion vector is available in the list of motion vector predictor candidates used for decoding the motion vector of the current block, and, if a spatial motion vector is available, to modify the first motion vector to a value that depends on the available spatial motion vector. The device of claim 9 , comprising:

13. When a first motion vector pointing to the second reference image cannot be obtained from a location within or near the collocated region, the device: an electronic circuit configured to modify the first motion vector into a spatial motion vector; The device of claim 9 , comprising:

14. The first motion vector is obtained from a position within a vicinity of the collocated region, and the device: obtaining a fourth motion vector from a position within the collocated region, the fourth motion vector pointing to the second reference image; obtaining a second shift position in the second reference image by applying a displacement to a position in the current block based on the fourth motion vector; determining whether a fifth motion vector is available at the second shift position, and if so, obtaining a sixth motion vector based on the fifth motion vector, the sixth motion vector being intended to be inserted into a list of motion vector predictor candidates used to reconstruct a motion vector of the current block; Select a first available motion vector in an ordered set of motion vectors to be inserted into the list, the ordered set including at least the third motion vector, the sixth motion vector, the first motion vector, and the fourth motion vector when these motion vectors are available. An electronic circuit constructed as follows: The device of claim 9 , comprising:

15. The current block is divided into sub-blocks; obtaining a shift position includes obtaining a shift position for each sub-block by applying a displacement to a position within the sub-block based on the first motion vector; The device of claim 9 , wherein for each sub-block, a third motion vector is obtained if a second motion vector is available at the shift position corresponding to the sub-block.

Images