Encoding / decoding video picture data

Abstract

The present application relates to encoding / decoding a bitstream of encoded video picture data for a video picture, the method comprising temporally predicting a video picture block by obtaining a temporal prediction block based on at least one reference block of at least one reference picture pointed by at least one motion vector associated with the video picture block, wherein if the temporal prediction of the video picture block is based on a single reference block of a reference, the method further comprises: padding (402) a padding region (SZ) of the reference block based on a motion vector derived from a nearest sub-block of the video picture block within the video picture; and determining a size (M) of the fill area (SZ) on the basis of the motion vector.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based upon and claims the benefit of priority of European patent application No. “22305900.7” filed on June 21, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application generally relates to video picture encoding and decoding. In particular, but not limited thereto, the technical field of the present application relates to temporal prediction of motion compensation where a reference block overlaps a boundary of a reference picture. Background Art

[0004] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one exemplary embodiment of the present application described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of various aspects of the present application. Therefore, it should be understood that these statements should be read in this light, and not as admissions of prior art.

[0005] In state-of-the-art video compression systems, such as HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en) or VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), low-level and high-level picture partitioning are provided to divide the video picture into picture blocks, so-called Codec Tree Units (CTUs), whose size may typically be between 16x16 and 64x64 pixels for HEVC and between 32x32, 64x64 or 128x128 pixels for VVC.

[0006] The CTU partitioning of a video picture forms a grid composed of CTUs of a fixed size, ie, a CTU grid, whose upper and left boundaries spatially coincide with the upper and left boundaries of the video picture. The CTU grid represents the spatial partitioning of the video picture.

[0007] In VVC and HEVC, the CTU sizes (CTU width and CTU height) of all CTUs of the CTU grid are equal to the same default CTU size (default CTU width CTU DW and default CTU height CTU DH). For example, the default CTU size (default CTU height, default CTU width) may be equal to 128 (CTU DW = CTU DH = 128). The default CTU size (height, width) is encoded into the bitstream, for example at the sequence level in a sequence parameter set (SPS).

[0008] The spatial position of a CTU in the CTU grid is determined by the CTU address ctuAddr, which defines the spatial position of the upper left corner of the CTU relative to the origin. Figure 1 As shown in , a CTU address may define a spatial position starting from the upper left corner of a higher level spatial structure S that contains the CTU.

[0009] Each CTU is associated with a codec tree to determine the tree partitioning of the CTU.

[0010] like Figure 1 As shown in , in HEVC, the codec tree is a quadtree partition of CTU, where each leaf is called a codec unit (CU). The spatial position of the CU in the video picture is defined by the CU index cuIdx, which indicates the spatial position starting from the upper left corner of the CTU. The CU is spatially partitioned into one or more prediction units (PUs). The spatial position of the PU in the video picture VP is defined by the PU index puIdx, which defines the spatial position starting from the upper left corner of the CTU, and the spatial position of the elements of the partitioned PU is defined by the PU partition index puPartIdx, which defines the spatial position starting from the upper left corner of the PU. Each PU is assigned some intra-frame or inter-frame prediction data.

[0011] The codec mode intra or inter is assigned at the CU level. This means that although the prediction parameters vary from PU to PU, the same intra / inter codec mode is assigned to each PU of a CU.

[0012] According to a quadtree called a transform tree, a CU can also be spatially partitioned into one or more transform units (TUs). A transform unit is a leaf of a transform tree. The spatial position of a TU in a video picture is defined by a TU index tuIdx, which defines the spatial position starting from the upper left corner of the CU. Each TU is assigned some transform parameters. The transform type is assigned at the TU level, and 2D individual transforms are performed at the TU level during encoding or decoding of a picture block.

[0013] Figure 2The existing PU partition types in HEVC are illustrated in Figure 1. They include square partitions (2Nx2N and NxN), which are the only partitions used in both intra-frame and inter-frame prediction CUs, symmetric non-square partitions (2NxN, Nx2N, only used in inter-frame prediction CUs) and asymmetric partitions (only used in inter-frame prediction CUs). For example, PU type 2NxnU represents an asymmetric horizontal partition of a PU, where the smaller partition is located at the top of the PU. According to another example, PU type 2NxnL represents an asymmetric horizontal partition of a PU, where the smaller partition is located at the top of the PU.

[0014] like Figure 3 As shown in , in VVC, the codec tree starts from the root node (i.e., CTU). Next, the quadtree (or quadtree) splits the root node into 4 nodes, corresponding to 4 equally sized sub-blocks (solid lines). Next, the quadtree (or quadtree) leaves can be further divided by the so-called multi-type tree, which involves Figure 4 The invention provides a binary or trifurcated splitting method using one of the four splitting modes shown in . These splitting types are vertical and horizontal binary splitting modes (denoted as SBTV and SBTH) and vertical and horizontal trifurcated splitting modes SPTTV and STTH.

[0015] In the case of a joint codec tree shared by luma and chroma components, the leaves of the codec tree of a CTU are CUs.

[0016] In contrast to HEVC, in VVC, in most cases, CU, PU and TU have equal sizes, which means that the codec unit is generally not partitioned into PUs or TUs except in some specific codec modes.

[0017] Figure 5 and Figure 6 An overview of video encoding / decoding methods, such as used in current video standard compression systems such as HEVC or VVC, is provided.

[0018] Figure 5 A schematic block diagram showing the steps of a method 100 for encoding a video picture VP according to the prior art is shown.

[0019] In step 110, the video picture VP is partitioned into blocks of samples and partition information data is signaled into the bitstream. Each block comprises samples of one component of the video picture VP. Thus, the blocks comprise samples defining each component of the video picture VP.

[0020] For example, in HEVC, a picture is divided into codec tree units (CTUs). Each CTU can be further subdivided using a quadtree partition, where each leaf of the quadtree is represented as a codec unit (CU). The partition information data may then include data describing the CTU and the quadtree subdivision of each CTU.

[0021] Thus, each sample block (block for short) may be a CU (if the CU includes a single PU) or a PU of the CU.

[0022] Each block is encoded along the coding loop (also called "in-loop") using intra or inter prediction mode.

[0023] Intra prediction (step 120) uses intra prediction data. Intra prediction consists in predicting the current block with the help of intra predicted blocks based on coded, decoded and reconstructed samples, which are located around the current block, usually at the top and left of the current block. Intra prediction is performed in the spatial domain.

[0024] In inter prediction mode, motion estimation (step 130) and motion compensation (135) are performed. Motion estimation searches for a reference block that is a good predictor of the current block in one or more reference pictures used to predictively encode the current video picture. In unidirectional motion estimation / compensation, the candidate reference block belongs to a single reference picture of the reference picture list denoted L0 or L1, while in bidirectional motion estimation / compensation, the candidate reference block is obtained from the reference blocks of the reference picture list L0 and the reference blocks of the reference picture list L1.

[0025] For example, a good predictor for the current block is a candidate reference block that is similar to the current block. It may also correspond to a reference block that provides a good trade-off between similarity to the current block and the rate cost of indicating its motion information required for temporal prediction of the current block.

[0026] The output of the motion estimation step 130 is inter-frame prediction data, which includes motion information associated with the current block and other information used to obtain the same prediction block on the encoding / decoding side. Typically, the motion information includes one motion vector and reference picture index for unidirectional estimation / compensation and two motion vectors and two reference picture indexes for bidirectional estimation / compensation. Next, motion compensation (step 135) obtains the prediction block with the help of the (one or more) motion vectors and (one or more) reference picture indexes determined by the motion estimation step 130. Basically, the reference block belonging to the selected reference picture and pointed to by the motion vector can be used as the prediction block of the current block. In addition, since the motion vector is expressed as a fraction of an integer pixel position (this is called sub-pixel accuracy motion vector representation), motion compensation generally involves spatial interpolation of some reconstructed samples of the reference picture to calculate the prediction block.

[0027] The prediction information data is signaled into the bitstream. The prediction information may include the prediction mode (intra, inter or skip), intra / inter prediction data and any other information used to obtain the same prediction block at the decoding side.

[0028] The method 100 selects a prediction mode (intra-frame or inter-frame prediction mode) by optimizing the rate-distortion tradeoff, taking into account the encoding of the calculated prediction residual block (e.g., by subtracting the candidate prediction block from the current block) and the signaling of the prediction information data required to determine the candidate prediction block at the decoding side.

[0029] Typically, the best prediction mode is given as the prediction mode of the best codec mode p* for the current block given by:

[0030]

[0031] Where P is the set of all candidate codec modes for the current block, p represents the candidate codec mode in the set, and RD cost (p) is the rate-distortion cost of candidate codec mode p, usually expressed as:

[0032] RD cost(p) =D(p)+λ.R(p)

[0033] D(p) is the distortion between the current block and the reconstructed block obtained after encoding / decoding the current block with the candidate codec mode p, R(p) is the rate cost associated with encoding and decoding the current block with the codec mode p, and λ is a Lagrangian parameter representing the rate constraint for encoding and decoding the current block and is usually calculated based on the quantization parameter used to encode the current block.

[0034] The current block is usually coded from a prediction residual block PR. More precisely, the prediction residual block PR is calculated, for example, by subtracting the best prediction block from the current block. The prediction residual block PR is then transformed (step 140) using, for example, a DCT (discrete cosine transform) or DST (discrete sine transform) type transform or any other suitable transform, and the obtained transformed coefficient block is quantized (step 150).

[0035] In a variant, the method 100 may also skip the transform step 140 and directly apply quantization (step 150) to the prediction residual block PR according to the so-called transform-skip coding mode.

[0036] The block of quantized transform coefficients (or the block of quantized prediction residuals) is entropy encoded into a bitstream (step 160).

[0037] Next, as part of the encoding loop, the quantized transform coefficient block (or quantized residual block) is dequantized (step 170) and inverse transformed (180) (or not) to obtain a decoded prediction residual block. The decoded prediction residual block and the prediction block are then combined (usually summed) to provide a reconstructed block.

[0038] Other information data may also be entropy encoded in step 160 to encode the current block of the video picture VP.

[0039] A loop filter (step 190) may be applied to the reconstructed picture (including the reconstructed blocks) to reduce compression artifacts. The loop filter may be applied after all picture blocks have been reconstructed. For example, they include a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter.

[0040] The reconstructed block or the filtered reconstructed block forms a reference picture, which can be stored in a decoded picture buffer (DPB) so that it can be used as a reference picture for encoding the next current block of the video picture VP or as a reference picture for encoding the next video picture to be encoded.

[0041] Figure 6 A schematic block diagram showing the steps of a method 200 for decoding a video picture VP according to the prior art is shown.

[0042] In step 210 , partition information data, prediction information data and quantized transform coefficient blocks (or quantized residual blocks) are obtained by entropy decoding a bit stream of encoded video picture data. For example, this bit stream has been generated according to method 100 .

[0043] Other information data may also be entropy decoded to decode the current block of the video picture VP from the bitstream.

[0044] In step 220, the reconstructed picture is divided into current blocks based on the partition information. Each current block is entropy decoded from the bitstream along a decoding loop (also referred to as "in-loop"). Each decoded current block is a quantized transform coefficient block, or a quantized prediction residual block.

[0045] In step 230, the current block is dequantized and possibly inversely transformed (step 240) to obtain a decoded prediction residual block.

[0046] On the other hand, the prediction information data is used to predict the current block. The prediction block is obtained by its intra-frame prediction (step 250) or its motion compensated temporal prediction (step 260). The prediction process performed on the decoding side is exactly the same as the prediction process on the encoding side.

[0047] Next, the decoded prediction residual block and the prediction block are combined (usually summed), which provides the reconstructed block.

[0048] In step 270, the filter in the loop may be applied to the reconstructed picture (including the reconstructed block), and the reconstructed block or the filtered reconstructed block forms a reference picture, which may be stored in a decoded picture buffer (DPB), as discussed above ( Figure 5 ).

[0049] In VVC, motion information is stored in 4×4 blocks in each video picture. This means that once a reference picture is stored in the decoded picture buffer (DPB, Figure 5 or Figure 6 ), the motion vectors and reference picture indices used for temporal prediction of video picture blocks are stored on a 4×4 block basis. They can be used as temporal prediction of motion information for encoding / decoding subsequent inter-frame predicted video pictures.

[0050] In VVC, the temporal prediction of the current block can be based on a reference block of a reference picture, which can overlap with the boundary of the reference picture. Figure 7 An example of a bidirectional motion compensated prediction of a current block based on a first reference block of reference picture list L0 and a second reference block of reference picture list L1 is shown. The first (respectively second) reference block is pointed to by a first (respectively second) motion vector. Figure 7 In the example of , the first reference block is partially located outside the reference picture of reference picture list L0, while the second reference block is located inside the reference picture of reference picture list L1. More generally, the reference block of one of the two reference pictures or both reference blocks of the two reference pictures may be partially or completely located outside the reference picture boundary.

[0051] In VVC, to handle such situations, the reference picture is extended by a simple padding method, which includes vertically padding the video picture boundary samples bs, such as Figure 8 This consists in simply repeating the boundary sample values ​​bs along the rows or columns of samples perpendicular to the reference picture boundary (beyond this boundary). The reference picture is extended by an extension of M samples along each of its boundaries.

[0052] In the prior art, the filling method based on inter-frame or intra-frame prediction improves Figure 8 Basic padding.

[0053] Section 2.1.7.5 of the JVET contribution JVET-J0014 describes a filling method based on intra prediction (“Description of SDR, HDR and 360° video coding technology proposal by Fraunhofer HHI”, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 10th Meeting: San Diego, USA, April 10-20, 2018, document JVET-J0014). Basically, multi-directional boundary filling (MDBP) uses angular intra prediction to extend the reference picture border whenever the referenced sample block is partially or completely outside a block of the reference picture. This is done by using a template block (such as a template block) that is located inside the reconstructed reference picture. Fig. 9 ), the best intra mode is estimated in both the encoder and the decoder. The template block is filled by using all available intra coding modes and deriving the sample values ​​of the template block by intra prediction from the outermost pixel lines. The best intra coding mode results in the minimum SAD with respect to the original reconstructed sample values.

[0054] The first inter-prediction based filling method is described in Section 3.1 of the JVET contribution JVET-J0021 (“Description of SDR, HDR and 360° video coding technology proposal by Qualcommand Technicolor-low and high complexity versions”, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IECJTC 1 / SC 29 / WG 11, 10th Meeting: San Diego, USA, April 10-20, 2018, document https: / / jvet-experts.org / doc_end_user / documents / 10_San%20Diego / wg11 / JVET-J0021-v5.zip). Basically, as Fig.10 As shown in , when the decoder performs motion compensation, if the motion vector MV1 associated with the block CB of the current video picture points to a block RB that is at least partially outside the border of the reference picture 1, then the region Z of the reference block RB is not available. Fig.11As shown in , the area Z is split into multiple padding areas. Each padding area is split into a first padding sub-area SZ of size 4×M or M×4 and a second padding sub-area PZ of size 4×(PZ), where P is the size of the complete padding block. For each padding sub-area SZ along the boundary of the reference picture, the motion vector MV2 is obtained from the nearest 4×4 sub-block NB in ​​the reference picture 1 and the 4×4 sub-block of the reference picture 2. If the nearest 4×4 sub-block NB is intra-coded (intra-predicted), then a zero motion vector MV2 is used. If the nearest 4×4 sub-block NB is coded (predicted) with bidirectional inter-frame prediction, only motion vectors pointing to samples away from the boundary of its reference picture are used in the motion compensation for padding. After the motion vector derivation, motion compensation is then performed using the selected motion vector MV2 and reference picture 2, taking into account the average sample value offset between the nearest 4×4 sub-block and its corresponding block in the reference picture 2, to obtain the samples in the padding sub-area SZ. Note that the size M of the filling sub-area SZ can be derived from the value of the motion vector MV2 and is related to the distance between the picture boundary and the position pointed to by the motion vector MV2, see Fig.11 Furthermore, the size M may be smaller than P. In this case, the padding area is further completed by the basic vertical padding of the VVC. This means that the first padding sub-area SZ is extended by the basic vertical padding mentioned above with reference to the VVC.

[0055] A second inter-frame prediction-based padding method is described in Section 2.1.7.5 of the JVET contribution JVET-J0025 (“Description of SDR, HDR and 360° video coding technology proposal by Huawei, GoPro, HiSilicon, and Samsung-general application scenario”, Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 10th Meeting: San Diego, USA, April 10-20, 2018, document https: / / jvet-experts.org / doc_end_user / documents / 10_San%20Diego / wg11 / JVET-J0025-v4.zip). Basically, if the boundary samples of the video frame are derived by motion compensation, then the padded samples are also derived by using the motion compensation, such as Fig.12As shown in . For neighbor samples using a reference block, motion compensation is performed on the block referenced by the padded sample. If the block is smaller than the padded block or the boundary block is not obtained by motion compensation, the left padded sample is obtained using the nearest sample. The upper left corner, upper right corner, lower left corner, and lower right corner of the padded block are obtained by using the nearest sample at each position. This method is performed after performing in-loop filtering.

[0056] A third inter prediction method for handling out-of-bounds reference blocks is described in section 2.1.7.5 of the JVET contribution JVET-Y0125 (“AHG12: Enhanced bi-directional motion compensation”, Yi-Wen Chen, Che-WeiKuo, Ning Yan, Wei Chen, Xiaoyu Xiu, Xianglin Wang, ITU-T SG 16WP 3 and ISO / IEC JTC1 / SC 29 Joint Video Experts Group (JVET) 25th Meeting, by teleconference, January 12-21, 2022, document JVET-Y0125). Basically, in order to avoid that the prediction (motion compensated prediction) block b at least partially exceeds the reference picture boundary (which is not very efficient), when more than one motion compensated prediction blocks are combined, the OOB (out-of-bounds) prediction samples of the prediction block b are discarded and only non-OOB (in-bounds) prediction samples are used to generate the final prediction block. More precisely, let (Pos_x i,j ,Pos_y i,j ) is the Cartesian coordinate of the sampling position (i, j) in the current block. Let Pos is the Cartesian coordinate of the motion vector (MV) associated with the sampling position (i, j) in the current block (Lx indicates whether the reference picture belongs to the reference picture list L0 (x=0) or the reference picture list L1 (x=1). Let Pos LeftBdry ,Pos RightBdry ,Pos TopBdry ,Pos BottomBdry These are the Cartesian coordinates of the four borders of the reference screen.

[0057] When at least one of the following conditions holds, the prediction block P Lx The predicted sampling of the sampling position (i, j) Considered OOB

[0058]

[0059] or

[0060]

[0061] When none of the above conditions are met, the prediction sampling is considered non-OOB. Here, half_sample represents half the distance between two adjacent luma samples, according to the codec's MV internal representation accuracy (1 / 16 pixel for VVC). Otherwise, the prediction sample will be considered OOB. If the prediction block P Lx If at least one sample of is OOB, then the prediction block will be considered OOB; and when all its samples are non-OOB, then the prediction block will be considered non-OOB. Here, according to the MV internal representation accuracy of the codec (1 / 16 pixel for VVC), half_sample represents half of the distance between 2 samples.

[0062] Next, if the prediction sample is OOB and predicts sampling is non-OOB, then the final prediction sampling Equal to the predicted sample

[0063] Otherwise, if the prediction sample Is non-OOB and predictive sampling is OOB, then the final prediction sampling Equal to the predicted sample

[0064] Otherwise, the final prediction sample Sampling by two predictions and This weighted average is usually the mean, so weights 1 / 2 and 1 / 2 are used).

[0065] The limitation of the third inter-prediction based filling method is that it cannot handle the case where the MV points to a unidirectional prediction block of an OOB reference block. Moreover, it cannot improve the bidirectional motion compensation method of VVC when all prediction blocks associated with reference pictures belonging to both reference picture lists L0 and L1 are OOB.

[0066] There is still room for improvement in the out-of-boundary prediction block management method proposed in JVET-Y0125.

[0067] The problem solved by the present invention is to further improve the compression of existing video coding standards (such as HEVC or VVC).

[0068] In particular, the problem addressed is to improve the efficiency of motion compensated temporal prediction in case a certain motion vector points to a reference block outside the borders of a reference picture.

[0069] At least one exemplary embodiment of the present application is designed in consideration of the above circumstances. Summary of the invention

[0070] The following section briefly summarizes at least one exemplary embodiment to provide a basic understanding of some aspects of the present application. This overview is not an exhaustive overview of the exemplary embodiments. Its purpose is not to identify the key or core elements of the exemplary embodiments. The following overview only presents some aspects of at least one exemplary embodiment in a simplified form as a preface to the more detailed description provided elsewhere in this document.

[0071] According to a first aspect of the present application, there is provided a method for encoding a video picture into a bitstream of encoded video picture data, the method comprising temporally predicting a video picture block by obtaining a temporal prediction block based on at least one reference block of at least one reference picture pointed to by at least one motion vector associated with the video picture block, wherein if the temporal prediction of the video picture block is based on a single reference block of the reference picture, the method further comprises:

[0072] - filling a filling area of ​​a reference block based on a motion vector derived from a nearest sub-block of a video picture block within the video picture; and

[0073] - Determining the size of the padding area based on the motion vector.

[0074] According to a second aspect of the present application, there is provided a method for decoding a video picture from a bit stream of coded video picture data, the method comprising temporally predicting a video picture block by obtaining a temporal prediction block based on at least one reference block of at least one reference picture pointed to by at least one motion vector associated with the video picture block, wherein if the temporal prediction of the video picture block is based on a single reference block of the reference picture, the method further comprises:

[0075] - filling a filling area of ​​a reference block based on a motion vector derived from a nearest sub-block of a video picture block within the video picture; and

[0076] - Determining the size of the padding area based on the motion vector.

[0077] In one exemplary embodiment, the fill area is a rectangular area.

[0078] In an exemplary embodiment, if the derived motion vector points to a position inside the reference picture, the size of the padding area is set to at least equal to 4.

[0079] In an exemplary embodiment, if the reference picture pointed to by the motion vector is an inter-prediction picture, the size of the padding area is set to at least equal to 4.

[0080] In an exemplary embodiment, if the temporal or depth level of the video picture is at least equal to 1, the size of the padding area is set to at least equal to 4.

[0081] In an exemplary embodiment, if the temporal or depth layer of the video picture is higher than or equal to a predetermined value, the size of the padding area is set to at least equal to 8.

[0082] In an exemplary embodiment, if the temporal or depth layer of the video picture is higher than or equal to a predetermined value, if the reference picture pointed to by the motion vector is an inter-frame prediction picture, and if the temporal or depth layer of the video picture is higher than or equal to a predetermined value, the size of the padding area is further increased by 4.

[0083] In an exemplary embodiment, if the temporal prediction of the video picture block is based on a single reference block of the reference picture and if the single reference block is located at a corner of the reference picture, the method further comprises:

[0084] - obtaining a first motion vector and a second motion vector associated with a sub-block of a reference block located at a corner of a video picture;

[0085] - calculating a first size of the filling area from the coordinates of the first motion vector and calculating a second size of the filling area from the coordinates of the second motion vector;

[0086] - selecting the first size or the second size as the size of the fill area based on the sign of the first size and the sign of the second size; and

[0087] - Filling the padding area based on the first motion vector or the second motion vector according to the selected size.

[0088] In an exemplary embodiment, the selected dimensions result in a maximum surface of the filled area.

[0089] In one exemplary embodiment, if the sign of the first dimension is strictly positive and at least one of the second dimensions is not strictly positive, the first dimension is selected.

[0090] In one exemplary embodiment, the second dimension is selected if the sign of the second dimension is strictly positive and at least one of the first dimensions is not strictly positive.

[0091] According to a third aspect of the present application, there is provided a bit stream of encoded video picture data generated by one of the methods according to the first aspect of the present application.

[0092] According to a fourth aspect of the present application, an apparatus is provided, comprising means for executing one of the methods according to the first and / or second aspect of the present application.

[0093] According to a fifth aspect of the present application, a computer program product comprising instructions is provided, which, when executed by one or more processors, causes the one or more processors to execute the method according to the first and / or second aspect of the present application.

[0094] According to a sixth aspect of the present application, there is provided a non-volatile storage medium carrying instructions of a program code for executing a method according to the first and / or second aspect of the present application.

[0095] The specific nature of at least one exemplary embodiment and other objects, advantages, features and uses of the at least one exemplary embodiment will become apparent from the following description of the examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Reference will now be made, by way of example, to the accompanying drawings which show exemplary embodiments of the present application, in which:

[0097] Figure 1 An example of a codec tree unit according to HEVC is shown;

[0098] Figure 2 An example of partitioning a codec unit into prediction units according to HEVC is shown;

[0099] Figure 3 An example of CTU partitioning according to VVC is shown;

[0100] Figure 4 An example of splitting modes supported in multi-type tree partitioning according to VVC is shown;

[0101] Figure 5 A schematic block diagram showing the steps of a method 100 for encoding a video picture VP according to the prior art;

[0102] Figure 6 A schematic block diagram showing the steps of a method 200 for decoding a video picture VP according to the prior art;

[0103] Figure 7 An example of prediction of bidirectional motion compensation according to the prior art is shown;

[0104] Figure 8 A filling method according to the prior art is shown;

[0105] Fig. 9 A filling method based on intra-frame prediction according to the prior art is shown;

[0106] Fig.10 and Fig.11 An example of a first inter-frame prediction-based filling method according to the prior art is shown;

[0107] Fig.12 An example of a second inter-frame prediction-based filling method according to the prior art is shown;

[0108] Fig.13 Schematically showing a block diagram of a method 300 for determining a size M of a padding area of ​​a reference block according to an exemplary embodiment of the present invention;

[0109] Fig.14 Schematically shows a block diagram of a motion compensation prediction method 400 according to at least one exemplary embodiment;

[0110] Fig.15 A block diagram schematically shows a method 500 for filling a reference block of a reference picture that is not located at a corner of a video picture based on inter-frame prediction according to an exemplary embodiment;

[0111] Fig.16 and Fig.17 An example of a method 600 for filling a reference block located at the upper left corner of a reference picture based on inter-frame prediction according to an exemplary embodiment is shown;

[0112] Fig.18 A block diagram schematically illustrates a method for filling a reference block located at the upper left corner of a reference picture based on inter-frame prediction according to an exemplary embodiment of the present invention; and

[0113] Fig.19 A schematic block diagram of an example of a system in which various aspects and exemplary embodiments are implemented is illustrated.

[0114] Similar or identical elements are referenced with the same reference numerals. DETAILED DESCRIPTION

[0115] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, wherein an example of at least one of the exemplary embodiments is depicted. However, the exemplary embodiments may be implemented in a variety of alternative forms and should not be construed as being limited to the examples set forth herein. Thus, it should be understood that the present invention is not intended to limit the exemplary embodiments to the specific forms disclosed. On the contrary, the present application is intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the present application.

[0116] At least one of the aspects generally relates to video picture encoding and decoding, another aspect generally relates to transmitting provided or encoded bitstreams, and another aspect relates to receiving / accessing decoded bitstreams.

[0117] At least one of the exemplary embodiments is described with respect to encoding / decoding one video picture, but extends to encoding / decoding multiple video pictures (sequences of pictures) in that each video picture is encoded / decoded in turn as described below.

[0118] Moreover, for example, at least one exemplary embodiment is not limited to MPEG standards, such as AVC (ISO / IEC 14496-10 Advanced Video Coding for generic audio-visual services, ITU-T Recommendation H.264, https: / / www.itu.int / rec / T-REC-H.264-202108-P / en), EVC (ISO / IEC 23094-1 Essential video coding), HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en), VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-I / en), but may be applied to other standards and recommendations, such as AV1 (AOMedia Video 1, http: / / aomedia.org / av1 / specification / ). At least one exemplary embodiment may be applicable to existing or future developments and extensions of any such standards and recommendations. Unless otherwise specified or technically excluded, the various aspects described in this application may be used alone or in combination.

[0119] A pixel corresponds to the smallest display unit on the screen, which can be composed of one or more light sources (1 for a monochrome screen and 3 or more for a color screen).

[0120] A video picture, also called a frame or picture frame, comprises at least one component (also called a picture component or channel) determined by a specific picture / video format, which specifies all information related to pixel values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video picture data related to the video picture.

[0121] A video frame comprises at least one component which is usually represented in the form of an array of samples.

[0122] A monochrome video picture includes a single component, while a color video picture may include three components.

[0123] For example, when the picture / video format is the well-known (Y, Cb, Cr) format, a color video picture may include one brightness (or luminance) component and two chrominance components, and when the picture / video format is the well-known (R, G, B) format, a color video picture may include three color components (one for red, one for green, and one for blue).

[0124] Each component of a video picture may comprise a number of samples relative to the number of pixels of the screen on which the video picture is to be displayed. In a variant, the number of samples comprised in a component may be a multiple (or fraction) of the number of samples comprised in another component of the same video picture.

[0125] For example, in case the video format includes one luma component and two chroma components (such as (Y, Cb, Cr) format), the chroma components may contain half the number of samples in width and / or height relative to the luma components, depending on the color format considered.

[0126] A sample is the smallest visual information unit of a component that makes up a video picture. A sample value can be, for example, a brightness or chrominance value, or a color value in (R, G, B) format.

[0127] A pixel value is the value of a pixel of the screen. For a monochrome video picture, a pixel value may be represented by one sample, while for a color video picture, a pixel value may be represented by multiple co-located samples. A co-located sample associated with a pixel refers to a sample corresponding to the position of the pixel in the screen.

[0128] A video picture is usually viewed as a set of pixel values, with each pixel represented by at least one sample.

[0129] A video picture block is a group of samples of a component of a video picture. When the picture / video format is the well-known (Y, Cb, Cr) format, a block of at least one luma sample or a block of at least one chroma sample may be considered, or when the picture / video format is the well-known (R, G, B) format, a block of at least one color sample may be considered.

[0130] At least one exemplary embodiment is not limited to a specific picture / video format.

[0131] In general, the present application relates to encoding / decoding a video picture into / from a bitstream of encoded video picture data, comprising temporally predicting a block of the video picture by obtaining a temporally predicted block based on at least one reference block of at least one reference picture pointed to by at least one motion vector associated with the block of the video picture, wherein if the temporal prediction of the block of the video picture is based on a single reference block of the reference picture, the method further comprises filling (302) a filling area of ​​the reference block based on inter-frame prediction based on a motion vector derived from a nearest subblock of the video picture block within the video picture and determining a size of the filling area based on the motion vector.

[0132] The present invention improves upon the third prior art method of inter-frame prediction discussed above, in that it provides improved filling texture blocks for reference block portions that are outside reference picture boundaries in the case of unidirectional temporal prediction of a current video picture block.

[0133] Padding may be applied to the encoding method 100 or the decoding method 200 to extend reference pictures stored in the DPB.

[0134] exist Fig.11 In FIG. 1 , it can be seen that the padding sub-region SZ associated with the 4×4 sub-region NB is equal to 4×M (in the case of bottom or top border padding) or M×4 (in the case of left or right border padding). The size M of the padding sub-region SZ varies according to the motion vector MV2 stored in the memory associated with the sub-block NB. The motion vector MV2 is represented in the (x, y) coordinate space.

[0135] In the prior art, when filling along the top border (y coordinate equals 0), the size M is obtained according to equation (1):

[0136] M=(((MV2 y >>MVFBITS)+3)>>2)<<2

[0137] or equivalently

[0138]

[0139] MV2 y is the y coordinate of the motion vector MV2, and MFBITS denotes the precision used to represent the motion vector in the considered video codec. More precisely, MVFBITS corresponds to the number of bits used to represent the fractional (sub-pixel) part of the motion vector x and y coordinates.

[0140] In the case of padding along the bottom border, the size M is obtained according to equation (2):

[0141]

[0142] In the case of padding along the left border (x coordinate equal to 0), the size M is obtained according to equation (3):

[0143]

[0144] In the case of padding along the left border, the size M is obtained according to equation (4):

[0145]

[0146] Therefore, in the prior art, equations (1)-(4) show that the size M is a multiple of 4 because motion compensation is performed on blocks whose size is at least 4 or a multiple of 4. However, the coordinates of the motion vector MV2 that can be used to fill the reference block may be lower than 4. In this case, in the prior art, the size M is set to 0 and no padding based on motion compensation occurs.

[0147] According to the invention, the size M of the filling sub-area SZ is determined according to the motion vector MV2 and can be non-zero even for some low-amplitude motion vectors MV2. This improves the filling of the reference block when considering small-amplitude motion vectors at the border of the video picture to be extended.

[0148] Fig.13 Schematically showing a block diagram of a method 300 for determining a size M of a padding area of ​​a reference block according to an exemplary embodiment of the present invention;

[0149] In step 301, the boundary type of the video picture to be extended is determined.

[0150] If the border is the top border of the video picture, then in step 302, the size M is obtained from equation (1).

[0151] If the border is the bottom border of the video picture, then in step 303, the size M is obtained from equation (2).

[0152] If the border is the left border of the video picture, then in step 304, the size M is obtained from equation (3).

[0153] If the border is the right border of the video picture, then in step 305, the size M is obtained from equation (4).

[0154] In a variant, if the motion vector MV2 is suitable for the filling of the border under consideration, the size M of the filling sub-zone SZ is set to at least equal to 4 by means of the operation M=max(4,M) (step 310 ).

[0155] More precisely, in the case of the top border, if the y component MV2 of the motion vector MV2 yis strictly above 0 (step 306), the size M is set to be at least equal to 4 (step 310).

[0156] In the case of the bottom border, if the y component of the motion vector MV2 y is strictly negative (step 307), then the size M is set to be at least equal to (step 310).

[0157] In the case of the left boundary, if the x component of the motion vector MV2 x is strictly positive (step 308), then M is set to at least equal to 4 (step 310).

[0158] In the case of the right boundary, if the x component of the motion vector MV2 x is strictly negative (step 309), the size M is set to at least equal to 4 (step 310).

[0159] In a variant of step 310, if the reference picture pointed to by the motion vector MV2 is an inter-frame prediction picture, the size M is set to at least equal to 4. The advantage of this variant is that when the size M of the block is set to be higher than the amplitude of the motion vector MV2 in the vertical or horizontal direction considered, the reference block NBR with a size of M and pointed to by the motion vector MV2 is likely to overlap with the boundary of the reference picture pointed to by the motion vector MV2. In addition, if the reference picture is of inter-frame type, it is likely that it has been extended by the motion compensation picture filling process, so its filling area has better quality than if it is extended only by the vertical repetition filling of VVC. Therefore, if the picture is an inter-frame picture, it may be advantageous to use MV2 that results in a reference block NBR overlapping with the boundary of the reference picture to which it belongs.

[0160] In a variant of step 310 , if the temporal or depth level of the video picture is at least equal to 1, the size M is set to at least equal to 4.

[0161] The advantage of this variant is that if the temporal level of a picture is at least 1, it is even more likely that padding areas around its borders will be produced by motion compensated padding. Therefore, they are even more likely to be of good quality. Thus, it increases the likelihood that NBR reference blocks overlapping a picture border will be used to increase the quality of a motion compensated picture padding in the current picture.

[0162] In a variant of step 310 , the size M is set to be at least equal to 8 if the temporal or depth level of the video picture is higher than or equal to a predetermined value (eg 2).

[0163] The rationale behind this variant is that as the temporal layer of the current picture gets higher, it becomes more likely that the size of the picture extension area produced by motion compensated filling is higher. Therefore, for the motion compensated filling of the current picture, a larger size M is allowed to be used, resulting in an overall filled area of ​​improved quality.

[0164] In a variation of step 310, if the temporal or depth layer of the video picture is higher than or equal to a predetermined value (e.g. 2), if the reference picture pointed to by the motion vector MV2 is an inter-frame prediction picture, and if the temporal or depth layer of the video picture is higher than or equal to a predetermined value, the size M is further increased by 4.

[0165] The rationale behind this variant is that as the temporal layer of the current picture gets higher, it becomes more likely that the size of the picture extension area produced by motion compensated filling is higher. Therefore, for motion compensated filling of the current picture, it is allowed to use a size M greater than 4, resulting in an overall filled area of ​​improved quality.

[0166] Fig.14 A block diagram schematically illustrates a method 400 for motion compensated prediction according to at least one exemplary embodiment.

[0167] For a current block of a video picture to be inter-predicted (PU of a current CU), in step 401 , the method 400 checks whether bidirectional motion compensation prediction is used.

[0168] If unidirectional motion compensated prediction is used for the current block, i.e., if the motion compensated prediction of the current block is based on a single reference block of a reference picture, then in step 4031, method 400 checks whether the reference block (pointed to by the motion vector associated with the current block) is OOB, i.e., whether at least one sample of the reference block is outside the single reference picture boundary.

[0169] If the single reference block is outside the border of the reference picture, i.e. the single reference block is located along the border of the reference picture, then in step 402 the method checks whether the single reference block is located in a corner of the reference picture. If this is the case, step 402 is followed by combining Figure 16-Figure 18 Otherwise, step 402 is followed by step 4033, in which the reference block is filled based on the motion vector MV2, which is derived from the nearest sub-block of the current block within the video picture, as described above with respect to Fig.10 and Fig.11 As discussed, wherein the size M is determined based on the motion vector MV2 discussed above.

[0170] For example, as defined in section 8.5.6.3 of the VVC specification, well-known unidirectional motion compensated prediction is performed based on reference pictures which may be padded.

[0171] Step 4031 is performed for the reference block as a non-OOB reference block and for the OOB reference block. For the non-OOB reference block, motion compensated prediction of the current block leads to the same result as in VVC, since all samples of the reference block are within a single reference picture boundary.

[0172] Step 4032 or method 600 is advantageous for OOB reference blocks (i.e., for prediction blocks having at least one OOB prediction sample) because the samples of those reference blocks used for motion compensated prediction are determined by Fig.10 and Fig.11 Those generated by the discussed inter-prediction based filling method, where the size M is determined according to the motion vector MV2 as described above. That is, these samples of the OOB reference block located outside the reference picture boundary are generated by motion compensated prediction using the stored motion vector associated with the 4×4 sub-block within a single reference picture that is closest to the considered sample of the reference block as discussed above. Therefore, these motion compensated prediction reference samples have better quality than the samples obtained by the VVC vertical filling method.

[0173] If, at step 4031 , the single reference block is non-OOB (ie, all samples of the single reference block are within the reference picture boundary), then at step 4032 , well-known unidirectional motion compensated prediction is performed based on the reference picture, e.g. as defined in VVC.

[0174] If the single reference block is OOB, then in step 4033, as described above with respect to Fig.10 and Fig.11 As discussed, a single reference block is padded based on a motion vector MV2 derived from a nearest sub-block of a current block within a video picture, wherein the size M is determined according to the motion vector MV2 as discussed above, and the well-known unidirectional motion compensated prediction is performed based on the padded reference picture, for example as defined in VVC.

[0175] According to a variant of step 402, only the OOB reference blocks are populated based on motion vectors derived from the nearest sub-blocks of the current block within the video picture.

[0176] This variant avoids padding of non-OOB reference blocks, thus limiting the computational complexity compared to padding all reference blocks (regardless of whether they are non-OOB or OOB).

[0177] If bidirectional motion compensated prediction is used for the current block (step 401), then in step 406, the first prediction block P of the current block is obtained based on the first reference block of the reference picture list (e.g. L0) using the well-known unidirectional motion compensated prediction, e.g. as defined in VVC. L0 .

[0178] In step 404, a second prediction block P of the current block is obtained based on a second reference block of a reference picture of another reference picture list (e.g., L1) using, for example, the well-known unidirectional motion compensated prediction as defined in VVC. L1 . Step 406 is followed by step 405 .

[0179] If the first reference block is non-OOB, then the well-known unidirectional motion compensation prediction is based on the first reference block. If the first reference block is OOB, then as described above with respect to Fig.10 and Fig.11 As discussed, a first reference block is filled based on a motion vector MV2 derived from a 4×4 nearest subblock of the current block within a video picture, wherein the size M is determined according to the motion vector MV2, and the well-known unidirectional motion compensated prediction is based on the first filled reference block.

[0180] If the second reference block is non-OOB, then the well-known unidirectional motion compensation prediction is based on the second reference block. If the second reference block is OOB, then as described above with respect to Fig.10 and Fig.11 As discussed, a second reference block is filled based on a motion vector MV2 derived from a nearest 4×4 sub-block of the current block within the video picture, wherein the size M is determined according to the motion vector MV2, and the well-known unidirectional motion compensated prediction is based on the second filled reference block.

[0181] Alternatively, at steps 406 and 404, the inter-prediction based filling method may be applied to the first reference block and the second reference block only when the first reference block and the second reference block associated with the reference picture lists L0 and L1, respectively, are both considered OOB. On the contrary, if at least one of the first reference block and the second reference block is considered non-OOB, the inter-prediction based filling method will not be applied to the first reference block and the second reference block.

[0182] This alternative has the advantage of reduced complexity compared to the other approach above, as it avoids performing inter-prediction based padding in many cases (ie, where at least one reference block used for bi-directional prediction is non-OOB).

[0183] In step 405, according to the first prediction block P L0 The sampling at a given position (i, j) of Is it OOB and according to the second prediction block P L1 The samples at the same position (i, j) of Whether it is OOB, the final motion compensated prediction block is obtained by averaging the first prediction block and the second prediction block on a sampling basis.

[0184] In an exemplary embodiment of step 405 , sub-steps 4051 - 4056 may be performed for each sampling position (i, j) of the current block.

[0185] In sub-step 4051, consider the first sample position (i, j) of the current block.

[0186] In sub-step 4052, method 400 checks the prediction sample at the current position (i, j) of the first prediction block. Whether it is OOB and the prediction sample at the current position (i, j) of the second prediction block Is it non-OOB? If the prediction sampling is OOB and predicts sampling is non-OOB, then sub-step 4052 is followed by sub-step 4053. If the predicted sample Is non-OOB or predictive sampling If it is OOB, sub-step 4052 is followed by sub-step 4054.

[0187] In step 4053, the prediction sample at the current position (i, j) of the final prediction block P is set equal to the first prediction block The predicted sample at the current position (i, j) of .

[0188] In sub-step 4054, method 400 checks the first prediction sample at the current position (i, j) of the first prediction block. Whether it is non-OOB and the second prediction sample at the current position (i, j) of the second prediction block Is it OOB? If the first sample is predicted is non-OOB and the second prediction sample is OOB, then sub-step 4054 is followed by sub-step 4055. If the predicted sample Is OOB or predictive sampling If it is non-OOB, sub-step 4054 is followed by sub-step 4056.

[0189] In step 4055, the prediction sample at the current position (i, j) of the final prediction block P is set equal to the prediction sample of the second prediction block The predicted sample at the current position (i, j) of .

[0190] In step 4056, the prediction sample at the current position (i, j) of the final prediction block P is set equal to the weighted average of the prediction samples at the current position (i, j) of the first prediction block and the second prediction block.

[0191] After steps 4052, 4053 and 4056, method 400 checks whether all positions of the current block have been considered. If not, the next position of the current block is considered and the method iterates at sub-step 4052. If all positions have been considered, step 405 is terminated.

[0192] According to an exemplary embodiment of step 405, if the unidirectional prediction sample position is non-OOB and the other position is OOB, the prediction sample of the final prediction block may be equal to the unidirectional prediction sample of one of the first prediction block or the second prediction block, and if both the first unidirectional prediction sample and the second unidirectional prediction sample are non-OOB, or if both are OOB, the prediction sample of the final prediction block is equal to the weighted average of the two unidirectional prediction samples. In the latter case (where both unidirectional prediction sample positions are OOB), the third prior art inter prediction method has the problem of using unidirectional prediction samples issued from the vertical filling method of the VVC specification, which may have low quality.

[0193] Method 400 improves upon the third prior art inter-frame prediction method discussed above for unidirectional and bidirectional motion compensated prediction sampling. For unidirectional motion compensated prediction sampling, the third prior art inter-frame prediction method does not provide any solution. For bidirectional motion compensated prediction sampling, method 400 benefits from the above prior art inter-frame prediction method. Fig.10 and Fig.11 The discussed use of an improved padding block in an out-of-boundary reference picture block, wherein the size M is determined based on a motion vector stored in a memory associated with a sub-block of the padding sub-region.

[0194] Thus, method 400 improves compression efficiency over conventional motion compensated prediction using padding of reference pictures as discussed with respect to any intra- or inter-prediction based padding method of the prior art, such as the first and second inter-prediction based padding as discussed above. It also improves compression efficiency over the bidirectional inter-prediction method of the third prior art inter-prediction method.

[0195] In an exemplary embodiment, only when the prediction sample or A prediction sample is considered OOB only when its position in the reference picture exceeds the temporal prediction block that extends the reference picture. Fig.11 When the position in the reference picture 1 is OOB, the size M is determined according to the motion vector MV2.

[0196] The exemplary embodiment is advantageous because bidirectional motion compensated prediction can benefit from a good quality padded reference picture block since it has been obtained by motion compensation instead of by basic vertical padding (e.g. Fig.11 The filling sub-area PZ) is filled. Therefore, the exemplary embodiment provides a synergy between the first prior art inter-frame prediction based filling method and the third prior art inter-frame prediction prior art method, because unidirectional and bidirectional motion compensated predictions generally benefit from better quality prediction samples than in the prior art. The first inter-frame prediction based filling method provides a good quality extension of the reference picture, the third prior art inter-frame prediction method provides good quality prediction samples for some bidirectional motion compensated predictions, and combining these two methods according to the exemplary embodiment also leads to more frequent use of the bidirectional motion compensated prediction of the third prior art inter-frame prediction method, and thus improves compression efficiency compared to the method in which the prediction sample is considered OOB when its position is outside the boundary of the reference picture.

[0197] In an exemplary embodiment, the predicted samples may be located outside the reference frame boundary. or It is considered non-OOB when it is derived through an inter-frame prediction process (e.g. Fig.11 ) and is considered OOB when it is not derived from an inter prediction process. Therefore, if the 4×4 block closest to the considered sample has no motion vector and is therefore intra-coded, the considered reference sample position is considered OOB.

[0198] Therefore, in the reference picture extension process, M×4 and 4×M padding sub-areas ( Fig.11 The SZ in the image may be marked as inter-coded. This marking information is then used to decide a prediction sample. or Is it OOB or non-OOB.

[0199] This last exemplary embodiment provides a more sophisticated and accurate way to ensure that the bidirectional motion compensated prediction benefits from a good quality extended reference picture portion, thereby further improving the compression efficiency of the current block.

[0200] In such Fig.15 In one exemplary embodiment shown, reference blocks that are not located at the corners of a video picture and are used to predict a current block of the video picture may be filled in from information associated with a 4×4 sub-block within an intra- or inter-coded video picture.

[0201] In an exemplary embodiment, the distance threshold is used to determine whether the prediction sample of the first or second prediction block is OOB. The distance threshold can be modified according to certain characteristics of the reference picture of the current video picture.

[0202] For example, if the reference picture belongs to a temporal layer at least equal to a given value (e.g., 2), then the distance threshold may be assigned a ratio of Fig.14 The distance threshold may be set to a higher value than half_sample used in method 400 (steps 4052 and 4054). For example, the distance threshold may be set to twice the distance between two adjacent brightness samples, denoted as double_sample.

[0203] The selection of the distance threshold may follow the following decision process: if the temporal layer associated with the reference picture is at least equal to 2 and at least two of the reference pictures considered are inter-coded pictures, then the distance threshold is set equal to double_sample. Otherwise, the distance threshold is set equal to half_sample.

[0204] In a variant, the distance threshold is signaled in some header of the bitstream, such as in a slice header, a picture header, a picture parameter set (PPS) or a sequence parameter set (SPS).

[0205] In an exemplary embodiment, the motion vector MV2 ( Fig.11 ) can be refined after being obtained from the nearest 4×4 sub-block in the reference picture. This refinement can be in accordance with the known Template Matching (TM) motion refinement used in ECM (“Algorithm description of Enhanced Compression Model 3 (ECM3)”, Muhammed Coban, Fabrice Le Léannec, Mohammed Golam Sarwer, Jacob Joint Video Experts Team (JVET) of ITU-T SG 16WP 3and ISO / IEC JTC 1 / SC 29 23rd Meeting, by teleconference, 7-16 July 2021, Document JVET-X2025). This may include searching for reference blocks around the block pointing to the considered motion vector MV2 by minimizing the template matching cost (usually distortion, optionally plus a motion vector cost metric). The distortion for TM cost calculation may be calculated between the template region adjacent to the current filling sub-region SZ and the corresponding template block in the reference picture of the current reference picture.

[0206] In an exemplary embodiment, the improved Fig.10 and Fig.11 A first filling method based on inter-frame prediction, where the size M is determined according to the motion vector MV2, can apply the following correction to inter-frame prediction.

[0207] exist Fig.11 In the prior art solution of , a correction is applied to the filling sub-region SZ. The correction is a superposition offset calculated as the average difference between the 4×4 sub-block NB and the 4×4 sub-block NBR pointed to by the motion vector MV2. This average difference is then applied to the filling sub-region SZ.

[0208] In this exemplary embodiment, this correction may be replaced by a linear model based inter prediction enhancement similar to the known ECM local illumination compensation.

[0209] This takes the form of a linear model that determines the linking sub-blocks NB and NBR:

[0210] (a * ,b * )=Argmin a,b {‖NB-(a.NBR+b)‖}

[0211] Then use the linear model parameters (a, b): SZ final =a.SZ+b enhances the prediction of the filled sub-region SZ.

[0212] An advantage of this exemplary embodiment is that the quality of filling the sub-region SZ is improved by enhanced temporal prediction.

[0213] In an exemplary embodiment, the decoded video pictures may be upscaled according to the present invention immediately after decoding the video pictures (at the encoder and decoder side).

[0214] This enables the video picture thus decoded to be prepared and then used as a reference picture for the next video picture to be encoded or decoded.

[0215] In a variation of the previous exemplary embodiment, a decoded video picture may be expanded according to the invention immediately after decoding said video picture and before encoding / decoding the next video picture.

[0216] In a variation of the previous exemplary embodiment, a decoded video picture is extended only when the video picture is used as a reference picture for future video picture encoding / decoding.

[0217] Then, during the construction or updating of the reference picture list during the encoding and decoding methods 100 and 200, the decision to extend the decoded video picture is made. On the decoder side, after having decoded the header part of the video picture, the decoder is able to know which video pictures are used as reference pictures for the current video picture. Therefore, for each reference picture used, if it has not been extended, it is extended and marked as filled.

[0218] In a variation of this last exemplary embodiment, the decision to scale a video picture may be made immediately after the video picture has been encoded (encoder side) or decoded (decoder side).

[0219] For example, if the current video picture has all its reference pictures in the past of the current video picture in display order, and if the temporal layer of the current video picture is different from the maximum temporal layer in the considered video sequence, or if the current video picture is an intra-coded picture or an IRAP (Intra Random Access Point) picture, the current video picture is extended. Otherwise, the current picture is not extended.

[0220] Fig.15 A block diagram of a method 500 for filling a reference block of a reference picture that is not located at a corner of a video picture based on inter-frame prediction according to an exemplary embodiment is schematically shown.

[0221] Basically, each reference block that is not located at the corner of the video picture is iteratively considered, and for each reference block, a 4×M or M×4 block b is derived around the reference picture to obtain the extended reference picture as follows: Figure 7 as shown (filled block).

[0222] In step 501, the first 4×M or M×4 block b is considered for the current reference block.

[0223] In step 502, the 4×4 sub-block nb closest to the current block b in the reference picture is determined, and its coding (prediction) mode is obtained.

[0224] In step 503, the codec mode of block nb is checked.

[0225] If the codec mode is the inter-frame codec mode, then in step 504, as described above with respect to Fig.10 and Fig.11 As discussed, the current reference block is filled based on the motion vector MV2 derived from the block nb, wherein the size M is determined according to the motion vector MV2.

[0226] If the coding mode is the intra coding mode, then in step 505, the current reference block is filled based on a filling method based on intra prediction.

[0227] In a variation of step 505, the intra prediction based filling method may be the first intra prediction based filling method as discussed above, wherein the best intra mode is replaced by the obtained intra coding mode.

[0228] In another variation of step 505, the intra-prediction based filling method may be the first intra-prediction based filling method as discussed above, wherein the best intra-mode is replaced by a prediction mode similar to DIMD (Decoder Intra Mode Derivation) of ECM (“Algorithm description of Enhanced Compression Model 3 (ECM 3)”, M. Coban, F. Le Léannec, M. Sarwer, J. Document JVET-X2025, Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, 23rd Meeting, bytleconference, 7-16July 2021, https: / / jvet-experts.org / doc_end_user / current_document.php?id=11231). Basically, DIMD consists of computing the histogram of the spatial gradients of the 2D reconstructed signal in block nb and deriving the intra prediction mode based on this gradient-based analysis. The basic principle is to extend the directions contained in the reconstructed intra signal to the OOB samples of the current reference block.

[0229] In step 506, method 500 checks whether all reference blocks along the border of the reference picture have been considered. If so, the method is terminated. Otherwise, a new reference block is considered and the method iterates from step 402.

[0230] The method 500 further improves the quality of the extended reference picture, thereby improving the compression efficiency.

[0231] Note that the main difference compared to the intra prediction based padding described in section 2.1.7.5 of the JVET Contribution JVET-J0014 is that here the switching process between intra prediction and inter prediction occurs during the extension step of the reference pictures.

[0232] In a variation of method 500, in step 503, a syntax element signaled in the bitstream indicates whether filling of the reference blocks of the reference picture (step 402) is based on a motion vector derived from a nearest sub-block of the current block within the video picture or based on an intra-prediction based filling method.

[0233] In a variant, the syntax elements may be signaled at sequence level, for example in an SPS (Sequence Parameter Set).

[0234] In a variant, the syntax elements may be signaled at picture level, for example in a picture parameter set (PPS).

[0235] In a variant, syntax elements may be signaled at slice level.

[0236] In a variation of the method 500 , the reference blocks are filled based on an intra-prediction based filling method only for certain types of video pictures, such as for certain screen and / or graphic video content.

[0237] Fig.16 and Fig.17 An example of a method 600 of filling a reference block located at the upper left corner of a reference picture based on inter prediction according to an exemplary embodiment is shown.

[0238] Similar methods can be easily derived to fill in reference blocks located in other corners (upper right, lower left and lower right) of the video picture.

[0239] Basically, if Fig.16 As shown, when the decoder performs motion compensation, if the motion MV1 associated with the block CB of the current video picture points to a block RB located at the upper left corner of the reference picture 1, and if the block RB is at least partially outside the border of the reference picture 1, the region Z of the reference block RB is not available. Fig.17 As shown, the region Z is split into a first padding sub-region SZ and a second padding sub-region PZ having a size M1×M2. Padding the reference block RB at the upper left corner of the reference picture 1 includes extending the reference picture 1 into the padding sub-region SZ. The samples of the padding sub-region SZ are obtained from the motion vector MV2 associated with the sub-block NB located at the upper left corner of the reference picture 1. Then, if the motion vector MV2 points to an inner part of the reference picture 2 of the reference video picture 1, the motion vector MV2 is used to predict the samples of the padding sub-region SZ. More precisely, for the case of the upper left corner, if the x-component and the y-component of the considered motion vector MV2 are positive, the motion vector MV2 points to a position inside the reference picture, as Fig.17 As shown, the samples of the padding sub-region SZ outside the boundary of the reference picture 1 (on its upper left side) can therefore be temporally predicted.

[0240] Fig.18 A block diagram schematically illustrates a method 600 for filling a reference block located at the upper left corner of a reference picture based on inter-frame prediction according to an exemplary embodiment of the present invention.

[0241] A similar method can be easily derived for other corners of the video picture (upper right, lower left and lower right). In step 601, the motion vector mv of the reference picture list L0 associated with the upper left subblock of the reference block RB is obtained. L0and the motion vector mv of the reference picture list L1 L1 .

[0242] In step 602, the motion vector mv in the (x, y) coordinate system is L0 Coordinates Calculate the first size of the filled sub-area SZ

[0243]

[0244] In step 603, the motion vector mv in the (x, y) coordinate system is L1 Coordinates Calculate the second size of the filled sub-area SZ

[0245]

[0246] In step 604, symbols of the first size and symbols of the second size are checked.

[0247] If the two first dimensions is strictly positive, this means that the size of the top left side of the reference picture is The rectangular block can be represented by the motion vector mv L0 Make time predictions.

[0248] If only the two first sizes are strictly positive, then in step 605 the reference picture list L0 is considered to be a so-called denoted useList.

[0249] If the two second dimensions is strictly positive, this means that the size of the top left side of the reference picture is The rectangular block can be represented by the motion vector mv L1 Make time predictions.

[0250] If only two second sizes are strictly positive, then in step 606 the reference picture list L1 is considered to be a so-called denoted useList.

[0251] If all sizes and If both are strictly positive, then in step 607, a single reference picture list is selected between reference picture lists L0 and L1, denoted as useList.

[0252] In an exemplary embodiment of step 607, the selected reference picture list denoted as useList is the one that results in the first surface With the second surface The reference picture list (L0 or L1) of the largest surface of the padding sub-area SZ between (useList is equal to L0 or L1).

[0253] In step 608, based on the motion vector mv useList Pair with size The filling sub-area SZ is filled (temporal prediction).

[0254] The position of this predicted padding sub-region SZ is outside the boundary of the reference picture 1, but adjacent to the upper left corner of the reference picture 1, such as Fig.17 shown.

[0255] In a variant, when the filling sub-region SZ is smaller than the second filling sub-region PZ, the region Z is further completed by a substantially vertical filling of the VVC. This means that the first filling sub-region SZ is extended through the substantially vertical filling mentioned above with reference to the VVC.

[0256] Fig.19 A schematic block diagram illustrating an example of a system 700 in which various aspects and exemplary embodiments are implemented is shown.

[0257] The system 700 may be embedded as one or more devices, including various components described below. In various exemplary embodiments, the system 700 may be configured to implement one or more aspects described in this application.

[0258] Examples of equipment that may constitute all or part of system 700 include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMD, perspective glasses), projectors (projectors), "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing outputs from video decoders, pre-processors for providing inputs to video encoders, web servers, video servers (e.g., broadcast servers, video-on-demand servers or web servers), static or video cameras, encoding or decoding chips or any other communication devices. The elements of system 700 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs and / or discrete components. For example, in at least one exemplary embodiment, the processing and encoder / decoder elements of system 700 may be distributed across multiple ICs and / or discrete components. In various exemplary embodiments, system 700 may be connected to other similar systems or other electronic devices in a communication manner via, for example, a communication bus or by dedicated input and / or output ports.

[0259] The system 700 may include at least one processor 710 configured to execute instructions loaded therein for implementing, for example, various aspects described in the present application. The processor 710 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 700 may include at least one memory 720 (e.g., a volatile memory device and / or a non-volatile memory device). The system 700 may include a storage device 740, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 740 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0260] The system 700 may include an encoder / decoder module 730, which is configured to, for example, process data to provide encoded / decoded video picture data, and the encoder / decoder module 730 may include its own processor and memory. The encoder / decoder module 730 may represent a module (or modules) that may be included in a device to perform encoding and / or decoding functions. As known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 730 may be implemented as a separate element of the system 700, or may be incorporated into the processor 710 as a combination of hardware and software known to those skilled in the art.

[0261] Program code to be loaded into the processor 710 or the encoder / decoder 730 to perform various aspects described in the present application may be stored in the storage device 740 and subsequently loaded into the memory 720 to be executed by the processor 710. According to various exemplary embodiments, during the execution of the processes described in the present application, one or more of the processor 710, the memory 720, the storage device 740, and the encoder / decoder module 730 may store one or more of various items. Such stored items may include, but are not limited to, video picture data, information data for encoding / decoding video picture data, bit streams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0262] In several exemplary embodiments, memory internal to the processor 710 and / or encoder / decoder module 730 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0263] However, in other exemplary embodiments, memory external to the processing device (e.g., the processing device may be the processor 710 or the encoder / decoder module 730) is used for one or more of these functions. The external memory may be a memory 720 and / or a storage device 740, such as a dynamic volatile memory and / or a non-volatile flash memory. In several exemplary embodiments, an external non-volatile flash memory is used to store an operating system for the television. In at least one exemplary embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, such as for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), AVC, HEVC, EVC, VVC, AV1, etc.

[0264] As indicated in block 790, input to the elements of system 700 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, for example, by a broadcast device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, (v) a bus such as a CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus when the present invention is implemented in the automotive field.

[0265] In various exemplary embodiments, the input device of block 790 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting the frequency band again to a narrower frequency band to select a signal band that may be referred to as a channel in certain exemplary embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various exemplary embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0266] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0267] Various exemplary embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0268] Adding components may include inserting components between existing components, such as, for example, inserting an amplifier and an analog-to-digital converter.In various exemplary embodiments, the RF portion may include an antenna.

[0269] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 700 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 710 when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 710 when necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 710 and the encoder / decoder 730, which operate in conjunction with memory and storage elements to process the data streams for presentation on output devices when necessary.

[0270] The various elements of system 700 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 790, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0271] The system 700 may include a communication interface 750 that enables communication with other devices via a communication channel 751. The communication interface 750 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 751. The communication interface 750 may include, but is not limited to, a modem or a network card, and the communication channel 751 may be implemented, for example, within a wired and / or wireless medium.

[0272] In various exemplary embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 700. The Wi-Fi signals of these exemplary embodiments may be received via a communication channel 751 suitable for Wi-Fi communications and a communication interface 750. The communication channel 751 of these exemplary embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0273] Other exemplary embodiments may provide streamed data to system 700 using a set-top box that delivers the data through an HDMI connection to input block 790 .

[0274] Still other exemplary embodiments may provide streaming data to the system 700 using an RF connection to input block 790 .

[0275] Streamed data may be used as a means of signaling information used by the system 700. The signaling information may include the bitstream B and / or information such as 7a the number of pixels of a video picture and / or any encoding / decoding setting parameters.

[0276] It should be appreciated that signaling can be implemented in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0277] The system 700 may provide output signals to various output devices, including a display 761, speakers 771, and other peripherals 781. In various examples of the exemplary embodiments, the other peripherals 781 may include one or more of a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 700.

[0278] In various exemplary embodiments, control signals may be communicated between the system 700 and the display 761, speakers 771, or other peripheral devices 781 using signaling such as AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention.

[0279] Output devices may be communicatively coupled to system 700 through respective interfaces 760 , 770 , and 780 via dedicated connections.

[0280] Alternatively, output devices may be connected to system 700 via communication interface 750 using communication channel 751. Display 761 and speaker 771 may be integrated with other components of system 700 in a single unit in an electronic device such as, for example, a television.

[0281] In various exemplary embodiments, the display interface 760 may include a display driver, such as, for example, a timing controller (T Con) chip.

[0282] For example, if the RF portion of input 790 is part of a separate set-top box, then display 761 and speaker 771 may optionally be separate from one or more of the other components. In various exemplary embodiments where display 761 and speaker 771 may be external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0283] exist Figure 1-19 In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0284] Some examples are described about block diagrams and / or operational flow charts. Each square block represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the square block may not occur in the order indicated. For example, depending on the functions involved, two square blocks shown in succession can actually be executed substantially concurrently, or sometimes these square blocks can be executed in reverse order.

[0285] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0286] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device.

[0287] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored on a computer-readable storage medium. The computer-readable storage medium can take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Considering the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein can be considered as a non-transitory storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. It should be appreciated that although more specific examples of computer-readable storage media to which the exemplary embodiment can be applied are provided below, as those of ordinary skill in the art will readily appreciate, it is merely illustrative and not exhaustive: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0288] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0289] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0290] Device can be realized in suitable hardware, software and firmware for example.The example of this device comprises personal computer, laptop computer, smart phone, tablet computer, digital multimedia set-top box, digital television receiver, personal video recording system, connected household appliances, head-mounted display device (HMD, perspective glasses), projector (projector), "cave" (system including multiple displays), server, video encoder, video decoder, post-processor for processing the output from video decoder, pre-processor for providing input to video encoder, web server, set-top box, and any other equipment for processing video picture, or other communication equipment.It should be clear that equipment can be mobile and even installed in a mobile vehicle.

[0291] Computer software may be implemented by processor 710 or by hardware, or by a combination of hardware and software. As a non-limiting example, exemplary embodiments may also be implemented by one or more integrated circuits. Memory 720 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). Processor 710 may be of any type suitable for the technical environment and may encompass one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0292] As will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described exemplary embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0293] The terms used herein are only used to describe the purpose of specific exemplary embodiments and are not intended to be limited. As used herein, the singular forms "one", "a kind of" and "the" may also be intended to include plural forms unless the context clearly indicates otherwise. It will be further understood that when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "responsive to" or "connected to" another element or "associated with another element", it may directly respond or be connected to or associated with another element, or there may be an intermediate element. In contrast, when an element is referred to as "directly responsive to" or "directly connected to" another element or "directly associated with another element", there is no intermediate element.

[0294] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0295] Various numerical values ​​may be used in the present application. Specific values ​​may be used for example purposes and the described aspects are not limited to these specific values.

[0296] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of this application, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0297] References to "an exemplary embodiment" or "exemplary embodiments" or "one implementation" or "implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an exemplary embodiment / implementation) is included in at least one exemplary embodiment / implementation. Thus, the appearances of the phrases "in an exemplary embodiment" or "in an exemplary embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places in this application are not necessarily all referring to the same exemplary embodiment.

[0298] Similarly, references herein to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing in various places in this application do not necessarily all refer to the same exemplary embodiment / example / implementation, nor are separate or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive of other exemplary embodiments / examples / implementations.

[0299] Reference numerals appearing in the claims are for illustration purposes only and have no limiting effect on the scope of the claims.The present exemplary embodiments / examples and variations may be employed in any combination or sub-combination although not explicitly described.

[0300] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0301] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0302] Various embodiments relate to decoding. As used in the present application, "decoding" may encompass, for example, all or part of a process performed on a received video picture (which may include a received bitstream encoding one or more video pictures) to produce a final output suitable for display or further processing in a reconstructed video domain. In various exemplary embodiments, such a process includes one or more of the processes typically performed by a decoder. In various exemplary embodiments, for example, such a process also includes or optionally includes a process performed by a decoder of the various embodiments described in the present application.

[0303] As a further example, in one exemplary embodiment, "decoding" may refer only to dequantization, in one exemplary embodiment, "decoding" may refer to entropy decoding, in another exemplary embodiment, "decoding" may refer only to differential decoding, and in another exemplary embodiment, "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. Whether the phrase "decoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader decoding process will be clear, and is believed to be well understood by those skilled in the art, based on the context of the particular description.

[0304] Various embodiments all relate to encoding. In a manner similar to the above discussion about "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input video picture to produce an output bitstream. In various exemplary embodiments, such a process includes one or more of the processes typically performed by an encoder. In various exemplary embodiments, such a process also includes or optionally includes a process performed by an encoder of the various embodiments described in this application.

[0305] As a further example, in one exemplary embodiment, "encoding" may refer only to quantization, in one exemplary embodiment, "encoding" may refer only to entropy coding, in another exemplary embodiment, "encoding" may refer only to differential coding, and in another exemplary embodiment, "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Based on the context of the particular description, whether the phrase "encoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader encoding process will be clear and is believed to be well understood by those skilled in the art.

[0306] Additionally, the present application may refer to "obtaining" various information. Obtaining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory, processing information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0307] Furthermore, this application may refer to "receiving" various pieces of information. Receiving information may include, for example, one or more of: accessing the information or receiving the information from a communication network.

[0308] Moreover, as used herein, the word "signal" especially refers to indicating something to a corresponding decoder, etc. For example, in some exemplary embodiments, the encoder signals specific information, such as codec parameters or encoded video picture data. In this way, in exemplary embodiments, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicitly signal) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various exemplary embodiments. It should be recognized that signaling can be completed in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" can also be used as a noun in this article.

[0309] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present application contemplates these and other embodiments.

Claims

1. A method of encoding a video picture into a bitstream of coded video picture data, the method comprising temporally predicting a block of the video picture by obtaining a temporal prediction block based on at least one reference block of at least one reference picture pointed to by at least one motion vector associated with the block of the video picture, wherein if the temporal prediction of the block of the video picture is based on a single reference block of the reference picture, the method further comprises: - filling (402) a filling zone (SZ) of the reference block based on a motion vector derived from a nearest sub-block of the video picture block within the video picture; and - determining a size (M) of said filling zone (SZ) based on said motion vector.

2. A method for decoding a video picture from a bitstream of coded video picture data, the method comprising temporally predicting a video picture block by obtaining a temporal prediction block based on at least one reference block of at least one reference picture pointed to by at least one motion vector associated with the video picture block, wherein if the temporal prediction of the video picture block is based on a single reference block of a reference picture, the method further comprises: - filling (402) a filling zone (SZ) of the reference block based on a motion vector derived from a nearest sub-block of the video picture block within the video picture; and - determining a size (M) of said filling zone (SZ) based on said motion vector. The method according to claim 1 , wherein the filling area is a rectangular area.

4. Method according to claim 3, wherein the size (M) of the padding zone (SZ) is set to at least equal to 4 if the derived motion vector points to a position inside the reference picture.

5. The method of claim 4, wherein the size (M) of the padding zone (SZ) is set to be at least equal to 4 if the reference picture pointed to by the motion vector is an inter-prediction picture.

6. Method according to claim 3, wherein if the temporal or depth level of the video picture is at least equal to 1, the size (M) of the padding zone (SZ) is set to be at least equal to 4.

7. The method according to claim 3, wherein the size (M) of the padding zone (SZ) is set to be at least equal to 8 if the temporal or depth layer of the video picture is higher than or equal to a predetermined value.

8. The method of claim 3, wherein if the temporal or depth layer of the video picture is higher than or equal to a predetermined value, if the reference picture pointed to by the motion vector is an inter-frame prediction picture, and if the temporal or depth layer of the video picture is higher than or equal to a predetermined value, the size (M) of the padding area (SZ) is further increased by 4.

9. The method of any one of claims 1 to 7, wherein if the temporal prediction of the video picture block is based on a single reference block of a reference picture and if the single reference block is located at a corner of the reference picture, the method further comprises: - obtaining (601) a first motion vector (mv) associated with a sub-block of the reference block located at a corner of the video picture L0 ) and the second motion vector (mv L0 ); - The first motion vector (mv L0 ) Calculate (602) a first size of the fill zone (SZ) And the second motion vector (mv L1 ) Calculate (603) the second size of the fill area - selecting (604) the first size or the second size as the size of the filling zone (SZ) based on the sign of the first size and the sign of the second size; and - filling (606) said filling zone (SZ) based on said first motion vector or said second motion vector according to said selected size.

10. The method as claimed in claim 8, wherein the selected dimensions result in a maximum surface of the filling zone (SZ).

11. The method of claim 8 or 9, wherein the first dimension is selected if the sign of the first dimension is strictly positive and at least one of the second dimensions is not strictly positive.

12. A method as claimed in any one of claims 8 to 10, wherein the second dimension is selected if the sign of the second dimension is strictly positive and at least one of the first dimensions is not strictly positive.

13. A bit stream formatted to include encoded video picture data and information data obtained from the method of any one of claims 1, 3 to 11.

14. An apparatus comprising means for carrying out one of the methods as claimed in any one of claims 1 to 11.

15. A computer program product comprising instructions which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 11.

16. A non-transitory storage medium carrying instructions of a program code for executing the method of any one of claims 1 to 11.

Images